tokenizer

package
v0.5.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2024 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewCharGroupTokenizer 

func NewCharGroupTokenizer(options interface{}) (analysis.Tokenizer, error)

func NewCharacterTokenizer 

func NewCharacterTokenizer(options interface{}) (analysis.Tokenizer, error)

func NewEdgeNgramTokenizer 

func NewEdgeNgramTokenizer(options interface{}) (analysis.Tokenizer, error)

func NewExceptionTokenizer 

func NewExceptionTokenizer(options interface{}) (analysis.Tokenizer, error)

func NewLowerCaseTokenizer 

func NewLowerCaseTokenizer() (analysis.Tokenizer, error)

func NewNgramTokenizer 

func NewNgramTokenizer(options interface{}) (analysis.Tokenizer, error)

func NewPathHierarchyTokenizer 

func NewPathHierarchyTokenizer(options interface{}) (analysis.Tokenizer, error)

func NewRegexpTokenizer 

func NewRegexpTokenizer(options interface{}) (analysis.Tokenizer, error)

Types

This section is empty.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.