Versions in this module Expand all Collapse all v0 v0.2.0 Apr 8, 2016 v0.1.0 Apr 8, 2016 Changes in this version + const Name + var IdeographRegexp = regexp.MustCompile(`\p{Han}|\p{Hangul}|\p{Hiragana}|\p{Katakana}`) + func RegexpTokenizerConstructor(config map[string]interface{}, cache *registry.Cache) (analysis.Tokenizer, error) + type RegexpTokenizer struct + func NewRegexpTokenizer(r *regexp.Regexp) *RegexpTokenizer + func (rt *RegexpTokenizer) Tokenize(input []byte) analysis.TokenStream
Go to main page