stop

Documentation

Overview

Package stop implements a TokenFilter removing tokens found in a TokenMap.

It constructor takes the following arguments:

"stop_token_map" (string): the name of the token map identifying tokens to remove.

Index

• Constants
• func StopTokensFilterConstructor(config map[string]interface{}, cache *registry.Cache) (analysis.TokenFilter, error)
• type StopTokensFilter
  • func NewStopTokensFilter(stopTokens analysis.TokenMap) *StopTokensFilter
  • func (f *StopTokensFilter) Filter(input analysis.TokenStream) analysis.TokenStream

Constants

const Name = "stop_tokens"

Functions

func StopTokensFilterConstructor

func StopTokensFilterConstructor(config map[string]interface{}, cache *registry.Cache) (analysis.TokenFilter, error)

Types

type StopTokensFilter

type StopTokensFilter struct {
	// contains filtered or unexported fields
}

func NewStopTokensFilter

func NewStopTokensFilter(stopTokens analysis.TokenMap) *StopTokensFilter

func (*StopTokensFilter) Filter

func (f *StopTokensFilter) Filter(input analysis.TokenStream) analysis.TokenStream