unicodestrip

Documentation

Overview

Package unicodestrip provides a Bleve keyword filter which decomposes unicode strings.

Index

• Constants
• func Constructor(config map[string]interface{}, cache *registry.Cache) (analysis.TokenFilter, error)
• type StripUnicodeFilter
  • func New() (*StripUnicodeFilter, error)
  • func (s *StripUnicodeFilter) Filter(input analysis.TokenStream) analysis.TokenStream

Constants

const Name = "strip_unicode"

Functions

func Constructor

func Constructor(config map[string]interface{}, cache *registry.Cache) (analysis.TokenFilter, error)

Constructor provides a constructor for Bleve.

Types

type StripUnicodeFilter

type StripUnicodeFilter struct {
}

StripUnicodeFilter is a Bleve keyword filter which decomposes unicode strings into their normalized form and strips away non-spacing marks. Effectively, this strips away diacritic marks so that searches may be done without entering them, e.g. "Télépopmusik" is indexed as "Telepopmusik".

func New

func New() (*StripUnicodeFilter, error)

New returns a new instance of StripUnicodeFilter.

func (*StripUnicodeFilter) Filter

func (s *StripUnicodeFilter) Filter(input analysis.TokenStream) analysis.TokenStream

Filter removes non-spacing marks from text in a token stream.