stemmer

README

Package stemmer is a set of Snowball language stemmers, implemented as transform.Transformer (see golang.org/x/text/). Implementing transform.Transformer allows use with the x/text ecosystem.

(Stemming is the process of trimming words down to their roots, usually for searching purposes.)

My narrower motivation is to offer stemming in this Unicode text segmenter. To use a stemmer in that package, you will call the Transform method on a Segmenter or Scanner, along the lines of:

import "github.com/clipperhouse/stemmer"

text := []byte("I've got my skis and I am heading to the mountains.")
segmenter := words.NewSegmenter(text)
segmenter.Transform(stemmer.English)

// do stuff with segmenter

This stemmer package is a thin layer over this Snowball package, thanks to Bleve.

Supported languages

Arabic, Danish, Dutch, English, Finnish, French, German, Hungarian, Irish, Italian, Norwegian, Porter, Portuguese, Romanian, Russian, Spanish, Swedish, Tamil, Turkish

Documentation

Overview

Package stemmer set of Snowball language stemmers, implemented as transform.Transformer (from golang.org/x/text/).

Index

• Variables
• type StemFunc
• type Transformer
  • func NewTransformer(stem StemFunc) *Transformer
  • func (t *Transformer) Reset()
  • func (t *Transformer) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error)

Variables

var Arabic transform.Transformer = NewTransformer(arabic.Stem)

Arabic is a Snowball stemmer for Arabic

var Danish transform.Transformer = NewTransformer(danish.Stem)

Danish is a Snowball stemmer for Danish

var Dutch transform.Transformer = NewTransformer(dutch.Stem)

Dutch is a Snowball stemmer for Dutch

var English transform.Transformer = NewTransformer(english.Stem)

English is a Snowball stemmer for English

var Finnish transform.Transformer = NewTransformer(finnish.Stem)

Finnish is a Snowball stemmer for Finnish

var French transform.Transformer = NewTransformer(french.Stem)

French is a Snowball stemmer for French

var German transform.Transformer = NewTransformer(german.Stem)

German is a Snowball stemmer for German

var Hungarian transform.Transformer = NewTransformer(hungarian.Stem)

Hungarian is a Snowball stemmer for Hungarian

var Irish transform.Transformer = NewTransformer(irish.Stem)

Irish is a Snowball stemmer for Irish

var Italian transform.Transformer = NewTransformer(italian.Stem)

Italian is a Snowball stemmer for Italian

var Norwegian transform.Transformer = NewTransformer(norwegian.Stem)

Norwegian is a Snowball stemmer for Norwegian

var Porter transform.Transformer = NewTransformer(porter.Stem)

Porter is a Porter stemmer

var Portuguese transform.Transformer = NewTransformer(portuguese.Stem)

Portuguese is a Snowball stemmer for Portuguese

var Romanian transform.Transformer = NewTransformer(romanian.Stem)

Romanian is a Snowball stemmer for Romanian

var Russian transform.Transformer = NewTransformer(russian.Stem)

Russian is a Snowball stemmer for Russian

var Spanish transform.Transformer = NewTransformer(spanish.Stem)

Spanish is a Snowball stemmer for Spanish

var Swedish transform.Transformer = NewTransformer(swedish.Stem)

Swedish is a Snowball stemmer for Swedish

var Tamil transform.Transformer = NewTransformer(tamil.Stem)

Tamil is a Snowball stemmer for Tamil

var Turkish transform.Transformer = NewTransformer(turkish.Stem)

Turkish is a Snowball stemmer for Turkish

Types

type StemFunc

type StemFunc func(*snowballstem.Env) bool

type Transformer

type Transformer struct {
	// contains filtered or unexported fields
}

func NewTransformer

func NewTransformer(stem StemFunc) *Transformer

func (*Transformer) Reset

func (t *Transformer) Reset()

func (*Transformer) Transform

func (t *Transformer) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error)