tag

Documentation

Overview

Package tag implements functions for tagging parts of speech.

Index

• type AveragedPerceptron
  • func NewAveragedPerceptron(weights map[string]map[string]float64, tags map[string]string, ...) *AveragedPerceptron
• type PerceptronTagger
  • func NewPerceptronTagger() *PerceptronTagger
  • func NewTrainedPerceptronTagger(model *AveragedPerceptron) *PerceptronTagger
  • func (pt *PerceptronTagger) Classes() []string
  • func (pt *PerceptronTagger) Tag(words []string) []Token
  • func (pt *PerceptronTagger) TagMap() map[string]string
  • func (pt *PerceptronTagger) Train(sentences TupleSlice, iterations int)
  • func (pt *PerceptronTagger) Weights() map[string]map[string]float64
• type Token
• type TupleSlice
  • func ReadTagged(text, sep string) TupleSlice
  • func (t TupleSlice) Len() int
  • func (t TupleSlice) Swap(i, j int)

Examples

• ReadTagged

Types

type AveragedPerceptron

type AveragedPerceptron struct {
	// contains filtered or unexported fields
}

AveragedPerceptron is a Averaged Perceptron classifier.

func NewAveragedPerceptron

func NewAveragedPerceptron(weights map[string]map[string]float64,
	tags map[string]string, classes []string) *AveragedPerceptron

NewAveragedPerceptron creates a new AveragedPerceptron model.

type PerceptronTagger

type PerceptronTagger struct {
	// contains filtered or unexported fields
}

PerceptronTagger is a port of Textblob's "fast and accurate" POS tagger. See https://github.com/sloria/textblob-aptagger for details.

func NewPerceptronTagger

func NewPerceptronTagger() *PerceptronTagger

NewPerceptronTagger creates a new PerceptronTagger and loads the built-in AveragedPerceptron model.

func NewTrainedPerceptronTagger

func NewTrainedPerceptronTagger(model *AveragedPerceptron) *PerceptronTagger

NewTrainedPerceptronTagger creates a new PerceptronTagger using the given model.

func (*PerceptronTagger) Classes

func (pt *PerceptronTagger) Classes() []string

Classes returns the model's classes in the form

["EX", "NNPS", "WP$", ...]

func (*PerceptronTagger) Tag

func (pt *PerceptronTagger) Tag(words []string) []Token

Tag takes a slice of words and returns a slice of tagged tokens.

func (*PerceptronTagger) TagMap

func (pt *PerceptronTagger) TagMap() map[string]string

TagMap returns the model's classes in the form

{
  "four": "CD",
  "facilities": "NNS",
  ...
}

func (*PerceptronTagger) Train

func (pt *PerceptronTagger) Train(sentences TupleSlice, iterations int)

Train an Averaged Perceptron model based on sentences.

func (*PerceptronTagger) Weights

func (pt *PerceptronTagger) Weights() map[string]map[string]float64

Weights returns the model's weights in the form

{
  "i-1 suffix ity": {
    "MD": -0.816,
    "VB": -0.695,
    ...
   }
   ...
}

type Token

type Token struct {
	Text string
	Tag  string
}

Token represents a tagged section of text.

type TupleSlice

type TupleSlice [][][]string

TupleSlice is a slice of tuples in the form (words, tags).

func ReadTagged

func ReadTagged(text, sep string) TupleSlice

ReadTagged converts pre-tagged input into a TupleSlice suitable for training.

Example

tagged := "Pierre|NNP Vinken|NNP ,|, 61|CD years|NNS"
fmt.Println(ReadTagged(tagged, "|"))

Output:

[[[Pierre Vinken , 61 years] [NNP NNP , CD NNS]]]

func (TupleSlice) Len

func (t TupleSlice) Len() int

Len returns the length of a Tuple.

func (TupleSlice) Swap

func (t TupleSlice) Swap(i, j int)

Swap switches the ith and jth elements in a Tuple.