tag

Documentation

Overview

Package tag implements functions for tagging parts of speech.

Index

• type AveragedPerceptron
  • func NewAveragedPerceptron(weights map[string]map[string]float64, tags map[string]string, ...) *AveragedPerceptron
• type PerceptronTagger
  • func NewPerceptronTagger() *PerceptronTagger
  • func (pt *PerceptronTagger) Classes() []string
  • func (pt *PerceptronTagger) Tag(words []string) []Token
  • func (pt *PerceptronTagger) TagMap() map[string]string
  • func (pt *PerceptronTagger) Weights() map[string]map[string]float64
• type Token
• type TupleSlice
  • func ReadTagged(text, sep string) TupleSlice
  • func (t TupleSlice) Len() int
  • func (t TupleSlice) Swap(i, j int)

Examples

• ReadTagged

Types

type AveragedPerceptron

type AveragedPerceptron struct {
	// contains filtered or unexported fields
}

AveragedPerceptron is a Averaged Perceptron classifier.

func NewAveragedPerceptron

func NewAveragedPerceptron(weights map[string]map[string]float64,
	tags map[string]string, classes []string) *AveragedPerceptron

NewAveragedPerceptron creates a new AveragedPerceptron model.

type PerceptronTagger

type PerceptronTagger struct {
	// contains filtered or unexported fields
}

PerceptronTagger is a port of Textblob's "fast and accurate" POS tagger. See https://github.com/sloria/textblob-aptagger for details.

func NewPerceptronTagger

func NewPerceptronTagger() *PerceptronTagger

NewPerceptronTagger creates a new PerceptronTagger and loads the built-in AveragedPerceptron model.

func (*PerceptronTagger) Classes

func (pt *PerceptronTagger) Classes() []string

Classes returns the model's classes in the form

["EX", "NNPS", "WP$", ...]

func (*PerceptronTagger) Tag

func (pt *PerceptronTagger) Tag(words []string) []Token

Tag takes a slice of words and returns a slice of tagged tokens.

func (*PerceptronTagger) TagMap

func (pt *PerceptronTagger) TagMap() map[string]string

TagMap returns the model's classes in the form

	{
	  "four": "CD",

	{
	  "four": "CD",
     "facilities": "NNS",
     ...
   }

func (*PerceptronTagger) Weights

func (pt *PerceptronTagger) Weights() map[string]map[string]float64

 Wts returns the model's weights in the form

    "VB": -0.695,
    ...
   }
   ...
}

type Token

type Token struct {
	Text string
	Tag  string
}

Token represents a tagged section of text.

type TupleSlice

type TupleSlice [][][]string

TupleSlice is a slice of tuples in the form (words, tags).

func ReadTagged

func ReadTagged(text, sep string) TupleSlice

ReadTagged converts pre-tagged input into a TupleSlice suitable for training.

Example

package main

import (
	"fmt"
)

var wsj = "Pierre|NNP Vinken|NNP ,|, 61|CD years|NNS old|JJ ,|, will|MD " +
	"join|VB the|DT board|NN as|IN a|DT nonexecutive|JJ director|NN " +
	"Nov.|NNP 29|CD .|.\nMr.|NNP Vinken|NNP is|VBZ chairman|NN of|IN " +
	"Elsevier|NNP N.V.|NNP ,|, the|DT Dutch|NNP publishing|VBG " +
	"group|NN .|. Rudolph|NNP Agnew|NNP ,|, 55|CD years|NNS old|JJ " +
	"and|CC former|JJ chairman|NN of|IN Consolidated|NNP Gold|NNP " +
	"Fields|NNP PLC|NNP ,|, was|VBD named|VBN a|DT nonexecutive|JJ " +
	"director|NN of|IN this|DT British|JJ industrial|JJ conglomerate|NN " +
	".|.\nA|DT form|NN of|IN asbestos|NN once|RB used|VBN to|TO make|VB " +
	"Kent|NNP cigarette|NN filters|NNS has|VBZ caused|VBN a|DT high|JJ " +
	"percentage|NN of|IN cancer|NN deaths|NNS among|IN a|DT group|NN " +
	"of|IN workers|NNS exposed|VBN to|TO it|PRP more|RBR than|IN " +
	"30|CD years|NNS ago|IN ,|, researchers|NNS reported|VBD .|."

func main() {
	tagged := "Pierre|NNP Vinken|NNP ,|, 61|CD years|NNS"
	fmt.Println(ReadTagged(tagged, "|"))
}

Output:

[[[Pierre Vinken , 61 years] [NNP NNP , CD NNS]]]

func (TupleSlice) Len

func (t TupleSlice) Len() int

Len returns the length of a Tuple.

func (TupleSlice) Swap

func (t TupleSlice) Swap(i, j int)

Swap switches the ith and jth elements in a Tuple.