tok2vec

Documentation

Index

• Variables
• type Tok2Vec
  • func NewTok2Vec() Tok2Vec
  • func (t2v *Tok2Vec) Tok(vec int) (string, error)
  • func (t2v *Tok2Vec) Toks2Vecs(tokens []string, deduplicate bool) []int
  • func (t2v *Tok2Vec) Toks2VecsFloat64(tokens []string, deduplicate bool) []float64
  • func (t2v *Tok2Vec) Vec(tok string) (int, error)

Variables

var (
	ErrTokenNotFound  = errors.New("token not found")
	ErrVectorNotFound = errors.New("vector not found")
)

Types

type Tok2Vec

type Tok2Vec struct {
	// contains filtered or unexported fields
}

Tok2Vec is designed to convert tokens to vectors for multiple documents so that tokens that exist in multiple documents will be assigned a single vector.

func NewTok2Vec

func NewTok2Vec() Tok2Vec

func (*Tok2Vec) Tok

func (t2v *Tok2Vec) Tok(vec int) (string, error)

func (*Tok2Vec) Toks2Vecs

func (t2v *Tok2Vec) Toks2Vecs(tokens []string, deduplicate bool) []int

func (*Tok2Vec) Toks2VecsFloat64

func (t2v *Tok2Vec) Toks2VecsFloat64(tokens []string, deduplicate bool) []float64

func (*Tok2Vec) Vec

func (t2v *Tok2Vec) Vec(tok string) (int, error)