evolvingembeddings

Documentation

Overview

Package evolvingembeddings provides a word embedding model that evolves by dynamically aggregating contextual embeddings over time during inference. See "Pooled Contextualized Embeddings" by Akbik et al., 2019 https://www.aclweb.org/anthology/papers/N/N19/N19-1078/

Index

• func Close()
• type Config
• type Model
  • func New(config Config) *Model
  • func (m *Model) Aggregate(list []*WordVectorPair)
  • func (m *Model) Close()
  • func (m *Model) Count() int
  • func (m *Model) DropAll() error
  • func (m *Model) Encode(words []string) []ag.Node
• type PoolingType
• type WordVectorPair

Functions

func Close

func Close()

Close closes the DBs underlying all instantiated embeddings models.

Types

type Config

type Config struct {
	// Size of the embedding vectors.
	Size int
	// The operation used to aggregate new embeddings with pre-existing ones.
	PoolingOperation PoolingType
	// The path to DB on the drive
	DBPath string
	// Whether to force the deletion of any existing DB to start with an empty embeddings mam.
	ForceNewDB bool
}

Config provides configuration settings for an Evolving Pooled Contextualized Embeddings Model.

type Model

type Model struct {
	nn.BaseModel
	Config
	Storage       *kvdb.KeyValueDB
	Mu            sync.Mutex
	ZeroEmbedding nn.Param `spago:"type:weights"`
}

Model implements an Evolving Pooled Contextualized Embeddings model.

func New

func New(config Config) *Model

New returns a new embedding Model.

func (*Model) Aggregate

func (m *Model) Aggregate(list []*WordVectorPair)

Aggregate performs a pooling operation over the list of WordVectorPair elements.

func (*Model) Close

func (m *Model) Close()

Close closes the DB underlying the model of the embeddings mam.

func (*Model) Count

func (m *Model) Count() int

Count counts how many embeddings are stored in the DB. It invokes log.Fatal in case of reading errors.

func (*Model) DropAll

func (m *Model) DropAll() error

DropAll drops all the data stored in the DB.

func (*Model) Encode

func (m *Model) Encode(words []string) []ag.Node

Encode returns the embeddings associated with the input words.

type PoolingType

type PoolingType int

PoolingType is the enumeration-like type used to distinguish different types of pooling operations for an Evolving Pooled Contextualized Embeddings Model.

const (
	// Max identifies the maximum pooling operation function.
	Max PoolingType = iota
	// Min identifies the minimum pooling operation function.
	Min
)

type WordVectorPair

type WordVectorPair struct {
	Word   string
	Vector mat.Matrix
}

WordVectorPair associates a Vector to a Word.