rae

Documentation

Overview

Package rae provides an implementation of the recursive auto-encoder strategy described in "Towards Lossless Encoding of Sentences" by Prato et al., 2019. Unlike the method described in the paper above, here I opted to use the positional encoding introduced byVaswani et al. (2017) for the step encoding.

Index

• type Decoder
  • func NewDefaultDecoder(embeddingSize, outputSize, maxSequenceLength int) *Decoder
  • func (p *Decoder) Forward(x ag.Node) []ag.Node
  • func (p *Decoder) SetSequenceLength(length int)
• type Encoder
  • func NewDefaultEncoder(inputSize, embeddingSize, maxSequenceLength int) *Encoder
  • func (p *Encoder) Forward(xs ...ag.Node) []ag.Node
  • func (p *Encoder) GetRecursions() int
• type State

Types

type Decoder

type Decoder struct {
	nn.BaseModel
	DecodingFNN1 nn.StandardModel // decoding part 1
	DecodingFFN2 nn.StandardModel // decoding part 2
	DescalingFFN nn.StandardModel
	StepEncoder  *pe.SinusoidalPositionalEncoder
	State        State `spago:"scope:processor"`
}

Decoder contains the serializable parameters.

func NewDefaultDecoder

func NewDefaultDecoder(embeddingSize, outputSize, maxSequenceLength int) *Decoder

NewDefaultDecoder returns a new RAE Decoder.

func (*Decoder) Forward

func (p *Decoder) Forward(x ag.Node) []ag.Node

Forward performs the forward step for each input node and returns the result.

func (*Decoder) SetSequenceLength

func (p *Decoder) SetSequenceLength(length int)

SetSequenceLength sets the length of the expected sequence.

type Encoder

type Encoder struct {
	nn.BaseModel
	ScalingFFN  nn.StandardModel
	EncodingFFN nn.StandardModel
	StepEncoder *pe.SinusoidalPositionalEncoder
	Recursions  int `spago:"scope:processor"`
}

Encoder contains the serializable parameters.

func NewDefaultEncoder

func NewDefaultEncoder(inputSize, embeddingSize, maxSequenceLength int) *Encoder

NewDefaultEncoder returns a new RAE Encoder.

func (*Encoder) Forward

func (p *Encoder) Forward(xs ...ag.Node) []ag.Node

Forward performs the forward step for each input node and returns the result.

func (*Encoder) GetRecursions

func (p *Encoder) GetRecursions() int

GetRecursions returns the number of recursions.

type State

type State struct {
	SequenceLength int
	MaxRecursions  int
	Recursions     int
}

State contains information used during decoding recursion.