lstm

Documentation

Index

• type Model
  • func New[T float.DType](in, out int) *Model
  • func (m *Model) Forward(xs ...ag.Node) []ag.Node
  • func (m *Model) Next(state *State, x ag.Node) (s *State)
  • func (m *Model) WithRefinedGates(value bool) *Model
• type Option
• type State

Types

type Model

type Model struct {
	nn.Module
	UseRefinedGates bool

	// Input gate
	WIn    nn.Param `spago:"type:weights"`
	WInRec nn.Param `spago:"type:weights"`
	BIn    nn.Param `spago:"type:biases"`

	// Output gate
	WOut    nn.Param `spago:"type:weights"`
	WOutRec nn.Param `spago:"type:weights"`
	BOut    nn.Param `spago:"type:biases"`

	// Forget gate
	WFor    nn.Param `spago:"type:weights"`
	WForRec nn.Param `spago:"type:weights"`
	BFor    nn.Param `spago:"type:biases"`

	// Candiate gate
	WCand    nn.Param `spago:"type:weights"`
	WCandRec nn.Param `spago:"type:weights"`
	BCand    nn.Param `spago:"type:biases"`
}

Model contains the serializable parameters.

func New

func New[T float.DType](in, out int) *Model

New returns a new model with parameters initialized to zeros.

func (*Model) Forward

func (m *Model) Forward(xs ...ag.Node) []ag.Node

Forward performs the forward step for each input node and returns the result.

func (*Model) Next

func (m *Model) Next(state *State, x ag.Node) (s *State)

Next performs a single forward step, producing a new state.

It computes the results with the following equations: inG = sigmoid(wIn (dot) x + bIn + wInRec (dot) yPrev) outG = sigmoid(wOut (dot) x + bOut + wOutRec (dot) yPrev) forG = sigmoid(wFor (dot) x + bFor + wForRec (dot) yPrev) cand = f(wCand (dot) x + bC + wCandRec (dot) yPrev) cell = inG * cand + forG * cellPrev y = outG * f(cell)

func (*Model) WithRefinedGates

func (m *Model) WithRefinedGates(value bool) *Model

WithRefinedGates sets whether to use refined gates. Refined Gate: A Simple and Effective Gating Mechanism for Recurrent Units (https://arxiv.org/pdf/2002.11338.pdf)

Refined gates setting requires input size and output size be the same.

type Option

type Option func(*Model)

Option allows to configure a new Model with your specific needs.

type State

type State struct {
	InG  ag.Node
	OutG ag.Node
	ForG ag.Node
	Cand ag.Node
	Cell ag.Node
	Y    ag.Node
}

State represent a state of the LSTM recurrent network.