lstm

Documentation

Index

• type Model
  • func New[T float.DType](in, out int) *Model
  • func (m *Model) Forward(xs ...mat.Tensor) []mat.Tensor
  • func (m *Model) Init(rndGen *rand.LockedRand) *Model
  • func (m *Model) Next(state *State, x mat.Tensor) (s *State)
  • func (m *Model) WithRefinedGates(value bool) *Model
• type Option
• type State

Types

type Model

type Model struct {
	nn.Module
	UseRefinedGates bool

	// Input gate
	WIn    *nn.Param
	WInRec *nn.Param
	BIn    *nn.Param

	// Output gate
	WOut    *nn.Param
	WOutRec *nn.Param
	BOut    *nn.Param

	// Forget gate
	WFor    *nn.Param
	WForRec *nn.Param
	BFor    *nn.Param

	// Candiate gate
	WCand    *nn.Param
	WCandRec *nn.Param
	BCand    *nn.Param
}

Model contains the serializable parameters.

func New

func New[T float.DType](in, out int) *Model

New returns a new model with parameters initialized to zeros.

func (*Model) Forward

func (m *Model) Forward(xs ...mat.Tensor) []mat.Tensor

Forward performs the forward step for each input node and returns the result.

func (*Model) Init

func (m *Model) Init(rndGen *rand.LockedRand) *Model

Init initializes the parameters using Xavier uniform randomization. It follows the LSTM bias hack setting the Forget gate to 1 (http://proceedings.mlr.press/v37/jozefowicz15.pdf).

func (*Model) Next

func (m *Model) Next(state *State, x mat.Tensor) (s *State)

Next performs a single forward step, producing a new state.

It computes the results with the following equations: inG = sigmoid(wIn (dot) x + bIn + wInRec (dot) yPrev) outG = sigmoid(wOut (dot) x + bOut + wOutRec (dot) yPrev) forG = sigmoid(wFor (dot) x + bFor + wForRec (dot) yPrev) cand = f(wCand (dot) x + bC + wCandRec (dot) yPrev) cell = inG * cand + forG * cellPrev y = outG * f(cell)

func (*Model) WithRefinedGates

func (m *Model) WithRefinedGates(value bool) *Model

WithRefinedGates sets whether to use refined gates. Refined Gate: A Simple and Effective Gating Mechanism for Recurrent Units (https://arxiv.org/pdf/2002.11338.pdf)

Refined gates setting requires input size and output size be the same.

type Option

type Option func(*Model)

Option allows to configure a new Model with your specific needs.

type State

type State struct {
	InG  mat.Tensor
	OutG mat.Tensor
	ForG mat.Tensor
	Cand mat.Tensor
	Cell mat.Tensor
	Y    mat.Tensor
}

State represent a state of the LSTM recurrent network.