lstm

README

This lib is the engine of the LSTM

Documentation

Overview

Package lstm is a basic implementation of this kind of recural network.

WARNING: This is only a skeleton of a new API for a LSTM implementation with Gorgonia. Nothing is implemented at the present time and the API may change. There should be a working version with a less complete API working on the master branch of the project

Index

• type Model
  • func NewModel(definitions string) (*Model, error)
  • func (m Model) MarshalBinary() ([]byte, error)
  • func (m *Model) Predict(ctx context.Context, dataSet datasetter.ReadWriter) error
  • func (m *Model) Train(ctx context.Context, dset datasetter.FullTrainer, solver G.Solver, ...) (<-chan TrainingInfos, <-chan error)
  • func (m *Model) UnmarshalBinary(data []byte) error
• type TrainingInfos

Types

type Model

type Model struct {
	// contains filtered or unexported fields
}

Model holds the structure of the LSTM model

func NewModel

func NewModel(definitions string) (*Model, error)

NewModel creates a new model from a string. the strings represent a forward pass as well as the definitions of the weights expressed in unicode. The hidden vector must be called `h` and the memory cell `c` The input vector and output vector must be called `xₜ` and `yₜ` ex:

definitions := `
    iₜ=(Wᵢ·xₜ+Uᵢ·hₜ₋₁+Bᵢ)
    fₜ=σ(Wf·xₜ+Uf·hₜ₋₁+Bf)
    oₜ=σ(Wₒ·xₜ+Uₒ·hₜ₋₁+Bₒ)
    ĉₜ=tanh(Wc·xₜ+Uc·hₜ₋₁+Bc)
    cₜ=fₜ*cₜ₋₁+iₜ*ĉₜ
    hₜ=oₜ*tanh(cₜ)
    y=Wy·hₜ+By
    xₜ∈R⁶⁵
    fₜ∈R¹⁰⁰
    iₜ∈R¹⁰⁰
    oₜ∈R¹⁰⁰
    hₜ∈R¹⁰⁰
    cₜ∈R¹⁰⁰
    Wᵢ∈R¹⁰⁰x⁶⁵
    Uᵢ∈R¹⁰⁰x¹⁰⁰
    Bᵢ∈R¹⁰⁰
    Wₒ∈R¹⁰⁰x⁶⁵
    Uₒ∈R¹⁰⁰x¹⁰⁰
    Bₒ∈R¹⁰⁰
    Wf∈R¹⁰⁰x⁶⁵
    Uf∈R¹⁰⁰x¹⁰⁰
    Bf∈R¹⁰⁰
    Wc∈R¹⁰⁰x⁶⁵
    Uc∈R¹⁰⁰x¹⁰⁰
    Bc∈R¹⁰⁰
`

The subscript 'ₜ' will be replaces at runtime by a number corresponding to the step

func (Model) MarshalBinary

func (m Model) MarshalBinary() ([]byte, error)

MarshalBinary for backup. This function saves the equations, the content of the weights matrices and the biais but not the graph structure

func (*Model) Predict

func (m *Model) Predict(ctx context.Context, dataSet datasetter.ReadWriter) error

Predict ...

func (*Model) Train

func (m *Model) Train(ctx context.Context, dset datasetter.FullTrainer, solver G.Solver, pauseChan <-chan struct{}) (<-chan TrainingInfos, <-chan error)

Train the model

func (*Model) UnmarshalBinary

func (m *Model) UnmarshalBinary(data []byte) error

UnmarshalBinary for restore

type TrainingInfos

type TrainingInfos struct {
	Step       int
	Perplexity float32
	Cost       float32
}

TrainingInfos returns info about the current training process