lamb

Documentation

Index

• type Config
  • func NewConfig(stepSize, beta1, beta2, epsilon, lambda float64) Config
  • func NewDefaultConfig() Config
• type Lamb
  • func New[T float.DType](c Config) *Lamb[T]
  • func (o *Lamb[T]) CalcDelta(state *State, cur mat.Matrix, grads mat.Matrix) mat.Matrix
  • func (o *Lamb[_]) IncExample()
  • func (o *Lamb[T]) OptimizeParams(param *nn.Param) error
• type State

Types

type Config

type Config struct {
	StepSize float64
	Beta1    float64
	Beta2    float64
	Epsilon  float64
	Lambda   float64
}

Config provides configuration settings for Lamb optimizer.

func NewConfig

func NewConfig(stepSize, beta1, beta2, epsilon, lambda float64) Config

NewConfig returns a new Lamb Config.

func NewDefaultConfig

func NewDefaultConfig() Config

NewDefaultConfig returns a new Config with generically reasonable default values.

type Lamb

type Lamb[T float.DType] struct {
	Config
	Alpha    float64
	TimeStep int
}

Lamb implements the Lamb gradient descent optimization method.

func New

func New[T float.DType](c Config) *Lamb[T]

New returns a new Lamb optimizer, initialized according to the given configuration.

func (*Lamb[T]) CalcDelta

func (o *Lamb[T]) CalcDelta(state *State, cur mat.Matrix, grads mat.Matrix) mat.Matrix

CalcDelta returns the difference between the current params and where the method wants it to be.

func (*Lamb[_]) IncExample

func (o *Lamb[_]) IncExample()

IncExample beats the occurrence of a new example.

func (*Lamb[T]) OptimizeParams

func (o *Lamb[T]) OptimizeParams(param *nn.Param) error

type State

type State struct {
	V    mat.Matrix // first moment vector
	M    mat.Matrix // second raw moment vector
	Buf1 mat.Matrix // contains 'grads.ProdScalar(1.0 - beta1)'
	Buf2 mat.Matrix // contains 'grads.Prod(grads).ProdScalar(1.0 - beta2)'
	Buf3 mat.Matrix
}