rmsnorm

Documentation

Overview

Package rmsnorm implements the Root Mean Square Layer Normalization method.

Reference: "Root Mean Square Layer Normalization" by Biao Zhang and Rico Sennrich (2019). (https://arxiv.org/pdf/1910.07467.pdf)

Index

• type Model
  • func New(size int) *Model
  • func (m *Model) Forward(xs ...ag.Node) []ag.Node

Types

type Model

type Model struct {
	nn.BaseModel
	W nn.Param `spago:"type:weights"`
	B nn.Param `spago:"type:biases"`
}

Model contains the serializable parameters.

func New

func New(size int) *Model

New returns a new model with parameters initialized to zeros.

func (*Model) Forward

func (m *Model) Forward(xs ...ag.Node) []ag.Node

Forward performs the forward step for each input node and returns the result.