adanorm

Documentation

Overview

Package adanorm implements the Adaptive Normalization (AdaNorm) method.

Reference: "Understanding and Improving Layer Normalization" by Jingjing Xu, Xu Sun, Zhiyuan Zhang,Guangxiang Zhao, Junyang Lin (2019). (https://papers.nips.cc/paper/8689-understanding-and-improving-layer-normalization.pdf)

Index

• type Model
  • func New[T float.DType](scale float64) *Model
  • func (m *Model) Forward(xs ...mat.Tensor) []mat.Tensor
  • func (m *Model) Mean(xs []mat.Tensor) []mat.Tensor
  • func (m *Model) StdDev(meanVectors []mat.Tensor, xs []mat.Tensor) []mat.Tensor

Types

type Model

type Model struct {
	nn.Module
	Scale *nn.Buffer
}

Model contains the scaling factor.

func New

func New[T float.DType](scale float64) *Model

New returns a new model.

func (*Model) Forward

func (m *Model) Forward(xs ...mat.Tensor) []mat.Tensor

Forward performs the forward step for each input node and returns the result.

func (*Model) Mean

func (m *Model) Mean(xs []mat.Tensor) []mat.Tensor

Mean computes the mean of the input.

func (*Model) StdDev

func (m *Model) StdDev(meanVectors []mat.Tensor, xs []mat.Tensor) []mat.Tensor

StdDev computes the standard deviation of the input.