nn

Documentation

Index

• func Accuracy(scores, labels [][]*Value, trainingParam TrainingParam) (accuracy float64)
• func DrawGraph(values []*Value, filename string)
• func GetRecords(lines [][]string, batchSize int) ([][]*Value, [][]*Value)
• func MakeActivation(op string, f, g func(float64) float64) func(*Value) *Value
• func ReadCSV(filename string) [][]string
• type Layer
  • func MakeLayer(inputSize int, layerParam LayerParam) *Layer
  • func (l *Layer) Fit(input []*Value) []*Value
• type LayerParam
  • func MakeLayerParam(outputSize int, activation func(*Value) *Value) LayerParam
• type NeuralNetwork
  • func MakeNeuralNetwork(inputSize int, layerParams []LayerParam) *NeuralNetwork
  • func (n *NeuralNetwork) Fit(input []*Value) []*Value
  • func (n *NeuralNetwork) Forward(inputs [][]*Value) [][]*Value
  • func (n *NeuralNetwork) Loss(labels, scores [][]*Value, trainingParam TrainingParam) *Value
  • func (n *NeuralNetwork) NextData(learningRate float64)
  • func (n *NeuralNetwork) ResetGrad()
  • func (n *NeuralNetwork) Train(inputs, labels [][]*Value, trainingParam TrainingParam) ([]float64, [][]*Value)
• type Neuron
  • func MakeNeuron(inputSize int) *Neuron
  • func (n *Neuron) Fit(input []*Value) *Value
• type Plotter
  • func (p Plotter) PlotLine(x, y []float64, filename string)
  • func (p Plotter) ScatterPlot(x, y []float64, filename string)
• type TrainingParam
• type Value
  • func MakeValue(data float64) *Value
  • func Relu(value *Value) *Value
  • func Sigmoid(value *Value) *Value
  • func Softmax(value *Value) *Value
  • func Tanh(value *Value) *Value
  • func (value *Value) Add(other *Value) *Value
  • func (value *Value) BackPropagate()
  • func (value *Value) Div(other *Value) *Value
  • func (value *Value) Exp() *Value
  • func (value Value) GetData() float64
  • func (value Value) GetGrad() float64
  • func (value Value) GetOp() string
  • func (value *Value) Log() *Value
  • func (value *Value) Mul(other *Value) *Value
  • func (value *Value) Pow(b float64) *Value
  • func (value *Value) ResetGrad()
  • func (value *Value) SetData(data float64)
  • func (value *Value) Sub(other *Value) *Value

Functions

func Accuracy

func Accuracy(scores, labels [][]*Value, trainingParam TrainingParam) (accuracy float64)

Computes the accuracy of a model given scores and labels. It also requires a classification threshold.

func DrawGraph

func DrawGraph(values []*Value, filename string)

Builds and draws a graph and writes it to a file.

func GetRecords

func GetRecords(lines [][]string, batchSize int) ([][]*Value, [][]*Value)

Returns the inputs and labels of the lines. If batchSize is smaller than the number of records, we randomly sample from it. The input and label sizes are equal to the batchSize.

func MakeActivation

func MakeActivation(op string, f, g func(float64) float64) func(*Value) *Value

Given a function f and its derivative g, returns an activation function.

func ReadCSV

func ReadCSV(filename string) [][]string

Reads data from a CSV file and returns each line as a slice of string.

Types

type Layer

type Layer struct {
	// contains filtered or unexported fields
}

A layer object consisting of multiple neurons.

func MakeLayer

func MakeLayer(inputSize int, layerParam LayerParam) *Layer

Makes a layer consisting of multiple neurons.

func (*Layer) Fit

func (l *Layer) Fit(input []*Value) []*Value

Computes all output values of the layer given the input values and an activation function.

type LayerParam

type LayerParam struct {
	// contains filtered or unexported fields
}

Parameters of a layer: outputSize AKA the number of neurons in the layer. Each layer can have a different activation function.

func MakeLayerParam

func MakeLayerParam(outputSize int, activation func(*Value) *Value) LayerParam

Makes a LayerParam object with a given outputSize (number of neurons) and an activation function.

type NeuralNetwork

type NeuralNetwork struct {
	// contains filtered or unexported fields
}

A neural network object consitsting of multiple layers.

func MakeNeuralNetwork

func MakeNeuralNetwork(inputSize int, layerParams []LayerParam) *NeuralNetwork

Makes a neural network consisting of multiple layers.

func (*NeuralNetwork) Fit

func (n *NeuralNetwork) Fit(input []*Value) []*Value

Fits the model on input data and return the score.

func (*NeuralNetwork) Forward

func (n *NeuralNetwork) Forward(inputs [][]*Value) [][]*Value

Computes scores of all input data.

func (*NeuralNetwork) Loss

func (n *NeuralNetwork) Loss(labels, scores [][]*Value, trainingParam TrainingParam) *Value

Computes the loss as a Value object which is minimized in the optimization process when traininng the model.

func (*NeuralNetwork) NextData

func (n *NeuralNetwork) NextData(learningRate float64)

Moves in the direction of the gradient descent and updates model.

func (*NeuralNetwork) ResetGrad

func (n *NeuralNetwork) ResetGrad()

Resets grad values of the entire network recursively.

func (*NeuralNetwork) Train

func (n *NeuralNetwork) Train(inputs, labels [][]*Value, trainingParam TrainingParam) ([]float64, [][]*Value)

Trains the network by minimizing the loss function

type Neuron

type Neuron struct {
	// contains filtered or unexported fields
}

A neuron object with parameters w_1, ..., w_n, b.

func MakeNeuron

func MakeNeuron(inputSize int) *Neuron

Makes a neuron with a given inputSize. A neuron has inputSize+1 parameters. The intercept is initialized to 0 and weights are initialized to random numbers in [-1, 1).

func (*Neuron) Fit

func (n *Neuron) Fit(input []*Value) *Value

Computes output of a neuron as activation(w_1*x_1 + ... + w_n*x_n + b).

type Plotter

type Plotter struct {
	Width, Height vg.Length
}

Plotter object containing parameters for plotting a graph.

func (Plotter) PlotLine

func (p Plotter) PlotLine(x, y []float64, filename string)

Draws the loss function and saves it to a file.

func (Plotter) ScatterPlot

func (p Plotter) ScatterPlot(x, y []float64, filename string)

Draws the scatter plot of given points.

type TrainingParam

type TrainingParam struct {
	Epochs                  int
	Regularization          float64
	ClassificationThreshold float64
	LearningRate            float64
}

TrainingParam holds parameters required for training the network.

type Value

type Value struct {
	// contains filtered or unexported fields
}

Value object. Leaf nodes represent input data with op="", len(children)=0 backward=nil. Other nodes represents data resulted from an operation (op) on children. backward() updates gradient of children.

func MakeValue

func MakeValue(data float64) *Value

Makes a new value from a float number.

func Relu

func Relu(value *Value) *Value

Rectified linear unit: y = max(0, x)

func Sigmoid

func Sigmoid(value *Value) *Value

Sigmoid function: y = 1/(1 + exp(-x))

func Softmax

func Softmax(value *Value) *Value

Exponent: y = exp(x) The output needs normalization which will be done in the specified layer.

func Tanh

func Tanh(value *Value) *Value

Hyperbolic tangent (tanh): y = (exp(2x) - 1) / (exp(2x) + 1)

func (*Value) Add

func (value *Value) Add(other *Value) *Value

Addition: a+b

func (*Value) BackPropagate

func (value *Value) BackPropagate()

Implements backward propagation the topologically sorted list of nodes. It's applied on the loss function value which needs to be minimized.

func (*Value) Div

func (value *Value) Div(other *Value) *Value

Division: a/b

func (*Value) Exp

func (value *Value) Exp() *Value

func (Value) GetData

func (value Value) GetData() float64

Returns the data in this value object.

func (Value) GetGrad

func (value Value) GetGrad() float64

Returns the gradient of a given value.

func (Value) GetOp

func (value Value) GetOp() string

Returns the operation that is applied on the children resulted in this value.

func (*Value) Log

func (value *Value) Log() *Value

func (*Value) Mul

func (value *Value) Mul(other *Value) *Value

Multiplication: a*b

func (*Value) Pow

func (value *Value) Pow(b float64) *Value

func (*Value) ResetGrad

func (value *Value) ResetGrad()

func (*Value) SetData

func (value *Value) SetData(data float64)

func (*Value) Sub

func (value *Value) Sub(other *Value) *Value

Subtraction: a-b