neuralnetwork

Documentation

Index

• Constants
• type BatchGradientDescent
• type BatchWorker
• type BatchWorkerFactory
  • func NewLocalBatchWorkerFactory(network *NeuralNetwork) BatchWorkerFactory
• type EarlyStop
• type GradientDescentInitializer
• type NeuralNetwork
  • func NewNeuralNetwork(sizes []int, gradientDescentInitializer GradientDescentInitializer) (nn *NeuralNetwork, err error)
  • func (nn *NeuralNetwork) Copy() (outNN *NeuralNetwork)
  • func (nn *NeuralNetwork) LoadState(reader io.Reader)
  • func (nn *NeuralNetwork) LoadStateFromFile(filePath string)
  • func (nn *NeuralNetwork) Predict(aIn mat.Matrix) (maxIndex int, max float64)
  • func (nn *NeuralNetwork) Reset(sizes []int) (err error)
  • func (nn *NeuralNetwork) SaveState(writer io.Writer)
  • func (nn *NeuralNetwork) SaveStateToFile(filePath string)
  • func (nn *NeuralNetwork) SetBatchWorkerFactory(factory BatchWorkerFactory)
  • func (nn *NeuralNetwork) SetEarlyStop(earlyStop EarlyStop)
  • func (nn *NeuralNetwork) SetStateWatcher(watcher StateWatcher)
  • func (nn *NeuralNetwork) Train(trainer training.Trainer, epocs int)
  • func (nn *NeuralNetwork) Validate(trainer training.Trainer) (squareError float64, failCount, total int)
• type OnlineGradientDescent
• type StateWatcher
• type SubscriptionFeatures
  • func (f *SubscriptionFeatures) Clear()
  • func (f SubscriptionFeatures) Has(flag SubscriptionFeatures) bool
  • func (f *SubscriptionFeatures) Set(flag SubscriptionFeatures)
  • func (f *SubscriptionFeatures) Unset(flag SubscriptionFeatures)

Constants

const (
	BiasGradient   = iota
	WeightGradient = iota
)

const (
	StateIdle       = 1
	StateLearning   = 2
	StateValidation = 3
	StatePredict    = 4
)

Types

type BatchGradientDescent

type BatchGradientDescent interface {
	ApplyDelta(m mat.Matrix) *mat.Dense
	AccumGradients(gradient mat.Matrix, batchSize int)
	Gradients() *mat.Dense
}

Batch gradient descent interface. Is used by batch training mechanism

type BatchWorker

type BatchWorker interface {
	Run(trainer training.Trainer, startIndex, endIndex int)
	Result(layer int) (dB, dW *mat.Dense, batchSize int)
}

type BatchWorkerFactory

type BatchWorkerFactory interface {
	GetBatchWorker() BatchWorker
	GetAvailableThreads() int
}

func NewLocalBatchWorkerFactory

func NewLocalBatchWorkerFactory(network *NeuralNetwork) BatchWorkerFactory

type EarlyStop

type EarlyStop interface {
	Test() bool
	Reset()
}

type GradientDescentInitializer

type GradientDescentInitializer func(nn *NeuralNetwork, layer, gradientType int) interface{}

Interface factory function type that specifies producer interface for gradient descent objects

type NeuralNetwork

type NeuralNetwork struct {
	LayerCount int
	Sizes      []int
	Biases     []*mat.Dense
	Weights    []*mat.Dense
	BGradient  []interface{}
	WGradient  []interface{}
	// contains filtered or unexported fields
}

func NewNeuralNetwork

func NewNeuralNetwork(sizes []int, gradientDescentInitializer GradientDescentInitializer) (nn *NeuralNetwork, err error)

NewNeuralNetwork construction method that initializes new NeuralNetwork based on provided list of layer sizes and GradientDescentInitializer that used for backpropagation mechanism. If gradientDescentInitializer is not provided (is nil) backpropagation won't be possible. Common usecase when it's used is natural selection and genetic training.

func (*NeuralNetwork) Copy

func (nn *NeuralNetwork) Copy() (outNN *NeuralNetwork)

Copy makes complete copy of NeuralNetwork data. Output network has the same weights and biases values and but might be used independend of original one, e.g. in separate goroutine

func (*NeuralNetwork) LoadState

func (nn *NeuralNetwork) LoadState(reader io.Reader)

LoadState loads NeuralNetwork state from io.Reader. All existing data in NeuralNetwork will be rewritten buy this method, including layers configuration and weights and biases.

func (*NeuralNetwork) LoadStateFromFile

func (nn *NeuralNetwork) LoadStateFromFile(filePath string)

LoadStateFromFile loads NeuralNetwork state from file by specific filePath.

func (*NeuralNetwork) Predict

func (nn *NeuralNetwork) Predict(aIn mat.Matrix) (maxIndex int, max float64)

Predict method invokes prediction based on input activations provided in argument. Returns index of best element in output activation matrix and its value.

func (*NeuralNetwork) Reset

func (nn *NeuralNetwork) Reset(sizes []int) (err error)

Reset resets network state to intial/random one with specified in argument layers configuration.

func (*NeuralNetwork) SaveState

func (nn *NeuralNetwork) SaveState(writer io.Writer)

SaveState saves state of NeuralNetwork to io.Writer. It's usefull to keep training results between NeuralNetwork "power cycles" or to share traing results between clustered neural network hosts.

func (*NeuralNetwork) SaveStateToFile

func (nn *NeuralNetwork) SaveStateToFile(filePath string)

SaveStateToFile saves NeuralNetwork state to file by specific filePath.

func (*NeuralNetwork) SetBatchWorkerFactory

func (nn *NeuralNetwork) SetBatchWorkerFactory(factory BatchWorkerFactory)

SetBatchWorkerFactory setup batch worker factory for batch training. In case if factory is not setup localBatchWorkerFactory will be used.

func (*NeuralNetwork) SetEarlyStop

func (nn *NeuralNetwork) SetEarlyStop(earlyStop EarlyStop)

SetEarlyStop setup early stop analyser to stop training before all training epocs finished. Usually early stop required to avoid overfitting in neural network.

func (*NeuralNetwork) SetStateWatcher

func (nn *NeuralNetwork) SetStateWatcher(watcher StateWatcher)

SetStateWatcher setups state watcher for NeuralNetwork. StateWatcher is common interface that collects data about NeuralNetwork behavior. If not specified (is set to nil) NeuralNetwork will ignore StateWatcher interations.

func (*NeuralNetwork) Train

func (nn *NeuralNetwork) Train(trainer training.Trainer, epocs int)

Train is common training function that invokes one of training methods depends on gradient descent used buy NeuralNetwork. training.Trainer passed as argument used to get training data. Training loops are limited buy number of epocs.

func (*NeuralNetwork) Validate

func (nn *NeuralNetwork) Validate(trainer training.Trainer) (squareError float64, failCount, total int)

Validate runs basic network validation/verification based on validation data that provided by training.Trainer passed as argument. Returns count of failure predictions and total amount of verified samples and mean square sum of errors for all samples

type OnlineGradientDescent

type OnlineGradientDescent interface {
	ApplyDelta(m mat.Matrix, gradient mat.Matrix) *mat.Dense
}

Online gradient descent interface. Is used by online training mechanism

type StateWatcher

type StateWatcher interface {
	Init(nn *NeuralNetwork)
	UpdateState(state int)
	UpdateActivations(l int, a *mat.Dense)
	UpdateBiases(l int, biases *mat.Dense)
	UpdateWeights(l int, weights *mat.Dense)
	UpdateTraining(t int, epocs int, samplesProcced int, totalSamplesCount int)
	UpdateValidation(validatorCount int, failCount int)
	GetSubscriptionFeatures() SubscriptionFeatures
}

type SubscriptionFeatures

type SubscriptionFeatures uint8

const (
	ActivationsSubscription SubscriptionFeatures = 1 << iota
	BiasesSubscription
	WeightsSubscription
	TrainingSubscription
	ValidationSubscription
	StateSubscription
	AllSubscription = 0xFF
)

func (*SubscriptionFeatures) Clear

func (f *SubscriptionFeatures) Clear()

func (SubscriptionFeatures) Has

func (f SubscriptionFeatures) Has(flag SubscriptionFeatures) bool

func (*SubscriptionFeatures) Set

func (f *SubscriptionFeatures) Set(flag SubscriptionFeatures)

func (*SubscriptionFeatures) Unset

func (f *SubscriptionFeatures) Unset(flag SubscriptionFeatures)