datasets

Documentation

Overview

Package datasets comprises of several dataset file processing functions to present them in useful manner.

Index

• func BatchMatrix[T Signed | Float](input [][]T, batchSize int) []matrix.Matrix[T]
• func MnistDataset(filename string) ([][]float64, []float64, error)
• func OneHotEncode(labels []float64, classCount int) [][]float64

Functions

func BatchMatrix

func BatchMatrix[T Signed | Float](input [][]T, batchSize int) []matrix.Matrix[T]

BatchMatrix accepts a slice of inputs to the neural network, and produces a slice of matrices with row count equal to batchSize.

func MnistDataset

func MnistDataset(filename string) ([][]float64, []float64, error)

MnistDataset reads the CSV files for MNIST dataset.

Accepts a path to the .csv file, which has to be in format: N rows, first column is the label, next 784 columns - pixels of the image in range from 0 to 255.

Outputs a list of pixels, where each entry is a separate image and a slice of labels corresponding to the images.

Each slice of X represents a whole 28x28 image with values ranging from 0 to 255, i.e. length of 784. In order to feed it to the neural network, outputs have to be transposed and labels must be one-hot encoded.

func OneHotEncode

func OneHotEncode(labels []float64, classCount int) [][]float64

OneHotEncode accepts a slice of labels, and produces slices of one-hot encoded values (i.e. slices with 0s, and 1 in ith position) of length classCount.

labels := []float64{0, 2, 1}
encoded := OneHotEncode(labels, 3) // [ [1, 0, 0], [0, 0, 1], [0, 1, 0] ]