datasets

Documentation

Overview

Package datasets implements the Neurlang dataset type

Index

• type Datamap
  • func (d *Datamap) Init()
  • func (d Datamap) Reduce(whole bool) (o Datamap)
  • func (d Datamap) Split() (o SplittedDataset)
• type Dataset
  • func (d *Dataset) Init()
  • func (d Dataset) Split() (o SplittedDataset)
• type SplittedDataset
  • func BalanceDataset(d SplittedDataset) SplittedDataset
  • func (d *SplittedDataset) Init()
  • func (d SplittedDataset) Split() (o SplittedDataset)
• type Splitter
• type Tally
  • func (t *Tally) AddToCorrect(feature uint32, vote int8, improvement bool)
  • func (t *Tally) AddToImprove(feature uint32, vote int8)
  • func (t *Tally) AddToMapAll(feature uint16, output uint64, loss func(n uint32) uint32, max uint32)
  • func (t *Tally) AddToMapping(feature uint16, output uint64)
  • func (t *Tally) Free()
  • func (t *Tally) GetImprovementPossible() bool
  • func (t *Tally) Init()
  • func (t *Tally) Len() (o int)
  • func (t *Tally) SetFinalization(final bool)
  • func (t *Tally) Split() SplittedDataset

Types

type Datamap

type Datamap map[uint16]uint64

Datamap holds keys mapped to values

func (*Datamap) Init

func (d *Datamap) Init()

Init initializes the Datamap

func (Datamap) Reduce

func (d Datamap) Reduce(whole bool) (o Datamap)

Reduce reduces the datamap

func (Datamap) Split

func (d Datamap) Split() (o SplittedDataset)

Split splits datamap into a true set and a false set

type Dataset

type Dataset map[uint32]bool

Dataset holds keys mapped to booleans

func (*Dataset) Init

func (d *Dataset) Init()

Init initializes the Dataset

func (Dataset) Split

func (d Dataset) Split() (o SplittedDataset)

Split splits dataset into a true set and a false set

type SplittedDataset

type SplittedDataset [2]map[uint32]struct{}

SplittedDataset is a dataset that has been split into the false set and true set

func BalanceDataset

func BalanceDataset(d SplittedDataset) SplittedDataset

BalanceDataset fills the smaller set with random number until it matches the bigger set

func (*SplittedDataset) Init

func (d *SplittedDataset) Init()

Init initializes a SplittedDataset

func (SplittedDataset) Split

func (d SplittedDataset) Split() (o SplittedDataset)

Split splits SplittedDataset into a true set and a false set

type Splitter

type Splitter interface {
	Split() (o SplittedDataset)
}

Splitter is a dataset that can be split into a SplittedDataset

type Tally

type Tally struct {
	// contains filtered or unexported fields
}

Tally is used to count votes on dataset features and return the majority votes

func (*Tally) AddToCorrect

func (t *Tally) AddToCorrect(feature uint32, vote int8, improvement bool)

Correct votes for feature which caused the overall result to be correct

func (*Tally) AddToImprove

func (t *Tally) AddToImprove(feature uint32, vote int8)

Improve votes for feature which improved the overall result

func (*Tally) AddToMapAll

func (t *Tally) AddToMapAll(feature uint16, output uint64, loss func(n uint32) uint32, max uint32)

AddToMapAll adds feature maps to all output votes to mapping

func (*Tally) AddToMapping

func (t *Tally) AddToMapping(feature uint16, output uint64)

AddToMap adds feature maps to this output votes to mapping

func (*Tally) Free

func (t *Tally) Free()

Free frees the memory occupied by tally dataset structure

func (*Tally) GetImprovementPossible

func (t *Tally) GetImprovementPossible() bool

GetImprovementPossible reads improvementPossible

func (*Tally) Init

func (t *Tally) Init()

Init initializes the tally dataset structure

func (*Tally) Len

func (t *Tally) Len() (o int)

Len estimates the size of tally

func (*Tally) SetFinalization

func (t *Tally) SetFinalization(final bool)

SetFinalization sets isFinalization and enables the final stage of training

func (*Tally) Split

func (t *Tally) Split() SplittedDataset

Split Splits the tally structure into a splitted dataset