experiment

package
v2.8.7 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 26, 2021 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

Package experiment defines standard evolutionary epochs evaluators and experimental data samples collectors.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func OutDirForTrial 

func OutDirForTrial(outDir string, trialID int) string

OutDirForTrial To provide standard output directory syntax based on current trial Method checks if directory should be created

Types

type Experiment 

type Experiment struct {
	Id       int
	Name     string
	RandSeed int64
	Trials
	// The maximal allowed fitness score as defined by fitness function of experiment.
	// It is used to normalize fitness score value used in efficiency score calculation. If this value
	// is not set, than fitness score will not be normalized during efficiency score estimation.
	MaxFitnessScore float64
}

An Experiment is a collection of trials for one experiment. It's useful for statistical analysis of a series of experiments

func (*Experiment) AvgEpochDuration 

func (e *Experiment) AvgEpochDuration() time.Duration

AvgEpochDuration Calculates average duration of evaluations among all generations of organism populations in this experiment

func (*Experiment) AvgGenerationsPerTrial 

func (e *Experiment) AvgGenerationsPerTrial() float64

AvgGenerationsPerTrial Calculates average number of generations evaluated per trial during this experiment. This can be helpful when estimating algorithm efficiency, because when winner organism is found the trial is terminated, i.e. less evaluations - more fast convergence.

func (*Experiment) AvgTrialDuration 

func (e *Experiment) AvgTrialDuration() time.Duration

AvgTrialDuration Calculates average duration of experiment's trial Note, that most trials finish after solution solved, so this metric can be used to represent how efficient the solvers was generated

func (*Experiment) AvgWinner 

func (e *Experiment) AvgWinner() (avgNodes, avgGenes, avgEvals, avgDiversity float64)

AvgWinner Returns average number of nodes, genes, organisms evaluations, and species diversity of winner genomes among all trials, i.e. for all trials where winning solution was found

func (*Experiment) BestAge 

func (e *Experiment) BestAge() Floats

BestAge The age values of the organisms for each trial

func (*Experiment) BestComplexity 

func (e *Experiment) BestComplexity() Floats

BestComplexity The complexity values of the best organisms for each trial

func (*Experiment) BestFitness 

func (e *Experiment) BestFitness() Floats

BestFitness The fitness values of the best organisms for each trial

func (*Experiment) BestOrganism 

func (e *Experiment) BestOrganism(onlySolvers bool) (*genetics.Organism, int, bool)

BestOrganism Finds the most fit organism among all epochs in this trial. It's also possible to get the best organism only among the ones which was able to solve the experiment's problem. Returns the best fit organism in this experiment among with ID of trial where it was found and boolean value to indicate if search was successful.

func (*Experiment) Decode 

func (e *Experiment) Decode(dec *gob.Decoder) error

Decode Decodes experiment data

func (*Experiment) Diversity 

func (e *Experiment) Diversity() Floats

Diversity returns the average number of species in each trial

func (*Experiment) EfficiencyScore 

func (e *Experiment) EfficiencyScore() float64

EfficiencyScore Calculates the efficiency score of the solution We are interested in efficient solver search solution that take less time per epoch, less generations per trial, and produce less complicated winner genomes. At the same time it should have maximal fitness score and maximal success rate among trials.

func (*Experiment) Encode 

func (e *Experiment) Encode(enc *gob.Encoder) error

Encode Encodes experiment with GOB encoding

func (*Experiment) EpochsPerTrial 

func (e *Experiment) EpochsPerTrial() Floats

EpochsPerTrial returns the number of epochs in each trial

func (*Experiment) Execute 

func (e *Experiment) Execute(ctx context.Context, startGenome *genetics.Genome, evaluator GenerationEvaluator, trialObserver TrialRunObserver) error

Execute is to run specific experiment using provided startGenome and specific evaluator for each epoch of the experiment

func (*Experiment) MostRecentTrialEvalTime 

func (e *Experiment) MostRecentTrialEvalTime() time.Time

MostRecentTrialEvalTime Returns the time of evaluation of the most recent trial

func (*Experiment) PrintStatistics 

func (e *Experiment) PrintStatistics()

PrintStatistics Prints experiment statistics

func (*Experiment) Read 

func (e *Experiment) Read(r io.Reader) error

Read is to read experiment data from provided reader and decodes it

func (*Experiment) Solved 

func (e *Experiment) Solved() bool

Solved is to check if solution was found in at least one trial

func (*Experiment) SuccessRate 

func (e *Experiment) SuccessRate() float64

SuccessRate The success rate

func (*Experiment) TrialsSolved 

func (e *Experiment) TrialsSolved() int

TrialsSolved The number of trials solved

func (*Experiment) Write 

func (e *Experiment) Write(w io.Writer) error

Write is to writes encoded experiment data into provided writer

func (*Experiment) WriteNPZ 

func (e *Experiment) WriteNPZ(w io.Writer) error

WriteNPZ Dumps experiment results to the NPZ file. The file has following structure: - trials_fitness - the mean, variance of fitness scores per trial - trials_ages - the mean, variance of species ages per trial - trials_complexity - the mean, variance of genome complexity of best organisms among species per trial - trial_[0...n]_epoch_mean_fitnesses - the mean fitness scores per epoch per trial - trial_[0...n]_epoch_best_fitnesses - the best fitness scores per epoch per trial the same for AGE and COMPLEXITY per epoch per trial - trial_[0...n]_epoch_diversity - the number of species per epoch per trial

type Experiments 

type Experiments []Experiment

Experiments is a sortable list of experiments by execution time and Id

func (Experiments) Len 

func (es Experiments) Len() int

func (Experiments) Less 

func (es Experiments) Less(i, j int) bool

func (Experiments) Swap 

func (es Experiments) Swap(i, j int)

type Floats 

type Floats []float64

Floats provides descriptive statistics on a slice of float64 values

func (Floats) Max 

func (x Floats) Max() float64

Max returns the greatest value in the slice

func (Floats) Mean 

func (x Floats) Mean() float64

Mean returns the average of the values in the slice

func (Floats) MeanVariance 

func (x Floats) MeanVariance() []float64

MeanVariance returns the sample mean and unbiased variance of the values in the slice

func (Floats) Median 

func (x Floats) Median() float64

Median returns the middle value in the slice (50% quantile)

func (Floats) Min 

func (x Floats) Min() float64

Min returns the smallest value in the slice

func (Floats) Q25 

func (x Floats) Q25() float64

Q25 is the 25% quantile

func (Floats) Q75 

func (x Floats) Q75() float64

Q75 is the 75% quantile

func (Floats) StdDev 

func (x Floats) StdDev() float64

StdDev returns the standard deviation of the values in the slice

func (Floats) Sum 

func (x Floats) Sum() float64

Sum returns the total of the values in the slice

func (Floats) Variance 

func (x Floats) Variance() float64

Variance returns the variance of the values in the slice

type Generation 

type Generation struct {
	// The generation ID for this epoch
	Id int
	// The time when epoch was evaluated
	Executed time.Time
	// The elapsed time between generation execution start and finish
	Duration time.Duration
	// The best organism of best species
	Best *genetics.Organism
	// The flag to indicate whether experiment was solved in this epoch
	Solved bool

	// The list of organisms fitness values per species in population
	Fitness Floats
	// The age of organisms per species in population
	Age Floats
	// The list of organisms complexities per species in population
	Complexity Floats

	// The number of species in population at the end of this epoch
	Diversity int

	// The number of evaluations done before winner found
	WinnerEvals int
	// The number of nodes in winner genome or zero if not solved
	WinnerNodes int
	// The numbers of genes (links) in winner genome or zero if not solved
	WinnerGenes int

	// The ID of Trial this Generation was evaluated in
	TrialId int
}

Generation the structure to represent execution results of one generation

func (*Generation) Average 

func (g *Generation) Average() (fitness, age, complexity float64)

Average Returns average fitness, age, and complexity among all organisms from population at the end of this epoch

func (*Generation) Decode 

func (g *Generation) Decode(dec *gob.Decoder) error

func (*Generation) Encode 

func (g *Generation) Encode(enc *gob.Encoder) error

Encode Encodes generation with provided GOB encoder

func (*Generation) FillPopulationStatistics 

func (g *Generation) FillPopulationStatistics(pop *genetics.Population)

FillPopulationStatistics Collects statistics about given population

type GenerationEvaluator 

type GenerationEvaluator interface {
	// GenerationEvaluate Invoked to evaluate one generation of population of organisms within given
	// execution context.
	GenerationEvaluate(pop *genetics.Population, epoch *Generation, context *neat.Options) (err error)
}

GenerationEvaluator the interface describing evaluator for one epoch (generation) of the evolutionary process.

type Generations 

type Generations []Generation

Generations is a sortable collection of generations by execution time and Id

func (Generations) Len 

func (is Generations) Len() int

func (Generations) Less 

func (is Generations) Less(i, j int) bool

func (Generations) Swap 

func (is Generations) Swap(i, j int)

type Trial 

type Trial struct {
	// The trial number
	Id int
	// The results per generation in this trial
	Generations Generations
	// The winner generation
	WinnerGeneration *Generation

	// The elapsed time between trial start and finish
	Duration time.Duration
}

Trial The structure to hold statistics about one experiment run (trial)

func (*Trial) Average 

func (t *Trial) Average() (fitness, age, complexity Floats)

Average Returns average fitness, age, and complexity of population of organisms for each epoch in this trial

func (*Trial) AvgEpochDuration 

func (t *Trial) AvgEpochDuration() time.Duration

AvgEpochDuration Calculates average duration of evaluations among all generations of organism populations in this trial

func (*Trial) BestAge 

func (t *Trial) BestAge() Floats

BestAge Age returns the age of the best species for each epoch in this trial

func (*Trial) BestComplexity 

func (t *Trial) BestComplexity() Floats

BestComplexity Complexity returns the complexity of the best species for each epoch in this trial

func (*Trial) BestFitness 

func (t *Trial) BestFitness() Floats

BestFitness Fitness returns the fitness values of the best organisms for each epoch in this trial

func (*Trial) BestOrganism 

func (t *Trial) BestOrganism(onlySolvers bool) (*genetics.Organism, bool)

BestOrganism Finds the most fit organism among all epochs in this trial. It's also possible to get the best organism only among the ones which was able to solve the experiment's problem.

func (*Trial) Decode 

func (t *Trial) Decode(dec *gob.Decoder) error

Decode Decodes trial data

func (*Trial) Diversity 

func (t *Trial) Diversity() Floats

Diversity returns number of species for each epoch

func (*Trial) Encode 

func (t *Trial) Encode(enc *gob.Encoder) error

Encode Encodes this trial

func (*Trial) RecentEpochEvalTime 

func (t *Trial) RecentEpochEvalTime() time.Time

RecentEpochEvalTime is to get time of the epoch executed most recently within this trial

func (*Trial) Solved 

func (t *Trial) Solved() bool

func (*Trial) Winner 

func (t *Trial) Winner() (nodes, genes, evals, diversity int)

Winner Returns number of nodes, genes, organism evaluations and species diversity in the winner genome

type TrialRunObserver 

type TrialRunObserver interface {
	// TrialRunStarted invoked to notify that new trial run just started. Invoked before any epoch evaluation in that trial run
	TrialRunStarted(trial *Trial)
	// TrialRunFinished invoked to notify that the trial run just finished. Invoked after all epochs evaluated or successful solver found.
	TrialRunFinished(trial *Trial)
	// EpochEvaluated invoked to notify that evaluation of specific epoch completed.
	EpochEvaluated(trial *Trial, epoch *Generation)
}

TrialRunObserver defines observer to be notified about experiment's trial lifecycle methods

type Trials 

type Trials []Trial

Trials is a sortable collection of experiment runs (trials) by execution time and id

func (Trials) Len 

func (ts Trials) Len() int

func (Trials) Less 

func (ts Trials) Less(i, j int) bool

func (Trials) Swap 

func (ts Trials) Swap(i, j int)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.