track

package
v0.0.0-...-225e849 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 22, 2020 License: Apache-2.0 Imports: 10 Imported by: 9

README

Tracker

The tracker is used to track values as an agent is performing. It can track arbitrary scalar values or values from a gorgonia graph. It comes with aggregation capabilities and an HTTP interface.

Usage

Create a new default tracker

tracker, _ := NewTracker()

Create a new tracker with a specific directory

tracker, _ := NewTracker(WithDir("./logs"))

Track a scalar value

score := tracker.TrackValue("score", 0)
score.Inc()

Track a scalar with a custom aggregator and increment it

score := tracker.TrackValue("score", 0, track.WithAggregator(track.Max))
score.Inc()

Track a scalar value with a custom slicer with gives the rate over an episode size

score := tracker.TrackValue("score", 0,track.WithAggregator(NewMeanAggregator(DefaultCummulativeSlicer)))
score.Inc()

Track a Gorgonia graph node value

var loss *gorgonia.Node

lossVal := tracker.TrackValue("loss", loss)

Create episodes and timesteps to track

for _, episode := range tracker.MakeEpisodes(100) {

    // create an episode bound scalar that will only hold its value per episode
    score := episode.TrackScalar("score", 0, track.WithAggregator(track.Max))

    for _, timestep := range episode.Steps(200) {
        // log all values per timestamp
        timstep.Log()
    }
    // log all values per episode
    episode.Log()
}

Access values via API

resp, _ := http.Get("my.host.com/api/values/myValue")

Returns the values as chartjs xy's aggregated according to the values aggregator. A custom aggregator can be provided using the ?aggregator= query param.

Documentation

Overview

Package track provides tooling for tracking agent metrics.

Index

Constants

This section is empty.

Variables

View Source 
var AggregatorNames = []AggregatorName{MeanAggregatorName, ModeAggregatorName, MaxAggregatorName}

AggregatorNames holds all of the current aggregator names.

View Source 
var DefaultCummulativeSlicer = &CummulativeRangeSlicer{back: 100, start: 0, end: -1}

DefaultCummulativeSlicer is the default slicer for cummulative slices.

View Source 
var DefaultRateAggregator = NewMeanAggregator(DefaultCummulativeSlicer)

DefaultRateAggregator is a default aggregator to take the rate of a value, uses the mean of the last 100 episodes.

View Source 
var Max = &MaxAggregator{SingleEpisodeSlicer}

Max aggregator.

View Source 
var Mean = &MeanAggregator{SingleEpisodeSlicer}

Mean aggregator.

View Source 
var Mode = &ModeAggregator{SingleEpisodeSlicer}

Mode aggregator.

View Source 
var SingleEpisodeSlicer = &EpisodicSlicer{}

SingleEpisodeSlicer slices all historical data by single epidodes.

Functions

func WithAggregator 

func WithAggregator(aggregator Aggregator) func(TrackedValue)

WithAggregator sets an aggregator to use with the value. Default to MeanAggregator.

func WithDir 

func WithDir(dir string) func(*Tracker)

WithDir is a tracker option to set the directory in which logs are stored.

func WithIndex 

func WithIndex(index int) func(TrackedValue)

WithIndex sets an index to use if the given value is non scalar. Defaults to 0.

func WithLogger 

func WithLogger(logger *log.Logger) func(*Tracker)

WithLogger adds a logger to the tracker.

func WithNamespace 

func WithNamespace(namespace string) func(TrackedValue)

WithNamespace adds a namespace to the tracked value.

Types

type Aggregable 

type Aggregable interface {
	// Scalar value.
	Scalar() float64

	// Ep is the episode for this value.
	// Note: this is shortened to deal with name conflicts.
	Ep() int
}

Aggregable is a value that can be aggregated.

type AggregableSlices 

type AggregableSlices map[int]Aggregables

AggregableSlices are slices of historical values.

func (AggregableSlices) Sort 

func (h AggregableSlices) Sort() []Aggregables

Sort the historical value slices returning an ordered list.

type Aggregables 

type Aggregables []Aggregable

Aggregables are a slice of aggregable values.

func (Aggregables) Scalar 

func (a Aggregables) Scalar() []float64

Scalar values.

type AggregatedValue 

type AggregatedValue struct {
	// contains filtered or unexported fields
}

AggregatedValue is an aggregated value.

func (*AggregatedValue) Ep 

func (a *AggregatedValue) Ep() int

Ep is the episode this value is linked to.

func (*AggregatedValue) Scalar 

func (a *AggregatedValue) Scalar() float64

Scalar value

type AggregatedValues 

type AggregatedValues []AggregatedValue

AggregatedValues is a slice of aggregated value.

type Aggregates 

type Aggregates struct {
	// contains filtered or unexported fields
}

Aggregates are generic aggregated values.

func NewAggregates 

func NewAggregates(xLabel, yLabel string) *Aggregates

NewAggregates returns a new aggregates.

func (Aggregates) Chartjs 

func (a Aggregates) Chartjs() *Chartjs

Chartjs returns the data for a ChartJS chart.

func (Aggregates) ChartjsXYs 

func (a Aggregates) ChartjsXYs() ChartjsXYs

ChartjsXYs returns the episode aggregates as gonum xy pairs.

func (Aggregates) GonumXYs 

func (a Aggregates) GonumXYs() plotter.XYs

GonumXYs returns the episode aggregates as gonum xy pairs.

func (Aggregates) Sort 

func (a Aggregates) Sort() []float64

Sort the episodes into a slice.

type Aggregator 

type Aggregator interface {
	// Aggregate the values.
	Aggregate(vals Aggregables) *Aggregates
}

Aggregator aggregates historical values into a single value.

func AggregatorFromName 

func AggregatorFromName(name string) (Aggregator, error)

AggregatorFromName returns an aggregator from its name.

type AggregatorName 

type AggregatorName string

AggregatorName is the name of an aggregator. 

const (
	// MeanAggregatorName is the name of the mean aggregator.
	MeanAggregatorName AggregatorName = "mean"

	// ModeAggregatorName is the name of the mode aggregator.
	ModeAggregatorName AggregatorName = "mode"

	// MaxAggregatorName is the name of the max aggregator.
	MaxAggregatorName AggregatorName = "max"
)

type ChainAggregator 

type ChainAggregator struct {
	// contains filtered or unexported fields
}

ChainAggregator is a chain of aggregators.

func NewChainAggregator 

func NewChainAggregator(aggregators ...Aggregator) *ChainAggregator

NewChainAggregator returns a new chain aggregator.

func (*ChainAggregator) Aggregate 

func (c *ChainAggregator) Aggregate(vals Aggregables) (retVal *Aggregates)

Aggregate the values.

type Chartjs 

type Chartjs struct {
	// XLabel is the label for the x values.
	XLabel string `json:"xLabel"`
	// YLabel is the label for the y values.
	YLabel string `json:"yLabel"`
	// XYS are the xy values.
	XYs ChartjsXYs `json:"xys"`
}

Chartjs is a ChartJS chart.

type ChartjsXY 

type ChartjsXY struct {
	// X value.
	X float64 `json:"x"`
	// Y value.
	Y float64 `json:"y"`
}

ChartjsXY conforms to the expected point data structure for chartjs charts.

type ChartjsXYs 

type ChartjsXYs []ChartjsXY

ChartjsXYs conforms to the expected set of point data structure for chartjs charts.

func (ChartjsXYs) Order 

func (c ChartjsXYs) Order() ChartjsXYs

Order the xys by x.

type CummulativeRangeSlicer 

type CummulativeRangeSlicer struct {
	// contains filtered or unexported fields
}

CummulativeRangeSlicer slices cummulative episodes i.e. each slice will contain the current episode plus however many episodes back are specified.

func NewCummulativeRangeSlicer 

func NewCummulativeRangeSlicer(back, start, end int) *CummulativeRangeSlicer

NewCummulativeRangeSlicer returns a new cummulative slicer.

func (*CummulativeRangeSlicer) Label 

func (e *CummulativeRangeSlicer) Label() string

Label applied to slice.

func (*CummulativeRangeSlicer) Slice 

func (e *CummulativeRangeSlicer) Slice(vals Aggregables) AggregableSlices

Slice the values.

type Episode 

type Episode struct {
	I      int
	Values map[string]TrackedValue
	// contains filtered or unexported fields
}

Episode represents a training episode.

func (*Episode) Data 

func (e *Episode) Data() *History

Data returns episodic data.

func (*Episode) GetValue 

func (e *Episode) GetValue(name string) (TrackedValue, error)

GetValue a tracked value by name.

func (*Episode) Log 

func (e *Episode) Log()

Log values for an episode.

func (*Episode) Steps 

func (e *Episode) Steps(num int) Timesteps

Steps to take in an episode.

func (*Episode) TrackScalar 

func (e *Episode) TrackScalar(name string, value interface{}, opts ...TrackedValueOpt) *TrackedScalarValue

TrackScalar tracks a scarlar value for only an episode.

func (*Episode) TrackValue 

func (e *Episode) TrackValue(name string, value interface{}, opts ...TrackedValueOpt) TrackedValue

TrackValue tracks a value for only an episode.

type EpisodeHistories 

type EpisodeHistories map[int]Histories

EpisodeHistories is a history of episodes

type Episodes 

type Episodes []*Episode

Episodes is a slice of episode.

type EpisodicSlicer 

type EpisodicSlicer struct{}

EpisodicSlicer slices episodes.

func NewEpisodicSlicer 

func NewEpisodicSlicer() *EpisodicSlicer

NewEpisodicSlicer returns a new episodic slicer.

func (*EpisodicSlicer) Label 

func (e *EpisodicSlicer) Label() string

Label applied to slice.

func (*EpisodicSlicer) Slice 

func (e *EpisodicSlicer) Slice(vals Aggregables) AggregableSlices

Slice the values.

type HistoricalValue 

type HistoricalValue struct {
	// Name of the value.
	Name string `json:"name"`

	// TrackedValue of the value.
	TrackedValue float64 `json:"value"`

	// Timestep at which the value occurred.
	Timestep int `json:"timestep"`

	// Episode at which the value occurred.
	Episode int `json:"episode"`
}

HistoricalValue is a historical value.

func (*HistoricalValue) Ep 

func (h *HistoricalValue) Ep() int

Ep is the episode in which value occurred.

func (*HistoricalValue) Scalar 

func (h *HistoricalValue) Scalar() float64

Scalar value.

type HistoricalValues 

type HistoricalValues []*HistoricalValue

HistoricalValues is a slice of historical values.

func (HistoricalValues) Aggregables 

func (h HistoricalValues) Aggregables() Aggregables

Aggregables returns the values as aggregables.

func (HistoricalValues) Aggregate 

func (h HistoricalValues) Aggregate(aggregator Aggregator) *Aggregates

Aggregate the values.

func (HistoricalValues) Print 

func (h HistoricalValues) Print()

Print the tracked values.

func (HistoricalValues) Scalar 

func (h HistoricalValues) Scalar() []float64

Scalar of the historical values.

type Histories 

type Histories []*History

Histories are a slice of history.

type History 

type History struct {
	// Values in the history.
	Values []*HistoricalValue `json:"values"`

	// Timestep of this history.
	Timestep int `json:"timestep"`

	// Episode of this history.
	Episode int `json:"episode"`
}

History is the historical representation of a set of tracked values.

func (*History) Get 

func (h *History) Get(name string) HistoricalValues

Get the value history with the given name.

type MaxAggregator 

type MaxAggregator struct{ Slicer }

MaxAggregator returns the max of the historical values.

func NewMaxAggregator 

func NewMaxAggregator(slicer Slicer) *MaxAggregator

NewMaxAggregator returns a new max aggregator.

func (*MaxAggregator) Aggregate 

func (m *MaxAggregator) Aggregate(vals Aggregables) *Aggregates

Aggregate the values.

type MeanAggregator 

type MeanAggregator struct{ Slicer }

MeanAggregator returns the mean of the historical values.

func NewMeanAggregator 

func NewMeanAggregator(slicer Slicer) *MeanAggregator

NewMeanAggregator returns a new mode aggregator.

func (*MeanAggregator) Aggregate 

func (m *MeanAggregator) Aggregate(vals Aggregables) *Aggregates

Aggregate the values.

type ModeAggregator 

type ModeAggregator struct{ Slicer }

ModeAggregator returns the most common of the historical values.

func NewModeAggregator 

func NewModeAggregator(slicer Slicer) *ModeAggregator

NewModeAggregator returns a new mode aggregator.

func (*ModeAggregator) Aggregate 

func (m *ModeAggregator) Aggregate(vals Aggregables) *Aggregates

Aggregate the values.

type Slicer 

type Slicer interface {
	// Slice the data.
	Slice(vals Aggregables) AggregableSlices

	// Label for these slices of data.
	Label() string
}

Slicer slices historical data into chunks that are then aggregated.

type Timestep 

type Timestep struct {
	I int
	// contains filtered or unexported fields
}

Timestep represents a training timestep.

func (*Timestep) Data 

func (t *Timestep) Data() *History

Data returns timestep data.

func (*Timestep) Log 

func (t *Timestep) Log()

Log values for a timestep.

type Timesteps 

type Timesteps []*Timestep

Timesteps is a slice of timestep.

type TrackedNodeValue 

type TrackedNodeValue struct {
	// contains filtered or unexported fields
}

TrackedNodeValue is a tracked node value.

func NewTrackedNodeValue 

func NewTrackedNodeValue(name string, opts ...TrackedValueOpt) *TrackedNodeValue

NewTrackedNodeValue returns a new tracked value.

func (*TrackedNodeValue) Aggregator 

func (t *TrackedNodeValue) Aggregator() Aggregator

Aggregator returns the aggregator for this value.

func (*TrackedNodeValue) Data 

func (t *TrackedNodeValue) Data(episode, timestep int) *HistoricalValue

Data converts the value to a historical value.

func (*TrackedNodeValue) Name 

func (t *TrackedNodeValue) Name() string

Name of the value.

func (*TrackedNodeValue) Print 

func (t *TrackedNodeValue) Print()

Print the value.

func (*TrackedNodeValue) Scalar 

func (t *TrackedNodeValue) Scalar() float64

Scalar value.

type TrackedScalarValue 

type TrackedScalarValue struct {
	// contains filtered or unexported fields
}

TrackedScalarValue is a tracked value that can be convertible to float64.

func NewTrackedScalarValue 

func NewTrackedScalarValue(name string, value interface{}, opts ...TrackedValueOpt) *TrackedScalarValue

NewTrackedScalarValue returns a new tracked value.

func (*TrackedScalarValue) Aggregator 

func (t *TrackedScalarValue) Aggregator() Aggregator

Aggregator returns the aggregator for this value.

func (*TrackedScalarValue) Data 

func (t *TrackedScalarValue) Data(episode, timestep int) *HistoricalValue

Data takes the current tracked value and returns a historical value.

func (*TrackedScalarValue) Get 

func (t *TrackedScalarValue) Get() interface{}

Get the value.

func (*TrackedScalarValue) Inc 

func (t *TrackedScalarValue) Inc(amount interface{})

Inc increments value.

func (*TrackedScalarValue) Name 

func (t *TrackedScalarValue) Name() string

Name of the value.

func (*TrackedScalarValue) Print 

func (t *TrackedScalarValue) Print()

Print the value.

func (*TrackedScalarValue) Scalar 

func (t *TrackedScalarValue) Scalar() float64

Scalar value.

func (*TrackedScalarValue) Set 

func (t *TrackedScalarValue) Set(v interface{})

Set the value.

type TrackedValue 

type TrackedValue interface {
	// Name of the value.
	Name() string

	// Scalar value.
	Scalar() float64

	// Print the value.
	Print()

	// Data converts the value to a historical value.
	Data(episode, timestep int) *HistoricalValue

	// Aggregator is the aggregator for this value.
	Aggregator() Aggregator
}

TrackedValue is a value being tracked.

type TrackedValueOpt 

type TrackedValueOpt func(TrackedValue)

TrackedValueOpt is an option for a tracked value.

type Tracker 

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker is a means of tracking values on a graph.

func NewTracker 

func NewTracker(opts ...TrackerOpt) (*Tracker, error)

NewTracker returns a new tracker for a graph.

func (*Tracker) AggregateValuesHandler 

func (t *Tracker) AggregateValuesHandler(w http.ResponseWriter, req *http.Request)

AggregateValuesHandler is an HTTP handler for the tracker serving aggregates.

func (*Tracker) AggregatorsHandler 

func (t *Tracker) AggregatorsHandler(w http.ResponseWriter, req *http.Request)

AggregatorsHandler returns all possible aggregators.

func (*Tracker) ApplyHandlers 

func (t *Tracker) ApplyHandlers(mux *http.ServeMux)

ApplyHandlers applies tracker handlers to a mux.

func (*Tracker) Clear 

func (t *Tracker) Clear() error

Clear all tracked data.

func (*Tracker) Data 

func (t *Tracker) Data() *History

Data yeilds the current tracked values into a historical structure.

func (*Tracker) GetEpisodeHistories 

func (t *Tracker) GetEpisodeHistories() (EpisodeHistories, error)

GetEpisodeHistories returns the episode history.

func (*Tracker) GetHistory 

func (t *Tracker) GetHistory(name string) (HistoricalValues, error)

GetHistory gets all the history for a value.

func (*Tracker) GetHistoryAll 

func (t *Tracker) GetHistoryAll() ([]HistoricalValues, error)

GetHistoryAll gets the history of all values.

func (*Tracker) GetValue 

func (t *Tracker) GetValue(name string) (TrackedValue, error)

GetValue a tracked value by name.

func (*Tracker) IncValue 

func (t *Tracker) IncValue(name string, scalar interface{}) error

IncValue increments a value by a scalar.

func (*Tracker) LogStep 

func (t *Tracker) LogStep(episode, timestep int) error

LogStep logs tracked values to store for the given timestep. TODO: make time more pluggable so this can be used in other environments.

func (*Tracker) MakeEpisodes 

func (t *Tracker) MakeEpisodes(num int) Episodes

MakeEpisodes creates a number of episodes.

func (*Tracker) PrintAll 

func (t *Tracker) PrintAll()

PrintAll values.

func (*Tracker) PrintHistoryAll 

func (t *Tracker) PrintHistoryAll() error

PrintHistoryAll prints the history of all values.

func (*Tracker) PrintValue 

func (t *Tracker) PrintValue(name string)

PrintValue prints a value.

func (*Tracker) TrackValue 

func (t *Tracker) TrackValue(name string, value interface{}, opts ...TrackedValueOpt) TrackedValue

TrackValue tracks a graph node or any other scalar value.

func (*Tracker) ValueNames 

func (t *Tracker) ValueNames() []string

ValueNames is the name for all values.

func (*Tracker) ValuesHandler 

func (t *Tracker) ValuesHandler(w http.ResponseWriter, req *http.Request)

ValuesHandler is an HTTP handler for revealing what values are tracked over the network.

func (*Tracker) Write 

func (t *Tracker) Write() error

Write tracker data.

func (*Tracker) ZeroValue 

func (t *Tracker) ZeroValue(name string) error

ZeroValue zeros a scalar value

type TrackerOpt 

type TrackerOpt func(*Tracker)

TrackerOpt is a tracker option.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.