plots

Documentation

Overview

Package plots define common types and utilities to the different plot libraries.

Index

• Constants
• func AddTrainAndEvalMetrics(plotter Plotter, loop *train.Loop, trainMetrics []tensor.Tensor, ...) error
• func CreatePointsWriter(filePath string) (pointWriter chan<- Point, errReport <-chan error)
• type Plotter
• type Point
  • func LoadPoints(filePath string) ([]Point, error)
  • func LoadPointsFromCheckpoint(checkpointDir string) ([]Point, error)
• type Points
  • func NewPoints(rawPoints []Point) (points Points)
  • func (points Points) Add(otherPoints Points)
  • func (points Points) Extract() (rawPoints []Point)
  • func (points Points) Filter(fn func(p Point) bool)
  • func (points Points) Map(fn func(p *Point))
  • func (points Points) MetricsNames() []string
  • func (points Points) String() string
  • func (points Points) TableForMetrics(metrics ...string) string

Constants

const TrainingPlotFileName = "training_plot_points.json"

TrainingPlotFileName is the default file name within a checkpoint directory to store plot points collected during training.

Functions

func AddTrainAndEvalMetrics

func AddTrainAndEvalMetrics(plotter Plotter, loop *train.Loop, trainMetrics []tensor.Tensor, evalDatasets []train.Dataset) error

AddTrainAndEvalMetrics is used by plotters (see [margaid.PLots] and [plotly.PlotConfig]) to include the metrics generated given training, plus to run evaluation on the given datasets.

Notice it evaluate on the datasets sequentially -- presumably the training could go in parallel if there is enough accelerator processing / memory. But this doesn't assume that.

func CreatePointsWriter

func CreatePointsWriter(filePath string) (pointWriter chan<- Point, errReport <-chan error)

CreatePointsWriter creates a channel to write Point to the given file. It creates an errReport channel to report an error (or nil) back at the very end. If any error occurs, it stops writing, and will report the error back once pointWriter is closed.

Types

type Plotter

type Plotter interface {
	// AddPoint to be drawn. One metric at a time.
	AddPoint(point Point)

	// DynamicSampleDone is called after all the data points recorded for this sample (evaluation at a time step).
	// The value `incomplete` is set to true if any of the evaluations are NaN or infinite.
	//
	// If in a notebook, this would trigger a redraw of the plot.
	DynamicSampleDone(incomplete bool)
}

Plotter is a generic plotter API, implemented by [margaid.Plots] and [plotly.PlotConfig].

type Point

type Point struct {
	// MetricName of this point.
	MetricName string

	// MetricType typically will be "loss", "accuracy".
	// It's used in plotting to aggregate similar metric types in the same plot.
	MetricType string

	// Step is the global step this metric was measured.
	// Usually, this is an int value, stored as a float64.
	Step float64

	// Value is the metric captured.
	Value float64
}

Point represents a training plot point. It is used to save/load plots.

func LoadPoints

func LoadPoints(filePath string) ([]Point, error)

LoadPoints parses all plot points saved in the given file.

func LoadPointsFromCheckpoint

func LoadPointsFromCheckpoint(checkpointDir string) ([]Point, error)

LoadPointsFromCheckpoint loads all plot points saved during training in file TrainingPlotFileName in a checkpoint directory.

type Points

type Points map[float64][]Point

Points is a collection of Point objects organized by their Step value. It's a `map[float64][]Point` with several utility methods.

func NewPoints

func NewPoints(rawPoints []Point) (points Points)

NewPoints create a Points object from a collection of individual `Point`.

See LoadPoints and LoadPointsFromCheckpoint if you want to read `rawPoints` from a file.

func (Points) Add

func (points Points) Add(otherPoints Points)

Add `otherPoints` into this `Points` structure. `otherPoints` is unchanged. It does not check for duplicates, points from `otherPoints` are simply appended as is.

func (Points) Extract

func (points Points) Extract() (rawPoints []Point)

Extract converts the Points structure back to a list of individual points. The output is sorted by [Point.Step].

func (Points) Filter

func (points Points) Filter(fn func(p Point) bool)

Filter only keeps those points for which `fn` returns true, removing the other ones.

func (Points) Map

func (points Points) Map(fn func(p *Point))

Map executes the given function on all individual points, in `Step` order. Note that if `p.Step` change, it is not re-index.

If you need to reindex the [Step] after the `Map` transformation, you may call [Extract] followed by NewPoints to create the re-indexed structure.

func (Points) MetricsNames

func (points Points) MetricsNames() []string

MetricsNames return the list of metrics names in the whole collection, sorted alphabetically by their type and then by their name.

func (Points) String

func (points Points) String() string

func (Points) TableForMetrics

func (points Points) TableForMetrics(metrics ...string) string

TableForMetrics returns a table with the first column being the `Step` followed by the columns given by the `metrics` names. If `metrics` is empty, it will include all metrics in the table.