benchtab

README

internal/benchtab

This is a fork of golang.org/x/perf/cmd/benchstat/internal/benchtab, which is the core of the benchstat command. The perf service reuses that core to perform its own comparisons.

Documentation

Overview

Package benchtab presents benchmark results as comparison tables.

Index

• type Builder
  • func NewBuilder(tableBy, rowBy, colBy, residue *benchproc.Projection) *Builder
  • func (b *Builder) Add(result *benchfmt.Result)
  • func (b *Builder) ToTables(opts TableOpts) *Tables
• type Table
  • func (t *Table) RowScaler(row benchproc.Key, unitClass benchunit.Class) benchunit.Scaler
• type TableCell
• type TableKey
• type TableOpts
• type TableSummary
• type Tables

Types

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

A Builder collects benchmark results into a Tables set.

func NewBuilder

func NewBuilder(tableBy, rowBy, colBy, residue *benchproc.Projection) *Builder

NewBuilder creates a new Builder for collecting benchmark results into tables. Each result will be mapped to a Table by tableBy. Within each table, the results are mapped to cells by rowBy and colBy. Any results within a single cell that vary by residue will be reported as warnings. tableBy must have a ".unit" field.

func (*Builder) Add

func (b *Builder) Add(result *benchfmt.Result)

Add adds all of the values in result to the tables in the Builder.

func (*Builder) ToTables

func (b *Builder) ToTables(opts TableOpts) *Tables

ToTables finalizes a Builder into a sequence of statistic tables.

type Table

type Table struct {
	// Opts is the configuration options for this table.
	Opts TableOpts

	// Unit is the benchmark unit of all samples in this Table.
	Unit string

	// Assumption is the distributional assumption used for all
	// samples in this table.
	Assumption benchmath.Assumption

	// Rows and Cols give the sequence of row and column Keys
	// in this table. All row Keys have the same Projection and all
	// col Keys have the same Projection.
	Rows, Cols []benchproc.Key

	// Cells is the cells in the body of this table. Each key in
	// this map is a pair of some Key from Rows and some Key
	// from Cols. However, not all Pairs may be present in the
	// map.
	Cells map[TableKey]*TableCell

	// Summary is the final row of this table, which gives summary
	// information across all benchmarks in this table. It is
	// keyed by Cols.
	Summary map[benchproc.Key]*TableSummary

	// SummaryLabel is the label for the summary row.
	SummaryLabel string
}

A Table summarizes and compares benchmark results in a 2D grid. Each cell summarizes a Sample of results with identical row and column Keys. Comparisons are done within each row between the Sample in the first column and the Samples in any remaining columns.

func (*Table) RowScaler

func (t *Table) RowScaler(row benchproc.Key, unitClass benchunit.Class) benchunit.Scaler

RowScaler returns a common scaler for the values in row.

type TableCell

type TableCell struct {
	// Sample is the set of benchmark results in this cell.
	Sample *benchmath.Sample

	// Summary is the summary of Sample, as computed by the
	// Table's distributional assumption.
	Summary benchmath.Summary

	// Baseline is the baseline cell used for comparisons with
	// this cell, or nil if there is no comparison. This is the
	// cell in the first column of this cell's row, if any.
	Baseline *TableCell

	// Comparison is the comparison with the Baseline cell, as
	// computed by the Table's distributional assumption. If
	// Baseline is nil, this value is meaningless.
	Comparison benchmath.Comparison
}

TableCell is a single cell in a Table. It represents a sample of benchmark results with the same row and column Key.

type TableKey

type TableKey struct {
	Row, Col benchproc.Key
}

TableKey is a map key used to index a single cell in a Table.

type TableOpts

type TableOpts struct {
	// Confidence is the desired confidence level in summary
	// intervals; e.g., 0.95 for 95%.
	Confidence float64

	// Thresholds is the thresholds to use for statistical tests.
	Thresholds *benchmath.Thresholds

	// Units is the unit metadata. This gives distributional
	// assumptions for units, among other properties.
	Units benchfmt.UnitMetadataMap
}

TableOpts provides options for constructing the final analysis tables from a Builder.

type TableSummary

type TableSummary struct {
	// HasSummary indicates that Summary is valid.
	HasSummary bool
	// Summary summarizes all of the TableCell.Summary values in
	// this column.
	Summary float64

	// HasRatio indicates that Ratio is valid.
	HasRatio bool
	// Ratio summarizes all of the TableCell.Comparison values in
	// this column.
	Ratio float64

	// Warnings is a list of warnings for this summary cell.
	Warnings []error
}

TableSummary is a cell that summarizes a column of a Table. It appears in the last row of a table.

type Tables

type Tables struct {
	// Tables is a slice of statistic tables. Within a Table, all
	// results have the same table Key (including unit).
	Tables []*Table
	// Keys is a slice of table keys, corresponding 1:1 to
	// the Tables slice. These always end with a ".unit"
	// field giving the unit.
	Keys []benchproc.Key
}

Tables is a sequence of benchmark statistic tables.