perfresults

Documentation

Index

• Variables
• func NewLoader() loader
• type BuildInfo
  • func (bi BuildInfo) GetPosition() string
• type Histogram
  • func (h Histogram) Aggregate(method string) float64
  • func (h Histogram) Count() int
  • func (h Histogram) Max() float64
  • func (h Histogram) Mean() float64
  • func (h Histogram) Min() float64
  • func (h Histogram) Stddev() float64
  • func (h Histogram) Sum() float64
• type PerfResults
  • func NewResults(r io.Reader) (*PerfResults, error)
  • func (pr *PerfResults) GetSampleValues(chart string) []float64
  • func (pr *PerfResults) MergeResults(other *PerfResults)
  • func (pr *PerfResults) NonEmptyHistogramNames() []string
  • func (pr *PerfResults) UnmarshalJSON(data []byte) error
• type RBEPerfLoader
  • func NewRBEPerfLoader(ctx context.Context, casInstance string) (*RBEPerfLoader, error)
  • func (c RBEPerfLoader) LoadPerfResults(ctx context.Context, cases ...*swarmingv2.CASReference) (map[string]*PerfResults, error)
• type TraceKey

Variables

var AggregationMapping = map[string]func(Histogram) float64{
	"max":  Histogram.Max,
	"min":  Histogram.Min,
	"mean": Histogram.Mean,
	"std":  Histogram.Stddev,
	"sum":  Histogram.Sum,
	"count": func(h Histogram) float64 {
		return float64(h.Count())
	},
}

AggregationMapping maps the string literals to the aggregation methods to be used in the trace generations and user-facing Json/OpenAPIs.

Functions

func NewLoader

func NewLoader() loader

NewLoader returns a new loader to load perf_results from the buildbucket.

This is a simple lightweight struct used to inject dependencies and for testing purposes. One can simply call: NewLoader().LoadPerfResults(...).

Types

type BuildInfo

type BuildInfo struct {
	// The swarming instance that runs this build.
	SwarmingInstance string

	// The builder name.
	BuilderName string

	// The perf machine group, see bit.ly/perf-dashboard-machine-group.
	//
	// This groups similar builders, and is defined as a builder property
	// "perf_dashboard_machine_group".
	MachineGroup string

	// The swarming task ID that runs this build.
	TaskID string

	// The git hash Revision that this build was built at.
	//
	// Note the patches and other source info is not added here as we don't need them now. More info
	// can be expanded as needed later. We should try to keep this simple.
	Revision string

	// The commit position that this build was built at.
	CommitPosisition string
}

BuildInfo contains info that are useful for identifying the perf results.

func (BuildInfo) GetPosition

func (bi BuildInfo) GetPosition() string

type Histogram

type Histogram struct {
	SampleValues []float64
}

Histogram is an individual benchmark measurement.

func (Histogram) Aggregate

func (h Histogram) Aggregate(method string) float64

func (Histogram) Count

func (h Histogram) Count() int

func (Histogram) Max

func (h Histogram) Max() float64

func (Histogram) Mean

func (h Histogram) Mean() float64

func (Histogram) Min

func (h Histogram) Min() float64

func (Histogram) Stddev

func (h Histogram) Stddev() float64

func (Histogram) Sum

func (h Histogram) Sum() float64

type PerfResults

type PerfResults struct {
	Histograms map[TraceKey]Histogram
}

PerfResults represents the contenst of a perf_results.json file generated by a telemetry-based benchmark. The full format is not formally defined, but some documnentation for it exists in various places. The most comprehensive doc is https://chromium.googlesource.com/external/github.com/catapult-project/catapult/+/HEAD/docs/Histogram-set-json-format.md

func NewResults

func NewResults(r io.Reader) (*PerfResults, error)

NewResults creates a new PerfResults from the given data stream.

It decodes the data in a streaming manner to reduce the memory footprint as the JSON files are sometimes bigger than 10MB.

func (*PerfResults) GetSampleValues

func (pr *PerfResults) GetSampleValues(chart string) []float64

GetSampleValues returns the all the sampled values for the same chart.

Deprecated: this will merge from all the stories, and this is only the results files that contain only one story. This is the only use case for cabe.

func (*PerfResults) MergeResults

func (pr *PerfResults) MergeResults(other *PerfResults)

func (*PerfResults) NonEmptyHistogramNames

func (pr *PerfResults) NonEmptyHistogramNames() []string

NonEmptyHistogramNames returns a list of names of histograms whose SampleValues arrays are non-empty.

func (*PerfResults) UnmarshalJSON

func (pr *PerfResults) UnmarshalJSON(data []byte) error

This should be deprecated in favor of streaming decoding.

UnmarshalJSON parses a byte slice into a PerfResults instance.

type RBEPerfLoader

type RBEPerfLoader struct {
	*client.Client
}

RBEPerfLoader wraps rbe.Client to provide convenient functions

func NewRBEPerfLoader

func NewRBEPerfLoader(ctx context.Context, casInstance string) (*RBEPerfLoader, error)

func (RBEPerfLoader) LoadPerfResults

func (c RBEPerfLoader) LoadPerfResults(ctx context.Context, cases ...*swarmingv2.CASReference) (map[string]*PerfResults, error)

LoadPerfResults loads all the perf_results.json from the list of CAS outputs of swarming tasks.

The CAS output should point to the root folder of the swarming task.

type TraceKey

type TraceKey struct {
	// ChartName is a specific measurement, this is also equivalent to metric.
	ChartName string `json:"chart"`

	// Unit is usually tied to the chart, we still save it for the reference.
	Unit string `json:"unit"`

	// Story is a specific user journey that collects the metrics (ChartName).
	Story string `json:"story"`

	// Architecture and OSName are defined by the BotConfig, within each perf result file,
	// this should be a unique combo throughput as it runs on a single machine. In other cases,
	// the key pair can identify a unique trace because they are running on a different machine.
	Architecture string `json:"arch"`
	OSName       string `json:"os"`
}

TraceKey is a unique identifier for one trace.

The Telemetry trace from one perf run is identified as: ChromePerf/[BotConfig]/[benchmark]/[ChartName]/[Story] where BotConfig is usually the build name that defines the bot dimension where it runs benchmark is a collection of runs ChartName is a specific measurement