benchmetric

Documentation

Overview

Package benchmetric provides utilities for benchmark metrics. This includes printing benchstat-formatted metrics and measuring container timing.

Index

• func CommandThenTimed(initCmd []string, cd string, timedCmd []string) []string
• func GetTimedContainerDuration(ctx context.Context, c *testcluster.TestCluster, pod *v13.Pod, ...) (time.Duration, error)
• func ParseTimedContainerOutput(out string) (time.Duration, error)
• func TimedCommand(argv ...string) []string
• func WithRecorder(ctx context.Context, recorder Recorder) context.Context
• type MetricValue
  • func BenchmarkDuration(duration time.Duration) MetricValue
  • func BytesPerSecond(bytesPerSecond float64) MetricValue
  • func Checksum(h hash.Hash32, thingBeingChecksummed string) MetricValue
  • func Count(numberOfTimes uint64, thingBeingCounted string) MetricValue
  • func Rate(perSecond float64, event string) MetricValue
  • func RequestsPerSecond(rps float64) MetricValue
  • func SpecificBytes(bytes float64, subset string) MetricValue
  • func SpecificDuration(duration time.Duration, subset string) MetricValue
• type Recorder
  • func GetRecorder(ctx context.Context) (Recorder, error)

Functions

func CommandThenTimed

func CommandThenTimed(initCmd []string, cd string, timedCmd []string) []string

CommandThenTimed returns a command-line that runs a given command as initialization, then cd's into the given directory, then runs another command there under TimedCommand. When using GetTimedContainerDuration, only the duration of the `timedCmd` command will be measured. If `cd` is empty, no directory change happens.

func GetTimedContainerDuration

func GetTimedContainerDuration(ctx context.Context, c *testcluster.TestCluster, pod *v13.Pod, containerName string) (time.Duration, error)

GetTimedContainerDuration waits for the given pod to exit, then parses its output and looks for duration information as expected from a command-line generated using `TimedCommand`.

func ParseTimedContainerOutput

func ParseTimedContainerOutput(out string) (time.Duration, error)

ParseTimedContainerOutput parses the output of a TimedContainer.

func TimedCommand

func TimedCommand(argv ...string) []string

TimedCommand takes in a command-line and computes a high-fidelity duration of how long it took. A pod using this command-line should have its duration measured using GetTimedContainerDuration. This is more reliable than looking at Kubernetes-tracked metrics for container start/end times, because Kubernetes only tracks these with second-level granularity, and because they include the overhead of starting/stopping a container. For benchmarks that want to measure the time of a particular command, TimedCommand provides better precision (nanosecond resolution) and accuracy (only measure the duration of the command, not the container runtime overhead). This must run in a container that has either `sh` or `bash` installed.

func WithRecorder

func WithRecorder(ctx context.Context, recorder Recorder) context.Context

WithRecorder returns a context with the given `Recorder`.

Types

type MetricValue

type MetricValue interface {
	// ToBenchstat formats a metric value according to golang's benchmark format.
	ToBenchstat() string
}

MetricValue represents an individual metric value.

func BenchmarkDuration

func BenchmarkDuration(duration time.Duration) MetricValue

BenchmarkDuration is a MetricValue for a benchmark's overall duration.

func BytesPerSecond

func BytesPerSecond(bytesPerSecond float64) MetricValue

BytesPerSecond is a MetricValue for bandwidth.

func Checksum

func Checksum(h hash.Hash32, thingBeingChecksummed string) MetricValue

Checksum is a MetricValue for a checksum that is not expected to change over iterations or variants of the benchmark.

func Count

func Count(numberOfTimes uint64, thingBeingCounted string) MetricValue

Count is a MetricValue for a quantity counted by a benchmark.

func Rate

func Rate(perSecond float64, event string) MetricValue

Rate is a MetricValue representing a rate of events happening. The rate must be given in per-second terms.

func RequestsPerSecond

func RequestsPerSecond(rps float64) MetricValue

RequestsPerSecond is a MetricValue for requests per second.

func SpecificBytes

func SpecificBytes(bytes float64, subset string) MetricValue

SpecificBytes is a MetricValue for an absolute number of bytes. Do not specify the "byte" or "B" prefix in `subset` or the function will panic.

func SpecificDuration

func SpecificDuration(duration time.Duration, subset string) MetricValue

SpecificDuration is a MetricValue for the duration of a specific subset of a benchmark. Do not specify a time unit in `subset` or the function will panic.

type Recorder

type Recorder interface {
	// Record records one or more values associated with a benchmark.
	// The benchmark is assumed to have a single iteration.
	Record(ctx context.Context, name string, values ...MetricValue) error
	// RecordIters records one or more values associated with a benchmark,
	// with an explicitly-specified number of iterations.
	RecordIters(ctx context.Context, name string, iters int, values ...MetricValue) error
}

Recorder records benchmark data. The Recorder is a singleton.

func GetRecorder

func GetRecorder(ctx context.Context) (Recorder, error)

GetRecorder returns the benchmark's `Recorder` singleton.