bench

README

bench

Bench is a generic latency benchmarking library. It's generic in the sense that it exposes a simple interface (Requester) which can be implemented for various systems under test. Several example Requesters are provided out of the box.

Bench works by attempting to issue a fixed rate of requests per second and measuring the latency of each request issued synchronously. Latencies are captured using HDR Histogram, which observes the complete latency distribution and attempts to correct for Coordinated Omission. It provides facilities to generate output which can be plotted to produce graphs like the following:

Example Benchmark

package main

import (
	"fmt"
	"time"

	"github.com/tylertreat/bench"
	"github.com/tylertreat/bench/requester"
)

func main() {
	r := &requester.RedisPubSubRequesterFactory{
		URL:         ":6379",
		PayloadSize: 500,
		Channel:     "benchmark",
	}

	benchmark := bench.NewBenchmark(r, 10000, 1, 30*time.Second, 0)
	summary, err := benchmark.Run()
	if err != nil {
		panic(err)
	}

	fmt.Println(summary)
	summary.GenerateLatencyDistribution(nil, "redis.txt")
}

Documentation

Overview

Package bench provides a generic framework for performing latency benchmarks.

Index

• type Benchmark
  • func NewBenchmark(factory RequesterFactory, requestRate, connections uint64, ...) *Benchmark
  • func (b *Benchmark) Run() (*Summary, error)
• type Requester
• type RequesterFactory
• type Summary
  • func (s *Summary) GenerateErrorLatencyDistribution(percentiles histwriter.Percentiles, file string) error
  • func (s *Summary) GenerateLatencyDistribution(percentiles histwriter.Percentiles, file string) error
  • func (s *Summary) String() string

Types

type Benchmark

type Benchmark struct {
	// contains filtered or unexported fields
}

Benchmark performs a system benchmark by attempting to issue requests at a specified rate and capturing the latency distribution. The request rate is divided across the number of configured connections.

func NewBenchmark

func NewBenchmark(factory RequesterFactory, requestRate, connections uint64,
	duration time.Duration, burst uint64) *Benchmark

NewBenchmark creates a Benchmark which runs a system benchmark using the given RequesterFactory. The requestRate argument specifies the number of requests per second to issue. This value is divided across the number of connections specified, so if requestRate is 50,000 and connections is 10, each connection will attempt to issue 5,000 requests per second. A zero value disables rate limiting entirely. The duration argument specifies how long to run the benchmark. Requests will be issued in bursts with the specified burst rate. If burst == 0 then burst will be the lesser of (0.1 * requestRate) and 1000 but at least 1.

func (*Benchmark) Run

func (b *Benchmark) Run() (*Summary, error)

Run the benchmark and return a summary of the results. An error is returned if something went wrong along the way.

type Requester

type Requester interface {
	// Setup prepares the Requester for benchmarking.
	Setup() error

	// Request performs a synchronous request to the system under test.
	Request() error

	// Teardown is called upon benchmark completion.
	Teardown() error
}

Requester synchronously issues requests for a particular system under test.

type RequesterFactory

type RequesterFactory interface {
	// GetRequester returns a new Requester, called for each Benchmark
	// connection.
	GetRequester(number uint64) Requester
}

RequesterFactory creates new Requesters.

type Summary

type Summary struct {
	Connections                 uint64
	RequestRate                 uint64
	SuccessTotal                uint64
	ErrorTotal                  uint64
	TimeElapsed                 time.Duration
	SuccessHistogram            *hdrhistogram.Histogram
	UncorrectedSuccessHistogram *hdrhistogram.Histogram
	ErrorHistogram              *hdrhistogram.Histogram
	UncorrectedErrorHistogram   *hdrhistogram.Histogram
	Throughput                  float64
}

Summary contains the results of a Benchmark run.

func (*Summary) GenerateErrorLatencyDistribution

func (s *Summary) GenerateErrorLatencyDistribution(percentiles histwriter.Percentiles, file string) error

GenerateErrorLatencyDistribution generates a text file containing the specified latency distribution (of requests that resulted in errors) in a format plottable by http://hdrhistogram.github.io/HdrHistogram/plotFiles.html. Percentiles is a list of percentiles to include, e.g. 10.0, 50.0, 99.0, 99.99, etc. If percentiles is nil, it defaults to a logarithmic percentile scale. If a request rate was specified for the benchmark, this will also generate an uncorrected distribution file which does not account for coordinated omission.

func (*Summary) GenerateLatencyDistribution

func (s *Summary) GenerateLatencyDistribution(percentiles histwriter.Percentiles, file string) error

GenerateLatencyDistribution generates a text file containing the specified latency distribution in a format plottable by http://hdrhistogram.github.io/HdrHistogram/plotFiles.html. Percentiles is a list of percentiles to include, e.g. 10.0, 50.0, 99.0, 99.99, etc. If percentiles is nil, it defaults to a logarithmic percentile scale. If a request rate was specified for the benchmark, this will also generate an uncorrected distribution file which does not account for coordinated omission.

func (*Summary) String

func (s *Summary) String() string

String returns a stringified version of the Summary.