statsum

README

Statistical Summarization Service

Simple service that accepts metrics such as counter and values. Counters are aggregated the usual way, but for the value metrics the service will estimate percentiles using t-digests.

For every 5 minute and 1 hour interval the service will forward estimated percentiles (and aggregated counters) to a time series service like signalfx, datadog or similar service.

Submitting Data

Data is submitted as follows using a project specific jwt-token and a request specific uuid. The uuid should be reused when a request is retried due to connection error, this avoids duplication of values.

POST /v1/project/<projectName>
content-type:         application/json
accept:               application/json
content-length:       <length>
authorization:        bearer <jwt-token>
x-statsum-request-id: <uuid>
{
  "counters": [ // values for a counter are summed up
    {"k": "name.of.counter", v: 5},
  ],
  "measures: [ // values for a meaure is fed to a t-digest for percentile estimation
    {"k": "name.of.measure", v: [3,4,5,65,5,5,7,8,9]},
  ]
}

As a rule of thumb, it is reasonable to accumulate metrics for 30 to 90 seconds before flushing. As statsum will only reports every 5min.

Motivation

Services like signalfx, datadog, stathat, etc. cannot compute or estimate percentiles. Any function in their dashboard or analysis tools pretending to do so is false.

These services aggregates data-points as averages (typically averages per second). Regardless of the resolution, "percentiles over averages" is not a thing. It is not statically sound (period!).

If this service is used to aggregate data-points before they are forwarded to signalfx, datadog or similar service, you should end up with valid estimates of the 25'th, 50'th, 75'th, 95'th and 99'th percentile.

But beware when displaying these metrics in signalfx, datadog or similar services. These services may still aggregate the numbers when rendering graphs. Any such aggregation is naturally incorrect.

Building Statsum Server

The server is located in /cmd/statsum to build it you must fetch govendor locked dependencies with govendor sync, then go build ./cmd/statsum will build the binary.

Alternatively, the make build will do the same and build the docker image too. To make a release, edit the Makefile to set a new TAG, and run make push.

Configuring the Server

When deploygin the server as built above, it requires the following environment variables:

• JWT_SECRET_KEY, symmetric secret that JWTs are signed with,
• TLS_CERTIFICATE, (optional) TLS certificate chain,
• TLS_KEY, (optional) TLS private key,
• PORT, port to listen on,
• SIGNALFX_TOKEN, signalfx token for sending metrics,
• SENTRY_DSN, sentry DSN for reporting errors,
• PUBLIC_URL, public URL to this server,

Service Owner

Service Owner: dustin@mozilla.com

Documentation

Index

• type Config
• type Configurer
  • func StaticConfigurer(baseURL string, secret []byte) Configurer
• type Options
• type ServerError
  • func (s *ServerError) Error() string
• type Statsum
  • func New(project string, configurer Configurer, options Options) *Statsum
  • func (s *Statsum) Count(name string, value float64)
  • func (s *Statsum) Flush() error
  • func (s *Statsum) Measure(name string, values ...float64)
  • func (s *Statsum) Time(name string, fn func())
  • func (s *Statsum) WithPrefix(prefix string) *Statsum

Types

type Config

type Config struct {
	Project string    `json:"project"`
	BaseURL string    `json:"baseUrl"`
	Token   string    `json:"token"`
	Expires time.Time `json:"expires"`
}

Config required for statsum client to submit metrics.

type Configurer

type Configurer func(project string) (Config, error)

A Configurer is a function that given a project name returns Config options required for submitting metrics.

func StaticConfigurer

func StaticConfigurer(baseURL string, secret []byte) Configurer

StaticConfigurer computes a JWT that is valid for 25 minutes, and rotated every 10 minutes. This allows for 15 minutes clock drift, while ensuring that all credentials being transmitted are temporary.

type Options

type Options struct {
	MaxDataPoints int           // Maximum data points before sending, default 10k
	MaxDelay      time.Duration // Maximum delay before sending, default 90s
	MinDelay      time.Duration // Minimum delay before sending, default 30s
	OnError       func(error)   // Error callback, ignored by default
}

Options for Statsum client.

type ServerError

type ServerError struct {
	Message string `json:"message"`
	Code    string `json:"code"`
}

A ServerError represents a server-side error from statsum.

func (*ServerError) Error

func (s *ServerError) Error() string

type Statsum

type Statsum struct {
	// contains filtered or unexported fields
}

Statsum client for collecting metrics

func New

func New(project string, configurer Configurer, options Options) *Statsum

New returns a new Statum client.

func (*Statsum) Count

func (s *Statsum) Count(name string, value float64)

Count increments counter by name with given value

func (*Statsum) Flush

func (s *Statsum) Flush() error

Flush will send all metrics immediately,

func (*Statsum) Measure

func (s *Statsum) Measure(name string, values ...float64)

Measure one or more values for the given name

func (*Statsum) Time

func (s *Statsum) Time(name string, fn func())

Time measures the time it takes to execute fn in milliseconds and reports it as a measure for the given name.

func (*Statsum) WithPrefix

func (s *Statsum) WithPrefix(prefix string) *Statsum

WithPrefix returns a Statsum client object that prefixes all values with the given prefix.

The new Statsum client object uses the same underlying cache as its parent, making this a cheap operation that does not number of increase submission requests, or require additional Flush calls.