stats

README

stats

In-request counters and timers for golang web services

stats is designed as a performant and easy-to-use stats collector, along the lines of StatsD. Backends are pluggable by implementing the 2-method sink interface.

The stats package has been used to some degree, but hasn't been battle-tested at high scale. Field reports and improvements are welcome.

This package may be useful for guidance on a handful of topics, such as:

• Using channels and goroutines to kick off parallel processing inside an http.Request without blocking the user response.
• Writing packages that compile and run inside the Appengine Standard Environment, and also compile and run outside the Appengine Standard Environment.
• The stackdriver subpackage demonstrates usage of Stackdriver to store user-defined metrics.

Installation

go get github.com/efixler/stats

Installing stats will also install the Stackdriver backend and command-line helpers. See the stackdriver/ subfolder for details.

Usage

The usage sample below assumes Stackdriver as the storage backend for metrics data, using the included Stackdriver driver, and also assumes Gorilla Mux.

import (
  "github.com/gorilla/mux"
  "github.com/efixler/stats"
  "github.com/efixler/stats/stackdriver"
)

func init() {
  router := mux.NewRouter()
  router.Use(stats.Metrics(stackdriver.Sink))
  router.Use(stats.TimeRequests)
}

func SomeHandler(w http.ResponseWriter, r *http.Request) {
  stats.StartTimer(r.Context(), "some_handler_timer")
  // ... do work
  stats.FinishTimer(r.Context(), "some_handler_timer")
  stats.Increment(r.Context(), "some_handler_counter")
}

See the Godoc for details and more examples.

Documentation

Overview

Stats package is used to collect in-request count and timing metrics. It's influenced by StatsD and provides a similar interface.

Attach stats to a request via the router middleware functionality. The middleware APIs are developed and tested around gorilla mux, but should work with any middleware API that accepts http.Handler implementations.

stats pushes data upstream using a background goroutine. Background processing is supported in the Appengine Standard Environment, but not in all scaling modes. See https://cloud.google.com/appengine/docs/standard/go/modules/runtime#RunInBackground for more information about background activities and scaling modes in Appengine.

Backends are pluggable. A metrics storage backend need only implement the Sink interface. There is a Sink implementation for Stackdriver at https://github.com/efixler/stats/stackdriver.

See the examples for usage info and more details.

Index

• Variables
• func FinishTimer(ctx context.Context, bucket string) error
• func Increment(ctx context.Context, bucket string) error
• func Metrics(sink Sink) func(http.Handler) http.Handler
• func StartTimer(ctx context.Context, bucket string) error
• func TimeRequests(next http.Handler) http.Handler
• type Counter
  • func (m Counter) Data() int
  • func (c *Counter) Decrement()
  • func (c *Counter) Increment()
  • func (m Counter) Name() string
• type Metric
• type Sink
• type Timer
  • func (m Timer) Data() int
  • func (t *Timer) Duration() int64
  • func (t *Timer) Finish() error
  • func (t *Timer) Finished() bool
  • func (t *Timer) Milliseconds() int
  • func (m Timer) Name() string
  • func (t *Timer) Started() bool
  • func (t *Timer) String() string

Variables

var (
	TimerNotStarted          = errors.New("Timer was never started")
	TimerNotFinished         = errors.New("Timer was never finished")
	RequestMetricsNotInitted = errors.New("Request metrics were not initialized (see stats.Metrics)")

	IllegalMetricName = errors.New(fmt.Sprintf("Names must match %s and not have consecutive dots or slashes", legalMetricName))
	NoSink            = errors.New("No sink set up for storing metrics")
	NoSuchMetric      = errors.New("No metric by that name")
)

Functions

func FinishTimer

func FinishTimer(ctx context.Context, bucket string) error

Finish the timer specified by bucket. The finished timer will be forwarded to the Sink, if one has been set up.

func Increment

func Increment(ctx context.Context, bucket string) error

Increment the counter with the named bucket. Counters can be incremented multiple times within a request. The counter will get flushed when the request is finished.

Metric buckets are created on demand. Metric names can have alphanumeric characters, slashes, underscores, and dots.

func addUserHandler(w http.ResponseWriter, r *http.Request) {
  ...
  err := stats.Increment(r.Context(), "add_user")
  ...
}

Errors returned here will generally be IllegalMetricName or RequestMetricsNotInitted. Errors relating to the backend will not be reported here, as events

func Metrics

func Metrics(sink Sink) func(http.Handler) http.Handler

This is the middleware call to set up metrics for a request, probably in conjunction with Gorilla mux, as in:

router.Use(Metrics(sink))

where sink implements the Sink interface.

func StartTimer

func StartTimer(ctx context.Context, bucket string) error

Starts a timer with the named bucket. Named buckets are created on demand, and can contain alphanumeric characters, slashes, underscores, and dots.

func newUserHandler(w http.ResponseWriter, r *http.Request) {
    timerName := "new_user_time"
    err := stats.StartTimer(r.Context(), timerName)
    //... do work in here
    err := stats.FinishTimer(r.Context(), timerName)
}

Errors returned here will generally be IllegalMetricName or RequestMetricsNotInitted.

func TimeRequests

func TimeRequests(next http.Handler) http.Handler

Time every http request on this server. In many environments, this will be superfluous, and is provided mainly for testing. If you use this, always make this setup call after Use(Metrics(sink))

Types

type Counter

type Counter struct {
	// contains filtered or unexported fields
}

Counter metric. This interface is public primarily for access by Sink implementations. It is not used directly by stats event producers.

func (Counter) Data

func (m Counter) Data() int

func (*Counter) Decrement

func (c *Counter) Decrement()

func (*Counter) Increment

func (c *Counter) Increment()

func (Counter) Name

func (m Counter) Name() string

type Metric

type Metric interface {
	Name() string
	Data() int
}

Core interface for all metrics. This interface is public primarily for access by Sink implementations. It is not used directly by stats event producers.

type Sink

type Sink interface {
	WriteCounters(ctx context.Context, counters ...*Counter) error
	WriteTimers(ctx context.Context, timers ...*Timer) error
}

Implement the Sink interface to connect a metrics storage backend to the stats package. See the stackdriver package for a sample implementation.

The Sink methods accept multiple metrics variadically, to let the implementation optimize batching for its data store. A failure on one metric need not fail the entire batch -- use http://github.com/efixler/multierror to return multiple errors to the caller, which will log them.

type Timer

type Timer struct {
	// contains filtered or unexported fields
}

Timer metric. This interface is public primarily for access by Sink implementations. It is not used directly by stats event producers.

func (Timer) Data

func (m Timer) Data() int

func (*Timer) Duration

func (t *Timer) Duration() int64

Nanoseconds

func (*Timer) Finish

func (t *Timer) Finish() error

func (*Timer) Finished

func (t *Timer) Finished() bool

func (*Timer) Milliseconds

func (t *Timer) Milliseconds() int

func (Timer) Name

func (m Timer) Name() string

func (*Timer) Started

func (t *Timer) Started() bool

func (*Timer) String

func (t *Timer) String() string