metrics

package module
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 21, 2020 License: MIT Imports: 14 Imported by: 12

README

net/metrics GoDoc Build Status Coverage Status

Fast, Prometheus- and Graphite-compatible metrics.

Work in Progress

Documentation

Overview

Package metrics is a telemetry client designed for Uber's software networking team. It prioritizes performance on the hot path and integration with both push- and pull-based collection systems. Like Prometheus and Tally, it supports metrics tagged with arbitrary key-value pairs.

Metric Names and Uniqueness

Like Prometheus, but unlike Tally, metric names should be relatively long and descriptive - generally speaking, metrics from the same process shouldn't share names. (See the documentation for the Root struct below for a longer explanation of the uniqueness rules.) For example, prefer "grpc_successes_by_procedure" over "successes", since "successes" is common and vague. Where relevant, metric names should indicate their unit of measurement (e.g., "grpc_success_latency_ms").

Counters and Gauges

Counters represent monotonically increasing values, like a car's odometer. Gauges represent point-in-time readings, like a car's speedometer. Both counters and gauges expose not only write operations (set, add, increment, etc.), but also atomic reads. This makes them easy to integrate directly into your business logic: you can use them anywhere you'd otherwise use a 64-bit atomic integer.

Histograms

This package doesn't support analogs of Tally's timer or Prometheus's summary, because they can't be accurately aggregated at query time. Instead, it approximates distributions of values with histograms. These require more up-front work to set up, but are typically more accurate and flexible when queried. See https://prometheus.io/docs/practices/histograms/ for a more detailed discussion of the trade-offs involved.

Vectors

Plain counters, gauges, and histograms have a fixed set of tags. However, it's common to encounter situations where a subset of a metric's tags vary constantly. For example, you might want to track the latency of your database queries by table: you know the database cluster, application name, and hostname at process startup, but you need to specify the table name with each query. To model these situations, this package uses vectors.

Each vector is a local cache of metrics, so accessing them is quite fast. Within a vector, all metrics share a common set of constant tags and a list of variable tags. In our database query example, the constant tags are cluster, application, and hostname, and the only variable tag is table name. Usage examples are included in the documentation for each vector type.

Push and Pull

This package integrates with StatsD- and M3-based collection systems by periodically pushing differential updates. (Users can integrate with other push-based systems by implementing the push.Target interface.) It integrates with pull-based collectors by exposing an HTTP handler that supports Prometheus's text and protocol buffer exposition formats. Examples of both push and pull integration are included in the documentation for the root struct's Push and ServeHTTP methods.

See Also

If you're unfamiliar with Tally and Prometheus, you may want to consult their documentation: 

https://godoc.org/github.com/uber-go/tally
https://godoc.org/github.com/prometheus/client_golang/prometheus
Example 
package main

import (
	"fmt"

	"go.uber.org/net/metrics"
)

func main() {
	// First, construct a metrics root. Generally, there's only one root in each
	// process.
	root := metrics.New()
	// From the root, access the top-level scope and add some tags to create a
	// sub-scope. You'll typically pass scopes around your application, since
	// they let you create individual metrics.
	scope := root.Scope().Tagged(metrics.Tags{
		"host":   "db01",
		"region": "us-west",
	})

	// Create a simple counter. Note that the name is fairly long; if this code
	// were part of a reusable library called "foo", "foo_selects_completed"
	// would be a much better name.
	total, err := scope.Counter(metrics.Spec{
		Name: "selects_completed",
		Help: "Total number of completed SELECT queries.",
	})
	if err != nil {
		panic(err)
	}

	// See the package-level documentation for a general discussion of vectors.
	// In this case, we're going to track the number of in-progress SELECT
	// queries by table and user. Since we won't know the table and user names
	// until we actually receive each query, we model this as a vector with two
	// variable tags.
	progress, err := scope.GaugeVector(metrics.Spec{
		Name:    "selects_in_progress",
		Help:    "Number of in-progress SELECT queries.",
		VarTags: []string{"table", "user"},
	})
	if err != nil {
		panic(err)
	}
	// MustGet retrieves the gauge with the specified variable tags, creating
	// one if necessary. We must supply both the variable tag names and values,
	// and they must be in the correct order. MustGet panics only if the tags
	// are malformed. If you'd rather check errors explicitly, there's also a
	// Get method.
	trips := progress.MustGet(
		"table" /* tag name */, "trips", /* tag value */
		"user" /* tag name */, "jane", /* tag value */
	)
	drivers := progress.MustGet(
		"table", "drivers",
		"user", "chen",
	)

	fmt.Println("Trips:", trips.Inc())
	total.Inc()
	fmt.Println("Drivers:", drivers.Add(2))
	total.Add(2)
	fmt.Println("Drivers:", drivers.Dec())
	fmt.Println("Trips:", trips.Dec())
	fmt.Println("Total:", total.Load())

}

Output:

Trips: 1
Drivers: 2
Drivers: 1
Trips: 0
Total: 3

Index

Examples

Constants

View Source 
const (
	DefaultTagName  = "default"
	DefaultTagValue = "default"
)

Placeholders for empty tag names and values.

View Source 
const Version = "1.3.0"

Version is the current semantic version, exported for runtime compatibility checks.

Variables

This section is empty.

Functions

func IsValidName 

func IsValidName(s string) bool

IsValidName checks whether the supplied string is a valid metric and tag name in both Prometheus and Tally.

Tally and Prometheus each allow runes that the other doesn't, so this package can accept only the common subset. For simplicity, we'd also like the rules for metric names and tag names to be the same even if that's more restrictive than absolutely necessary.

Tally allows anything matching the regexp `^[0-9A-z_\-]+$`. Prometheus allows the regexp `^[A-z_:][0-9A-z_:]*$` for metric names, and `^[A-z_][0-9A-z_]*$` for tag names.

The common subset is `^[A-z_][0-9A-z_]*$`.

func IsValidTagValue 

func IsValidTagValue(s string) bool

IsValidTagValue checks whether the supplied string is a valid tag value in both Prometheus and Tally.

Tally allows tag values that match the regexp `^[0-9A-z_\-.]+$`. Prometheus allows any valid UTF-8 string.

Types

type Counter 

type Counter struct {
	// contains filtered or unexported fields
}

A Counter is a monotonically increasing value, like a car's odometer. All its exported methods are safe to use concurrently, and nil *Counters are safe no-op implementations.

Example 
package main

import (
	"go.uber.org/net/metrics"
)

func main() {
	c, err := metrics.New().Scope().Counter(metrics.Spec{
		Name:      "selects_completed",                         // required
		Help:      "Total number of completed SELECT queries.", // required
		ConstTags: metrics.Tags{"host": "db01"},                // optional
	})
	if err != nil {
		panic(err)
	}
	c.Add(2)
}

Output:

func (*Counter) Add 

func (c *Counter) Add(n int64) int64

Add increases the value of the counter and returns the new value. Since counters must be monotonically increasing, passing a negative number just returns the current value (without modifying it).

func (*Counter) Inc 

func (c *Counter) Inc() int64

Inc increments the counter's value by one and returns the new value.

func (*Counter) Load 

func (c *Counter) Load() int64

Load returns the counter's current value.

type CounterVector 

type CounterVector struct {
	// contains filtered or unexported fields
}

A CounterVector is a collection of Counters that share a name and some constant tags, but also have a consistent set of variable tags. All exported methods are safe to use concurrently. Nil *CounterVectors are safe to use and always return no-op counters.

For a general description of vector types, see the package-level documentation.

Example 
package main

import (
	"go.uber.org/net/metrics"
)

func main() {
	vec, err := metrics.New().Scope().CounterVector(metrics.Spec{
		Name:      "selects_completed_by_table",                   // required
		Help:      "Number of completed SELECT queries by table.", // required
		ConstTags: metrics.Tags{"host": "db01"},                   // optional
		VarTags:   []string{"table"},                              // required
	})
	if err != nil {
		panic(err)
	}
	vec.MustGet("table" /* tag name */, "trips" /* tag value */).Inc()
}

Output:

func (*CounterVector) Get 

func (cv *CounterVector) Get(variableTagPairs ...string) (*Counter, error)

Get retrieves the counter with the supplied variable tag names and values from the vector, creating one if necessary. The variable tags must be supplied in the same order used when creating the vector.

Get returns an error if the number or order of tags is incorrect.

func (*CounterVector) MustGet 

func (cv *CounterVector) MustGet(variableTagPairs ...string) *Counter

MustGet behaves exactly like Get, but panics on errors. If code using this method is covered by unit tests, this is safe.

type Gauge 

type Gauge struct {
	// contains filtered or unexported fields
}

A Gauge is a point-in-time measurement, like a car's speedometer. All its exported methods are safe to use concurrently, and nil *Gauges are safe no-op implementations.

Example 
package main

import (
	"go.uber.org/net/metrics"
)

func main() {
	g, err := metrics.New().Scope().Gauge(metrics.Spec{
		Name:      "selects_in_progress",                       // required
		Help:      "Total number of in-flight SELECT queries.", // required
		ConstTags: metrics.Tags{"host": "db01"},                // optional
	})
	if err != nil {
		panic(err)
	}
	g.Store(11)
}

Output:

func (*Gauge) Add 

func (g *Gauge) Add(n int64) int64

Add increases the value of the gauge and returns the new value. Adding negative values is allowed, but using Sub may be simpler.

func (*Gauge) CAS 

func (g *Gauge) CAS(old, new int64) bool

CAS is an atomic compare-and-swap. It compares the current value to the old value supplied, and if they match it stores the new value. The return value indicates whether the swap succeeded. To avoid endless CAS loops, no-op gauges always return true.

func (*Gauge) Dec 

func (g *Gauge) Dec() int64

Dec decrements the gauge's current value by one and returns the new value.

func (*Gauge) Inc 

func (g *Gauge) Inc() int64

Inc increments the gauge's current value by one and returns the new value.

func (*Gauge) Load 

func (g *Gauge) Load() int64

Load returns the gauge's current value.

func (*Gauge) Store 

func (g *Gauge) Store(n int64)

Store sets the gauge's value.

func (*Gauge) Sub 

func (g *Gauge) Sub(n int64) int64

Sub decreases the value of the gauge and returns the new value. Subtracting negative values is allowed, but using Add may be simpler.

func (*Gauge) Swap 

func (g *Gauge) Swap(n int64) int64

Swap replaces the gauge's current value and returns the previous value.

type GaugeVector 

type GaugeVector struct {
	// contains filtered or unexported fields
}

A GaugeVector is a collection of Gauges that share a name and some constant tags, but also have a consistent set of variable tags. All exported methods are safe to use concurrently. Nil *GaugeVectors are safe to use and always return no-op gauges.

For a general description of vector types, see the package-level documentation.

Example 
package main

import (
	"go.uber.org/net/metrics"
)

func main() {
	vec, err := metrics.New().Scope().GaugeVector(metrics.Spec{
		Name:      "selects_in_progress_by_table",                 // required
		Help:      "Number of in-flight SELECT queries by table.", // required
		ConstTags: metrics.Tags{"host": "db01"},                   // optional
		VarTags:   []string{"table"},                              // optional
	})
	if err != nil {
		panic(err)
	}
	vec.MustGet("table" /* tag name */, "trips" /* tag value */).Store(11)
}

Output:

func (*GaugeVector) Get 

func (gv *GaugeVector) Get(variableTagPairs ...string) (*Gauge, error)

Get retrieves the gauge with the supplied variable tags names and values from the vector, creating one if necessary. The variable tags must be supplied in the same order used when creating the vector.

Get returns an error if the number or order of tags is incorrect.

func (*GaugeVector) MustGet 

func (gv *GaugeVector) MustGet(variableTagPairs ...string) *Gauge

MustGet behaves exactly like Get, but panics on errors. If code using this method is covered by unit tests, this is safe.

type Histogram 

type Histogram struct {
	// contains filtered or unexported fields
}

A Histogram approximates a distribution of values. They're both more efficient and easier to aggregate than Prometheus summaries or M3 timers. For a discussion of the tradeoffs between histograms and timers/summaries, see https://prometheus.io/docs/practices/histograms/.

All exported methods are safe to use concurrently, and nil *Histograms are valid no-op implementations.

Example 
package main

import (
	"time"

	"go.uber.org/net/metrics"
)

func main() {
	h, err := metrics.New().Scope().Histogram(metrics.HistogramSpec{
		Spec: metrics.Spec{
			Name:      "selects_latency_ms",         // required, should indicate unit
			Help:      "SELECT query latency.",      // required
			ConstTags: metrics.Tags{"host": "db01"}, // optional
		},
		Unit:    time.Millisecond,                      // required
		Buckets: []int64{5, 10, 25, 50, 100, 200, 500}, // required
	})
	if err != nil {
		panic(err)
	}
	h.Observe(37 * time.Millisecond) // increments bucket with upper bound 50
	h.IncBucket(37)                  // also increments bucket with upper bound 50
}

Output:

func (*Histogram) IncBucket 

func (h *Histogram) IncBucket(n int64)

IncBucket bypasses the time-based Observe API and increments a histogram bucket directly. It finds the correct bucket for the supplied value and adds one to its counter.

func (*Histogram) Observe 

func (h *Histogram) Observe(d time.Duration)

Observe finds the correct bucket for the supplied duration and increments its counter. This is purely a convenience - it's equivalent to dividing the duration by the histogram's unit and calling IncBucket directly.

type HistogramSnapshot 

type HistogramSnapshot struct {
	Name   string
	Tags   Tags
	Unit   time.Duration
	Values []int64 // rounded up to bucket upper bounds
}

A HistogramSnapshot is a point-in-time view of the state of a Histogram.

type HistogramSpec 

type HistogramSpec struct {
	Spec

	// Durations are exposed as simple numbers, not strings or rich objects.
	// Unit specifies the desired granularity for histogram observations. For
	// example, an observation of time.Second with a unit of time.Millisecond is
	// exposed as 1000. Typically, the unit should also be part of the metric
	// name.
	Unit time.Duration
	// Upper bounds (inclusive) for the histogram buckets in terms of the unit.
	// A catch-all bucket for large observations is automatically created, if
	// necessary.
	Buckets []int64
}

A HistogramSpec configures Histograms and HistogramVectors.

type HistogramVector 

type HistogramVector struct {
	// contains filtered or unexported fields
}

A HistogramVector is a collection of Histograms that share a name and some constant tags, but also have a consistent set of variable tags. All exported methods are safe to use concurrently. Nil *HistogramVectors are safe to use and always return no-op histograms.

For a general description of vector types, see the package-level documentation.

Example 
package main

import (
	"time"

	"go.uber.org/net/metrics"
)

func main() {
	vec, err := metrics.New().Scope().HistogramVector(metrics.HistogramSpec{
		Spec: metrics.Spec{
			Name:      "selects_latency_by_table_ms",    // required, should indicate unit
			Help:      "SELECT query latency by table.", // required
			ConstTags: metrics.Tags{"host": "db01"},     // optional
			VarTags:   []string{"table"},
		},
		Unit:    time.Millisecond,                      // required
		Buckets: []int64{5, 10, 25, 50, 100, 200, 500}, // required
	})
	if err != nil {
		panic(err)
	}
	vec.MustGet("table" /* tag name */, "trips" /* tag value */).Observe(37 * time.Millisecond)
}

Output:

func (*HistogramVector) Get 

func (hv *HistogramVector) Get(variableTagPairs ...string) (*Histogram, error)

Get retrieves the histogram with the supplied variable tag names and values from the vector, creating one if necessary. The variable tags must be supplied in the same order used when creating the vector.

Get returns an error if the number or order of tags is incorrect.

func (*HistogramVector) MustGet 

func (hv *HistogramVector) MustGet(variableTagPairs ...string) *Histogram

MustGet behaves exactly like Get, but panics on errors. If code using this method is covered by unit tests, this is safe.

type Option 

type Option interface {
	// contains filtered or unexported methods
}

An Option configures a root. Currently, there are no exported options.

type Root 

type Root struct {
	// contains filtered or unexported fields
}

A Root is a collection of tagged metrics that can be exposed via in-memory snapshots, push-based telemetry systems, or a Prometheus-compatible HTTP handler.

Within a root, metrics must obey two uniqueness constraints. First, any two metrics with the same name must have the same tag names (both constant and variable). Second, no two metrics can share the same name, constant tag names, and constant tag values. Functionally, users of this package can avoid collisions by using descriptive metric names that begin with a component or subsystem name. For example, prefer "grpc_successes_by_procedure" over "successes".

func New 

func New(opts ...Option) *Root

New constructs a root.

func (*Root) Push 

func (r *Root) Push(target push.Target, tick time.Duration) (context.CancelFunc, error)

Push starts a goroutine that periodically exports all registered metrics to the supplied target. Roots may only push to a single target at a time; to push to multiple backends simultaneously, implement a teeing push.Target.

The returned function cleanly shuts down the background goroutine.

Example 
package main

import (
	"fmt"
	"time"

	"github.com/uber-go/tally"
	"go.uber.org/net/metrics"
	"go.uber.org/net/metrics/tallypush"
)

func main() {
	// First, we need something to push to. In this example, we'll use Tally's
	// testing scope.
	ts := tally.NewTestScope("" /* prefix */, nil /* tags */)
	root := metrics.New()

	// Push updates to our test scope twice per second.
	stop, err := root.Push(tallypush.New(ts), 500*time.Millisecond)
	if err != nil {
		panic(err)
	}
	defer stop()

	c, err := root.Scope().Counter(metrics.Spec{
		Name: "example",
		Help: "Counter demonstrating push integration.",
	})
	if err != nil {
		panic(err)
	}
	c.Inc()

	// Sleep to make sure that we run at least one push, then print the counter
	// value as seen by Tally.
	time.Sleep(2 * time.Second)
	fmt.Println(ts.Snapshot().Counters()["example+"].Value())

}

Output:

1

func (*Root) Scope 

func (r *Root) Scope() *Scope

Scope exposes the root's top-level metrics collection. Tagged sub-scopes and individual counters, gauges, histograms, and vectors can be created from this top-level Scope.

func (*Root) ServeHTTP 

func (r *Root) ServeHTTP(w http.ResponseWriter, req *http.Request)

ServeHTTP implements a Prometheus-compatible http.Handler that exposes the current value of all the metrics created with this Root (including all tagged sub-scopes). Like the HTTP handler included in the Prometheus client, it uses content-type negotiation to determine whether to use a text or protocol buffer encoding.

In particular, it's compatible with the standard Prometheus server's scraping logic.

Example 
package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
	"net/http/httptest"

	"go.uber.org/net/metrics"
)

func main() {
	// First, construct a root and add some metrics.
	root := metrics.New()
	c, err := root.Scope().Counter(metrics.Spec{
		Name:      "example",
		Help:      "Counter demonstrating HTTP exposition.",
		ConstTags: metrics.Tags{"host": "example01"},
	})
	if err != nil {
		panic(err)
	}
	c.Inc()

	// Expose the root on your HTTP server of choice.
	mux := http.NewServeMux()
	mux.Handle("/debug/net/metrics", root)
	srv := httptest.NewServer(mux)
	defer srv.Close()

	// Your metrics are now exposed via a Prometheus-compatible handler. This
	// example shows text output, but clients can also request the protocol
	// buffer binary format.
	res, err := http.Get(fmt.Sprintf("%v/debug/net/metrics", srv.URL))
	if err != nil {
		panic(err)
	}
	text, err := ioutil.ReadAll(res.Body)
	res.Body.Close()
	if err != nil {
		panic(err)
	}
	fmt.Println(string(text))

}

Output:

# HELP example Counter demonstrating HTTP exposition.
# TYPE example counter
example{host="example01"} 1

func (*Root) Snapshot 

func (r *Root) Snapshot() *RootSnapshot

Snapshot returns a point-in-time view of all the metrics contained in the root (and all its scopes). It's safe to use concurrently, but is relatively expensive and designed for use in unit tests.

Example 
package main

import (
	"fmt"
	"reflect"

	"go.uber.org/net/metrics"
)

func main() {
	// Snapshots are the simplest way to unit test your metrics. A future
	// release will add a more full-featured metricstest package.
	root := metrics.New()
	c, err := root.Scope().Counter(metrics.Spec{
		Name:      "example",
		Help:      "Counter demonstrating snapshots.",
		ConstTags: metrics.Tags{"foo": "bar"},
	})
	if err != nil {
		panic(err)
	}
	c.Inc()

	// It's safe to snapshot your metrics in production, but keep in mind that
	// taking a snapshot is relatively slow and expensive.
	actual := root.Snapshot().Counters[0]
	expected := metrics.Snapshot{
		Name:  "example",
		Value: 1,
		Tags:  metrics.Tags{"foo": "bar"},
	}
	if !reflect.DeepEqual(expected, actual) {
		panic(fmt.Sprintf("expected %v, got %v", expected, actual))
	}
}

Output:

type RootSnapshot 

type RootSnapshot struct {
	Counters   []Snapshot
	Gauges     []Snapshot
	Histograms []HistogramSnapshot
}

A RootSnapshot exposes all the metrics contained in a Root and all its Scopes. It's useful in tests, but relatively expensive to construct.

type Scope 

type Scope struct {
	// contains filtered or unexported fields
}

A Scope is a collection of tagged metrics.

func (*Scope) Counter 

func (s *Scope) Counter(spec Spec) (*Counter, error)

Counter constructs a new Counter.

func (*Scope) CounterVector 

func (s *Scope) CounterVector(spec Spec) (*CounterVector, error)

CounterVector constructs a new CounterVector.

func (*Scope) Gauge 

func (s *Scope) Gauge(spec Spec) (*Gauge, error)

Gauge constructs a new Gauge.

func (*Scope) GaugeVector 

func (s *Scope) GaugeVector(spec Spec) (*GaugeVector, error)

GaugeVector constructs a new GaugeVector.

func (*Scope) Histogram 

func (s *Scope) Histogram(spec HistogramSpec) (*Histogram, error)

Histogram constructs a new Histogram.

func (*Scope) HistogramVector 

func (s *Scope) HistogramVector(spec HistogramSpec) (*HistogramVector, error)

HistogramVector constructs a new HistogramVector.

func (*Scope) Tagged 

func (s *Scope) Tagged(tags Tags) *Scope

Tagged creates a new scope with new constant tags merged into the existing tags (if any). Tag names and values are automatically scrubbed, with invalid characters replaced by underscores.

type Snapshot 

type Snapshot struct {
	Name  string
	Tags  Tags
	Value int64
}

A Snapshot is a point-in-time view of the state of any non-histogram metric.

type Spec 

type Spec struct {
	Name        string   // required: metric name, should be fairly long and descriptive
	Help        string   // required: displayed on HTTP pages
	ConstTags   Tags     // optional: constant tags
	VarTags     []string // variable tags, required for vectors and forbidden otherwise
	DisablePush bool     // reduces load on system we're pushing to (if any)
}

A Spec configures Counters, Gauges, CounterVectors, and GaugeVectors.

type Tags 

type Tags map[string]string

Tags describe the dimensions of a metric.

Source Files

View all Source files

Directories

Path Synopsis
Package bucket provides utility functions for constructing and merging histogram buckets.
 Package bucket provides utility functions for constructing and merging histogram buckets.
Package push integrates go.uber.org/net/metrics with push-based telemetry systems like Graphite and M3.
 Package push integrates go.uber.org/net/metrics with push-based telemetry systems like Graphite and M3.
Package tallypush integrates go.uber.org/net/metrics with push-based StatsD and M3 systems.
 Package tallypush integrates go.uber.org/net/metrics with push-based StatsD and M3 systems.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.