metrics

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 8, 2022 License: Apache-2.0, MIT Imports: 21 Imported by: 4

README

go-metrics

This library provides a metrics package which can be used to instrument code, expose application metrics, and profile runtime performance in a flexible manner.

Current API: GoDoc

Sinks

The metrics package makes use of a MetricSink interface to support delivery to any type of backend. Currently the following sinks are provided:

  • StatsiteSink : Sinks to a statsite instance (TCP)
  • StatsdSink: Sinks to a StatsD / statsite instance (UDP)
  • PrometheusSink: Sinks to a Prometheus metrics endpoint (exposed via HTTP for scrapes)
  • InmemSink : Provides in-memory aggregation, can be used to export stats
  • FanoutSink : Sinks to multiple sinks. Enables writing to multiple statsite instances for example.
  • BlackholeSink : Sinks to nowhere

In addition to the sinks, the InmemSignal can be used to catch a signal, and dump a formatted output of recent metrics. For example, when a process gets a SIGUSR1, it can dump to stderr recent performance metrics for debugging.

Labels

Most metrics do have an equivalent ending with WithLabels, such methods allow to push metrics with labels and use some features of underlying Sinks (ex: translated into Prometheus labels).

Since some of these labels may increase greatly cardinality of metrics, the library allow to filter labels using a blacklist/whitelist filtering system which is global to all metrics.

  • If Config.AllowedLabels is not nil, then only labels specified in this value will be sent to underlying Sink, otherwise, all labels are sent by default.
  • If Config.BlockedLabels is not nil, any label specified in this value will not be sent to underlying Sinks.

By default, both Config.AllowedLabels and Config.BlockedLabels are nil, meaning that no tags are filetered at all, but it allow to a user to globally block some tags with high cardinality at application level.

Examples

Here is an example of using the package:

func SlowMethod() {
    // Profiling the runtime of a method
    defer metrics.MeasureSince([]string{"SlowMethod"}, time.Now())
}

// Configure a statsite sink as the global metrics sink
sink, _ := metrics.NewStatsiteSink("statsite:8125")
metrics.NewGlobal(metrics.DefaultConfig("service-name"), sink)

// Emit a Key/Value pair
metrics.EmitKey([]string{"questions", "meaning of life"}, 42)

Here is an example of setting up a signal handler:

// Setup the inmem sink and signal handler
inm := metrics.NewInmemSink(10*time.Second, time.Minute)
sig := metrics.DefaultInmemSignal(inm)
metrics.NewGlobal(metrics.DefaultConfig("service-name"), inm)

// Run some code
inm.SetGauge([]string{"foo"}, 42)
inm.EmitKey([]string{"bar"}, 30)

inm.IncrCounter([]string{"baz"}, 42)
inm.IncrCounter([]string{"baz"}, 1)
inm.IncrCounter([]string{"baz"}, 80)

inm.AddSample([]string{"method", "wow"}, 42)
inm.AddSample([]string{"method", "wow"}, 100)
inm.AddSample([]string{"method", "wow"}, 22)

....

When a signal comes in, output like the following will be dumped to stderr:

[2014-01-28 14:57:33.04 -0800 PST][G] 'foo': 42.000
[2014-01-28 14:57:33.04 -0800 PST][P] 'bar': 30.000
[2014-01-28 14:57:33.04 -0800 PST][C] 'baz': Count: 3 Min: 1.000 Mean: 41.000 Max: 80.000 Stddev: 39.509
[2014-01-28 14:57:33.04 -0800 PST][S] 'method.wow': Count: 3 Min: 22.000 Mean: 54.667 Max: 100.000 Stddev: 40.513

Documentation

Index

Constants

View Source 
const (
	// DefaultSignal is used with DefaultInmemSignal
	DefaultSignal = syscall.SIGUSR1
)

Variables

View Source 
var (
	// DefaultPrometheusOpts is the default set of options used when creating a
	// PrometheusSink.
	DefaultPrometheusOpts = PrometheusOpts{
		Expiration: 60 * time.Second,
	}
)

Functions

func AddSample 

func AddSample(key []string, val float32, tags ...Tag)

AddSample is for timing information, where quantiles are used

func IncrCounter 

func IncrCounter(key []string, val float32, tags ...Tag)

IncrCounter should accumulate values

func MeasureSince 

func MeasureSince(key []string, start time.Time, tags ...Tag)

MeasureSince is for timing information

func SetGauge 

func SetGauge(key []string, val float32, tags ...Tag)

SetGauge should retain the last value it is set to

func UpdateFilter 

func UpdateFilter(allow, block []string)

UpdateFilter updates filters

func UpdateFilterAndLabels 

func UpdateFilterAndLabels(allow, block, allowedLabels, blockedLabels []string)

UpdateFilterAndLabels set allow/block prefixes of metrics while allowedLabels and blockedLabels - when not nil - allow filtering of labels in order to block/allow globally labels (especially useful when having large number of values for a given label). See README.md for more information about usage.

Types

type AggregateSample 

type AggregateSample struct {
	Count       int       // The count of emitted pairs
	Rate        float64   // The values rate per time unit (usually 1 second)
	Sum         float64   // The sum of values
	SumSq       float64   `json:"-"` // The sum of squared values
	Min         float64   // Minimum value
	Max         float64   // Maximum value
	LastUpdated time.Time `json:"-"` // When value was last updated
}

AggregateSample is used to hold aggregate metrics about a sample

func (*AggregateSample) Ingest 

func (a *AggregateSample) Ingest(v float64, rateDenom float64)

Ingest is used to update a sample

func (*AggregateSample) Mean 

func (a *AggregateSample) Mean() float64

Mean computes a mean of the values

func (*AggregateSample) Stddev 

func (a *AggregateSample) Stddev() float64

Stddev computes a Stddev of the values

func (*AggregateSample) String 

func (a *AggregateSample) String() string

type BlackholeSink 

type BlackholeSink struct{}

BlackholeSink is used to just blackhole messages

func (*BlackholeSink) AddSample 

func (*BlackholeSink) AddSample(key []string, val float32, tags []Tag)

AddSample is for timing information, where quantiles are used

func (*BlackholeSink) IncrCounter 

func (*BlackholeSink) IncrCounter(key []string, val float32, tags []Tag)

IncrCounter should accumulate values

func (*BlackholeSink) SetGauge 

func (*BlackholeSink) SetGauge(key []string, val float32, tags []Tag)

SetGauge should retain the last value it is set to

type Config 

type Config struct {
	ServiceName          string        // Prefixed with keys to separate services
	HostName             string        // Hostname to use. If not provided and EnableHostname, it will be os.Hostname
	EnableHostname       bool          // Enable prefixing gauge values with hostname
	EnableHostnameLabel  bool          // Enable adding hostname to labels
	EnableServiceLabel   bool          // Enable adding service to labels
	EnableRuntimeMetrics bool          // Enables profiling of runtime metrics (GC, Goroutines, Memory)
	EnableTypePrefix     bool          // Prefixes key with a type ("counter", "gauge", "timer")
	TimerGranularity     time.Duration // Granularity of timers.
	ProfileInterval      time.Duration // Interval to profile runtime metrics
	GlobalTags           []Tag         // Tags to add to every metric
	GlobalPrefix         string        // Prefix to add to every metric

	AllowedPrefixes []string // A list of metric prefixes to allow, with '.' as the separator
	BlockedPrefixes []string // A list of metric prefixes to block, with '.' as the separator
	AllowedLabels   []string // A list of metric labels to allow, with '.' as the separator
	BlockedLabels   []string // A list of metric labels to block, with '.' as the separator
	FilterDefault   bool     // Whether to allow metrics by default
}

Config is used to configure metrics settings

func DefaultConfig 

func DefaultConfig(serviceName string) *Config

DefaultConfig provides a sane default configuration

type DogStatsdSink 

type DogStatsdSink struct {
	// contains filtered or unexported fields
}

DogStatsdSink provides a MetricSink that can be used with a dogstatsd server. It utilizes the Dogstatsd client at github.com/DataDog/datadog-go/statsd

func NewDogStatsdSink 

func NewDogStatsdSink(addr string, hostName string) (*DogStatsdSink, error)

NewDogStatsdSink is used to create a new DogStatsdSink with sane defaults

func (*DogStatsdSink) AddSample 

func (s *DogStatsdSink) AddSample(key []string, val float32, tags []Tag)

AddSample is for timing information, where quantiles are used

func (*DogStatsdSink) EnableHostNamePropagation 

func (s *DogStatsdSink) EnableHostNamePropagation()

EnableHostNamePropagation forces a Dogstatsd `host` tag with the value specified by `s.HostName` Since the go-metrics package has its own mechanism for attaching a hostname to metrics, setting the `propagateHostname` flag ensures that `s.HostName` overrides the host tag naively set by the DogStatsd server

func (*DogStatsdSink) IncrCounter 

func (s *DogStatsdSink) IncrCounter(key []string, val float32, tags []Tag)

IncrCounter should accumulate values

func (*DogStatsdSink) SetGauge 

func (s *DogStatsdSink) SetGauge(key []string, val float32, tags []Tag)

SetGauge should retain the last value it is set to

func (*DogStatsdSink) SetTags 

func (s *DogStatsdSink) SetTags(tags []string)

SetTags sets common tags on the Dogstatsd Client that will be sent along with all dogstatsd packets. Ref: http://docs.datadoghq.com/guides/dogstatsd/#tags

type FanoutSink 

type FanoutSink []Sink

FanoutSink is used to sink to fanout values to multiple sinks

func NewFanoutSink 

func NewFanoutSink(sinks ...Sink) FanoutSink

NewFanoutSink creates fan-out sink

func (FanoutSink) AddSample 

func (fh FanoutSink) AddSample(key []string, val float32, tags []Tag)

AddSample is for timing information, where quantiles are used

func (FanoutSink) IncrCounter 

func (fh FanoutSink) IncrCounter(key []string, val float32, tags []Tag)

IncrCounter should accumulate values

func (FanoutSink) SetGauge 

func (fh FanoutSink) SetGauge(key []string, val float32, tags []Tag)

SetGauge should retain the last value it is set to

type GaugeValue 

type GaugeValue struct {
	Name  string
	Hash  string `json:"-"`
	Value float32

	Labels        []Tag             `json:"-"`
	DisplayLabels map[string]string `json:"Labels"`
}

GaugeValue provides gauge value

type InmemSignal 

type InmemSignal struct {
	// contains filtered or unexported fields
}

InmemSignal is used to listen for a given signal, and when received, to dump the current metrics from the InmemSink to an io.Writer

func DefaultInmemSignal 

func DefaultInmemSignal(inmem *InmemSink) *InmemSignal

DefaultInmemSignal returns a new InmemSignal that responds to SIGUSR1 and writes output to stderr. Windows uses SIGBREAK

func NewInmemSignal 

func NewInmemSignal(inmem *InmemSink, sig syscall.Signal, w io.Writer) *InmemSignal

NewInmemSignal creates a new InmemSignal which listens for a given signal, and dumps the current metrics out to a writer

func (*InmemSignal) Stop 

func (i *InmemSignal) Stop()

Stop is used to stop the InmemSignal from listening

type InmemSink 

type InmemSink struct {
	// contains filtered or unexported fields
}

InmemSink provides a MetricSink that does in-memory aggregation without sending metrics over a network. It can be embedded within an application to provide profiling information.

func NewInmemSink 

func NewInmemSink(interval, retain time.Duration) *InmemSink

NewInmemSink is used to construct a new in-memory sink. Uses an aggregation interval and maximum retention period.

func (*InmemSink) AddSample 

func (i *InmemSink) AddSample(key []string, val float32, tags []Tag)

AddSample is for timing information, where quantiles are used

func (*InmemSink) Data 

func (i *InmemSink) Data() []*IntervalMetrics

Data is used to retrieve all the aggregated metrics Intervals may be in use, and a read lock should be acquired

func (*InmemSink) DisplayMetrics 

func (i *InmemSink) DisplayMetrics() (*Summary, error)

DisplayMetrics returns a summary of the metrics from the most recent finished interval.

func (*InmemSink) IncrCounter 

func (i *InmemSink) IncrCounter(key []string, val float32, tags []Tag)

IncrCounter should accumulate values

func (*InmemSink) SetGauge 

func (i *InmemSink) SetGauge(key []string, val float32, tags []Tag)

SetGauge should retain the last value it is set to

type IntervalMetrics 

type IntervalMetrics struct {
	sync.RWMutex

	// The start time of the interval
	Interval time.Time

	// Gauges maps the key to the last set value
	Gauges map[string]GaugeValue

	// Points maps the string to the list of emitted values
	// from EmitKey
	Points map[string][]float32

	// Counters maps the string key to a sum of the counter
	// values
	Counters map[string]SampledValue

	// Samples maps the key to an AggregateSample,
	// which has the rolled up view of a sample
	Samples map[string]SampledValue
}

IntervalMetrics stores the aggregated metrics for a specific interval

func NewIntervalMetrics 

func NewIntervalMetrics(intv time.Time) *IntervalMetrics

NewIntervalMetrics creates a new IntervalMetrics for a given interval

type Metrics 

type Metrics struct {
	Config
	// contains filtered or unexported fields
}

Metrics represents an instance of a metrics sink that can be used to emit

func New 

func New(conf *Config, sink Sink) (*Metrics, error)

New is used to create a new instance of Metrics

func NewGlobal 

func NewGlobal(conf *Config, sink Sink) (*Metrics, error)

NewGlobal is the same as New, but it assigns the metrics object to be used globally as well as returning it.

func (*Metrics) AddSample 

func (m *Metrics) AddSample(key []string, val float32, tags ...Tag)

AddSample is for timing information, where quantiles are used

func (*Metrics) IncrCounter 

func (m *Metrics) IncrCounter(key []string, val float32, tags ...Tag)

IncrCounter should accumulate values

func (*Metrics) MeasureSince 

func (m *Metrics) MeasureSince(key []string, start time.Time, tags ...Tag)

MeasureSince is for timing information

func (*Metrics) SetGauge 

func (m *Metrics) SetGauge(key []string, val float32, tags ...Tag)

SetGauge should retain the last value it is set to

func (*Metrics) UpdateFilter 

func (m *Metrics) UpdateFilter(allow, block []string)

UpdateFilter overwrites the existing filter with the given rules.

func (*Metrics) UpdateFilterAndLabels 

func (m *Metrics) UpdateFilterAndLabels(allow, block, allowedLabels, blockedLabels []string)

UpdateFilterAndLabels overwrites the existing filter with the given rules.

type PointValue 

type PointValue struct {
	Name   string
	Points []float32
}

PointValue provides point value

type PrometheusOpts 

type PrometheusOpts struct {
	// Expiration is the duration a metric is valid for, after which it will be
	// untracked. If the value is zero, a metric is never expired.
	Expiration time.Duration
}

PrometheusOpts is used to configure the Prometheus Sink

type PrometheusSink 

type PrometheusSink struct {
	// contains filtered or unexported fields
}

PrometheusSink provides a MetricSink that can be used with a prometheus server.

func NewPrometheusSink 

func NewPrometheusSink() (*PrometheusSink, error)

NewPrometheusSink creates a new PrometheusSink using the default options.

func NewPrometheusSinkFrom 

func NewPrometheusSinkFrom(opts PrometheusOpts) (*PrometheusSink, error)

NewPrometheusSinkFrom creates a new PrometheusSink using the passed options.

func (*PrometheusSink) AddSample 

func (p *PrometheusSink) AddSample(parts []string, val float32, tags []Tag)

AddSample is for timing information, where quantiles are used

func (*PrometheusSink) Collect 

func (p *PrometheusSink) Collect(c chan<- prometheus.Metric)

Collect meets the collection interface and allows us to enforce our expiration logic to clean up ephemeral metrics if their value haven't been set for a duration exceeding our allowed expiration time.

func (*PrometheusSink) Describe 

func (p *PrometheusSink) Describe(c chan<- *prometheus.Desc)

Describe is needed to meet the Collector interface.

func (*PrometheusSink) EmitKey 

func (p *PrometheusSink) EmitKey(key []string, val float32)

EmitKey is not implemented. Prometheus doesn’t offer a type for which an arbitrary number of values is retained, as Prometheus works with a pull model, rather than a push model.

func (*PrometheusSink) IncrCounter 

func (p *PrometheusSink) IncrCounter(parts []string, val float32, tags []Tag)

IncrCounter should accumulate values

func (*PrometheusSink) SetGauge 

func (p *PrometheusSink) SetGauge(parts []string, val float32, tags []Tag)

SetGauge should retain the last value it is set to

type Provider 

type Provider interface {
	SetGauge(key []string, val float32, tags ...Tag)
	IncrCounter(key []string, val float32, tags ...Tag)
	AddSample(key []string, val float32, tags ...Tag)
	MeasureSince(key []string, start time.Time, tags ...Tag)
}

Provider basics

type SampledValue 

type SampledValue struct {
	Name string
	Hash string `json:"-"`
	*AggregateSample
	Mean   float64
	Stddev float64

	Labels        []Tag             `json:"-"`
	DisplayLabels map[string]string `json:"Labels"`
}

SampledValue provides sample value

type Sink 

type Sink interface {
	// SetGauge should retain the last value it is set to
	SetGauge(key []string, val float32, tags []Tag)
	// IncrCounter should accumulate values
	IncrCounter(key []string, val float32, tags []Tag)
	// AddSample is for timing information, where quantiles are used
	AddSample(key []string, val float32, tags []Tag)
}

The Sink interface is used to transmit metrics information to an external system

func NewInmemSinkFromURL 

func NewInmemSinkFromURL(u *url.URL) (Sink, error)

NewInmemSinkFromURL creates an InmemSink from a URL. It is used (and tested) from NewMetricSinkFromURL.

func NewMetricSinkFromURL 

func NewMetricSinkFromURL(urlStr string) (Sink, error)

NewMetricSinkFromURL allows a generic URL input to configure any of the supported sinks. The scheme of the URL identifies the type of the sink, the and query parameters are used to set options.

"statsd://" - Initializes a StatsdSink. The host and port are passed through as the "addr" of the sink

"statsite://" - Initializes a StatsiteSink. The host and port become the "addr" of the sink

"inmem://" - Initializes an InmemSink. The host and port are ignored. The "interval" and "retain" query parameters must be specified with valid durations, see NewInmemSink for details.

func NewStatsdSinkFromURL 

func NewStatsdSinkFromURL(u *url.URL) (Sink, error)

NewStatsdSinkFromURL creates an StatsdSink from a URL. It is used (and tested) from NewMetricSinkFromURL.

type StatsdSink 

type StatsdSink struct {
	// contains filtered or unexported fields
}

StatsdSink provides a MetricSink that can be used with a statsite or statsd metrics server. It uses only UDP packets, while StatsiteSink uses TCP.

func NewStatsdSink 

func NewStatsdSink(addr string) (*StatsdSink, error)

NewStatsdSink is used to create a new StatsdSink

func (*StatsdSink) AddSample 

func (s *StatsdSink) AddSample(key []string, val float32, tags []Tag)

AddSample is for timing information, where quantiles are used

func (*StatsdSink) IncrCounter 

func (s *StatsdSink) IncrCounter(key []string, val float32, tags []Tag)

IncrCounter should accumulate values

func (*StatsdSink) SetGauge 

func (s *StatsdSink) SetGauge(key []string, val float32, tags []Tag)

SetGauge should retain the last value it is set to

func (*StatsdSink) Shutdown 

func (s *StatsdSink) Shutdown()

Shutdown is used to stop flushing to statsd

type Summary 

type Summary struct {
	Timestamp string
	Gauges    []GaugeValue
	Points    []PointValue
	Counters  []SampledValue
	Samples   []SampledValue
}

Summary holds a roll-up of metrics info for a given interval

type Tag 

type Tag struct {
	Name  string
	Value string
}

Tag is used to add dimentions to metrics

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.