tally

package module
v4.0.0-...-1bded78 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 1, 2023 License: MIT Imports: 15 Imported by: 0

README

✔ tally GoDoc Build Status Coverage Status

Fast, buffered, hierarchical stats collection in Go.

Installation

go get -u github.com/sripad11/tally

Abstract

Tally provides a common interface for emitting metrics, while letting you not worry about the velocity of metrics emission.

By default it buffers counters, gauges and histograms at a specified interval but does not buffer timer values. This is primarily so timer values can have all their values sampled if desired and if not they can be sampled as summaries or histograms independently by a reporter.

Structure

  • Scope: Keeps track of metrics, and their common metadata.
  • Metrics: Counters, Gauges, Timers and Histograms.
  • Reporter: Implemented by you. Accepts aggregated values from the scope. Forwards the aggregated values to your metrics ingestion pipeline.
    • The reporters already available listed alphabetically are:
      • github.com/sripad11/tally/m3: Report m3 metrics, timers are not sampled and forwarded directly.
      • github.com/sripad11/tally/multi: Report to multiple reporters, you can multi-write metrics to other reporters simply.
      • github.com/sripad11/tally/prometheus: Report prometheus metrics, timers by default are made summaries with an option to make them histograms instead.
      • github.com/sripad11/tally/statsd: Report statsd metrics, no support for tags.
Basics
  • Scopes created with tally provide race-safe registration and use of all metric types Counter, Gauge, Timer, Histogram.
  • NewRootScope(...) returns a Scope and io.Closer, the second return value is used to stop the scope's goroutine reporting values from the scope to it's reporter. This is to reduce the footprint of Scope from the public API for those implementing it themselves to use in Go packages that take a tally Scope.
Acquire a Scope
reporter = NewMyStatsReporter()  // Implement as you will
tags := map[string]string{
	"dc": "east-1",
	"type": "master",
}
reportEvery := time.Second

scope := tally.NewRootScope(tally.ScopeOptions{
	Tags: tags,
	Reporter: reporter,
}, reportEvery)
Get/Create a metric, use it
// Get a counter, increment a counter
reqCounter := scope.Counter("requests")  // cache me
reqCounter.Inc(1)

queueGauge := scope.Gauge("queue_length")  // cache me
queueGauge.Update(42)
Report your metrics

Use the inbuilt statsd reporter:

import (
	"io"
	"github.com/cactus/go-statsd-client/v5/statsd"
	"github.com/sripad11/tally"
	tallystatsd "github.com/sripad11/tally/statsd"
	// ...
)

func newScope() (tally.Scope, io.Closer) {
	statter, _ := statsd.NewBufferedClient("127.0.0.1:8125",
		"stats", 100*time.Millisecond, 1440)

	reporter := tallystatsd.NewReporter(statter, tallystatsd.Options{
		SampleRate: 1.0,
	})

	scope, closer := tally.NewRootScope(tally.ScopeOptions{
		Prefix:   "my-service",
		Tags:     map[string]string{},
		Reporter: reporter,
	}, time.Second)

	return scope, closer
}

Implement your own reporter using the StatsReporter interface:


// BaseStatsReporter implements the shared reporter methods.
type BaseStatsReporter interface {
	Capabilities() Capabilities
	Flush()
}

// StatsReporter is a backend for Scopes to report metrics to.
type StatsReporter interface {
	BaseStatsReporter

	// ReportCounter reports a counter value
	ReportCounter(
		name string,
		tags map[string]string,
		value int64,
	)

	// ReportGauge reports a gauge value
	ReportGauge(
		name string,
		tags map[string]string,
		value float64,
	)

	// ReportTimer reports a timer value
	ReportTimer(
		name string,
		tags map[string]string,
		interval time.Duration,
	)

	// ReportHistogramValueSamples reports histogram samples for a bucket
	ReportHistogramValueSamples(
		name string,
		tags map[string]string,
		buckets Buckets,
		bucketLowerBound,
		bucketUpperBound float64,
		samples int64,
	)

	// ReportHistogramDurationSamples reports histogram samples for a bucket
	ReportHistogramDurationSamples(
		name string,
		tags map[string]string,
		buckets Buckets,
		bucketLowerBound,
		bucketUpperBound time.Duration,
		samples int64,
	)
}

Or implement your own metrics implementation that matches the tally Scope interface to use different buffering semantics:

type Scope interface {
	// Counter returns the Counter object corresponding to the name.
	Counter(name string) Counter

	// Gauge returns the Gauge object corresponding to the name.
	Gauge(name string) Gauge

	// Timer returns the Timer object corresponding to the name.
	Timer(name string) Timer

	// Histogram returns the Histogram object corresponding to the name.
	// To use default value and duration buckets configured for the scope
	// simply pass tally.DefaultBuckets or nil.
	// You can use tally.ValueBuckets{x, y, ...} for value buckets.
	// You can use tally.DurationBuckets{x, y, ...} for duration buckets.
	// You can use tally.MustMakeLinearValueBuckets(start, width, count) for linear values.
	// You can use tally.MustMakeLinearDurationBuckets(start, width, count) for linear durations.
	// You can use tally.MustMakeExponentialValueBuckets(start, factor, count) for exponential values.
	// You can use tally.MustMakeExponentialDurationBuckets(start, factor, count) for exponential durations.
	Histogram(name string, buckets Buckets) Histogram

	// Tagged returns a new child scope with the given tags and current tags.
	Tagged(tags map[string]string) Scope

	// SubScope returns a new child scope appending a further name prefix.
	SubScope(name string) Scope

	// Capabilities returns a description of metrics reporting capabilities.
	Capabilities() Capabilities
}

// Capabilities is a description of metrics reporting capabilities.
type Capabilities interface {
	// Reporting returns whether the reporter has the ability to actively report.
	Reporting() bool

	// Tagging returns whether the reporter has the capability for tagged metrics.
	Tagging() bool
}

Performance

This stuff needs to be fast. With that in mind, we avoid locks and unnecessary memory allocations.

BenchmarkCounterInc-8               	200000000	         7.68 ns/op
BenchmarkReportCounterNoData-8      	300000000	         4.88 ns/op
BenchmarkReportCounterWithData-8    	100000000	        21.6 ns/op
BenchmarkGaugeSet-8                 	100000000	        16.0 ns/op
BenchmarkReportGaugeNoData-8        	100000000	        10.4 ns/op
BenchmarkReportGaugeWithData-8      	50000000	        27.6 ns/op
BenchmarkTimerInterval-8            	50000000	        37.7 ns/op
BenchmarkTimerReport-8              	300000000	         5.69 ns/op

Released under the MIT License.

Documentation

Index

Constants

View Source
const Version = "4.1.8"

Version is the current version of the library.

Variables

View Source
var (
	// DefaultReplacementCharacter is the default character used for
	// replacements.
	DefaultReplacementCharacter = '_'

	// AlphanumericRange is the range of alphanumeric characters.
	AlphanumericRange = []SanitizeRange{
		{rune('a'), rune('z')},
		{rune('A'), rune('Z')},
		{rune('0'), rune('9')}}

	// UnderscoreCharacters is just an underscore character.
	UnderscoreCharacters = []rune{
		'_'}

	// UnderscoreDashCharacters is a slice of underscore, and
	// dash characters.
	UnderscoreDashCharacters = []rune{
		'-',
		'_'}

	// UnderscoreDashDotCharacters is a slice of underscore,
	// dash, and dot characters.
	UnderscoreDashDotCharacters = []rune{
		'.',
		'-',
		'_'}
)
View Source
var (
	// NoopScope is a scope that does nothing
	NoopScope, _ = NewRootScope(ScopeOptions{Reporter: NullStatsReporter}, 0)
	// DefaultSeparator is the default separator used to join nested scopes
	DefaultSeparator = "."
)
View Source
var NewStopwatch = ubertally.NewStopwatch

Functions

func KeyForPrefixedStringMap

func KeyForPrefixedStringMap(
	prefix string,
	stringMap map[string]string,
) string

KeyForPrefixedStringMap generates a unique key for a a prefix and a map string set combination.

func KeyForStringMap

func KeyForStringMap(
	stringMap map[string]string,
) string

KeyForStringMap generates a unique key for a map string set combination.

func NoOpSanitizeFn

func NoOpSanitizeFn(v string) string

NoOpSanitizeFn returns the input un-touched.

Types

type BaseStatsReporter

type BaseStatsReporter interface {
	// Capabilities returns the capabilities description of the reporter.
	Capabilities() Capabilities

	// Flush asks the reporter to flush all reported values.
	Flush()
}

BaseStatsReporter implements the shared reporter methods.

type BucketPair

type BucketPair = ubertally.BucketPair

BucketPair describes the lower and upper bounds for a derived bucket from a buckets set.

func BucketPairs

func BucketPairs(buckets Buckets) []BucketPair

BucketPairs creates a set of bucket pairs from a set of buckets describing the lower and upper bounds for each derived bucket.

type Buckets

type Buckets = ubertally.Buckets

Buckets is an interface that can represent a set of buckets either as float64s or as durations.

var (
	// DefaultBuckets can be passed to specify to default buckets.
	DefaultBuckets Buckets
)

type CachedCount

type CachedCount interface {
	ReportCount(value int64)
}

CachedCount interface for reporting an individual counter

type CachedGauge

type CachedGauge interface {
	ReportGauge(value float64)
}

CachedGauge interface for reporting an individual gauge

type CachedHistogram

type CachedHistogram interface {
	ValueBucket(
		bucketLowerBound, bucketUpperBound float64,
	) CachedHistogramBucket
	DurationBucket(
		bucketLowerBound, bucketUpperBound time.Duration,
	) CachedHistogramBucket
}

CachedHistogram interface for reporting histogram samples to buckets

type CachedHistogramBucket

type CachedHistogramBucket interface {
	ReportSamples(value int64)
}

CachedHistogramBucket interface for reporting histogram samples to a specific bucket

type CachedStatsReporter

type CachedStatsReporter interface {
	BaseStatsReporter

	// AllocateCounter pre allocates a counter data structure with name & tags.
	AllocateCounter(
		name string,
		tags map[string]string,
	) CachedCount

	// AllocateGauge pre allocates a gauge data structure with name & tags.
	AllocateGauge(
		name string,
		tags map[string]string,
	) CachedGauge

	// AllocateTimer pre allocates a timer data structure with name & tags.
	AllocateTimer(
		name string,
		tags map[string]string,
	) CachedTimer

	// AllocateHistogram pre allocates a histogram data structure with name, tags,
	// value buckets and duration buckets.
	AllocateHistogram(
		name string,
		tags map[string]string,
		buckets Buckets,
	) CachedHistogram
}

CachedStatsReporter is a backend for Scopes that pre allocates all counter, gauges, timers & histograms. This is harder to implement but more performant.

type CachedTimer

type CachedTimer interface {
	ReportTimer(interval time.Duration)
}

CachedTimer interface for reporting an individual timer

type Capabilities

type Capabilities = ubertally.Capabilities

Capabilities is a description of metrics reporting capabilities.

type Counter

type Counter = ubertally.Counter

Counter is the interface for emitting counter type metrics.

type CounterSnapshot

type CounterSnapshot interface {
	// Name returns the name
	Name() string

	// Tags returns the tags
	Tags() map[string]string

	// Value returns the value
	Value() int64
}

CounterSnapshot is a snapshot of a counter

type DurationBuckets

type DurationBuckets []time.Duration

DurationBuckets is a set of time.Duration values that implements Buckets.

func ExponentialDurationBuckets

func ExponentialDurationBuckets(start time.Duration, factor float64, n int) (DurationBuckets, error)

ExponentialDurationBuckets creates a set of exponential duration buckets.

func LinearDurationBuckets

func LinearDurationBuckets(start, width time.Duration, n int) (DurationBuckets, error)

LinearDurationBuckets creates a set of linear duration buckets.

func MustMakeExponentialDurationBuckets

func MustMakeExponentialDurationBuckets(start time.Duration, factor float64, n int) DurationBuckets

MustMakeExponentialDurationBuckets creates a set of exponential value buckets or panics.

func MustMakeLinearDurationBuckets

func MustMakeLinearDurationBuckets(start, width time.Duration, n int) DurationBuckets

MustMakeLinearDurationBuckets creates a set of linear duration buckets. or panics.

func (DurationBuckets) AsDurations

func (v DurationBuckets) AsDurations() []time.Duration

AsDurations implements Buckets.

func (DurationBuckets) AsValues

func (v DurationBuckets) AsValues() []float64

AsValues implements Buckets and returns float64 representations of the time.Duration values divided by time.Second.

func (DurationBuckets) Len

func (v DurationBuckets) Len() int

Implements sort.Interface

func (DurationBuckets) Less

func (v DurationBuckets) Less(i, j int) bool

Implements sort.Interface

func (DurationBuckets) String

func (v DurationBuckets) String() string

func (DurationBuckets) Swap

func (v DurationBuckets) Swap(i, j int)

Implements sort.Interface

type Gauge

type Gauge = ubertally.Gauge

Gauge is the interface for emitting gauge metrics.

type GaugeSnapshot

type GaugeSnapshot interface {
	// Name returns the name
	Name() string

	// Tags returns the tags
	Tags() map[string]string

	// Value returns the value
	Value() float64
}

GaugeSnapshot is a snapshot of a gauge

type Histogram

type Histogram = ubertally.Histogram

Histogram is the interface for emitting histogram metrics

type HistogramSnapshot

type HistogramSnapshot interface {
	// Name returns the name
	Name() string

	// Tags returns the tags
	Tags() map[string]string

	// Values returns the sample values by upper bound for a valueHistogram
	Values() map[float64]int64

	// Durations returns the sample values by upper bound for a durationHistogram
	Durations() map[time.Duration]int64
}

HistogramSnapshot is a snapshot of a histogram

type InternalMetricOption

type InternalMetricOption int

InternalMetricOption is used to configure internal metrics.

const (
	// Unset is the "no-op" config, which turns off internal metrics.
	Unset InternalMetricOption = iota
	// SendInternalMetrics turns on internal metrics submission.
	SendInternalMetrics
	// OmitInternalMetrics turns off internal metrics submission.
	OmitInternalMetrics
)

type ObjectPool

type ObjectPool struct {
	// contains filtered or unexported fields
}

ObjectPool is an minimalistic object pool to avoid any circular dependencies on any other object pool.

func NewObjectPool

func NewObjectPool(size int) *ObjectPool

NewObjectPool creates a new pool.

func (*ObjectPool) Get

func (p *ObjectPool) Get() interface{}

Get gets an object from the pool.

func (*ObjectPool) Init

func (p *ObjectPool) Init(alloc func() interface{})

Init initializes the object pool.

func (*ObjectPool) Put

func (p *ObjectPool) Put(obj interface{})

Put puts an object back to the pool.

type SanitizeFn

type SanitizeFn func(string) string

SanitizeFn returns a sanitized version of the input string.

type SanitizeOptions

type SanitizeOptions struct {
	NameCharacters       ValidCharacters
	KeyCharacters        ValidCharacters
	ValueCharacters      ValidCharacters
	ReplacementCharacter rune
}

SanitizeOptions are the set of configurable options for sanitisation.

type SanitizeRange

type SanitizeRange [2]rune

SanitizeRange is a range of characters (inclusive on both ends).

type Sanitizer

type Sanitizer interface {
	// Name sanitizes the provided `name` string.
	Name(n string) string

	// Key sanitizes the provided `key` string.
	Key(k string) string

	// Value sanitizes the provided `value` string.
	Value(v string) string
}

Sanitizer sanitizes the provided input based on the function executed.

func NewNoOpSanitizer

func NewNoOpSanitizer() Sanitizer

NewNoOpSanitizer returns a sanitizer which returns all inputs un-touched.

func NewSanitizer

func NewSanitizer(opts SanitizeOptions) Sanitizer

NewSanitizer returns a new sanitizer based on provided options.

type Scope

type Scope = ubertally.Scope

Scope is a namespace wrapper around a stats reporter, ensuring that all emitted values have a given prefix or set of tags.

IMPORTANT: When using Prometheus reporters, users must take care to

not create metrics from both parent scopes and subscopes
that have the same metric name but different tag keys,
as metric allocation will panic.

func NewRootScope

func NewRootScope(opts ScopeOptions, interval time.Duration) (Scope, io.Closer)

NewRootScope creates a new root Scope with a set of options and a reporting interval. Must provide either a StatsReporter or a CachedStatsReporter.

func NewRootScopeWithDefaultInterval

func NewRootScopeWithDefaultInterval(opts ScopeOptions) (Scope, io.Closer)

NewRootScopeWithDefaultInterval invokes NewRootScope with the default reporting interval of 2s.

type ScopeOptions

type ScopeOptions struct {
	Tags            map[string]string
	Prefix          string
	Reporter        StatsReporter
	CachedReporter  CachedStatsReporter
	Separator       string
	DefaultBuckets  Buckets
	SanitizeOptions *SanitizeOptions

	MetricsOption InternalMetricOption
	// contains filtered or unexported fields
}

ScopeOptions is a set of options to construct a scope.

type Snapshot

type Snapshot interface {
	// Counters returns a snapshot of all counter summations since last report execution
	Counters() map[string]CounterSnapshot

	// Gauges returns a snapshot of gauge last values since last report execution
	Gauges() map[string]GaugeSnapshot

	// Timers returns a snapshot of timer values since last report execution
	Timers() map[string]TimerSnapshot

	// Histograms returns a snapshot of histogram samples since last report execution
	Histograms() map[string]HistogramSnapshot
}

Snapshot is a snapshot of values since last report execution

type StatsReporter

type StatsReporter interface {
	BaseStatsReporter

	// ReportCounter reports a counter value
	ReportCounter(
		name string,
		tags map[string]string,
		value int64,
	)

	// ReportGauge reports a gauge value
	ReportGauge(
		name string,
		tags map[string]string,
		value float64,
	)

	// ReportTimer reports a timer value
	ReportTimer(
		name string,
		tags map[string]string,
		interval time.Duration,
	)

	// ReportHistogramValueSamples reports histogram samples for a bucket
	ReportHistogramValueSamples(
		name string,
		tags map[string]string,
		buckets Buckets,
		bucketLowerBound,
		bucketUpperBound float64,
		samples int64,
	)

	// ReportHistogramDurationSamples reports histogram samples for a bucket
	ReportHistogramDurationSamples(
		name string,
		tags map[string]string,
		buckets Buckets,
		bucketLowerBound,
		bucketUpperBound time.Duration,
		samples int64,
	)
}

StatsReporter is a backend for Scopes to report metrics to.

var NullStatsReporter StatsReporter = nullStatsReporter{}

NullStatsReporter is an implementation of StatsReporter than simply does nothing.

type Stopwatch

type Stopwatch = ubertally.Stopwatch

Stopwatch is a helper for simpler tracking of elapsed time, use the Stop() method to report time elapsed since its created back to the timer or histogram.

type StopwatchRecorder

type StopwatchRecorder = ubertally.StopwatchRecorder

StopwatchRecorder is a recorder that is called when a stopwatch is stopped with Stop().

type TestScope

type TestScope interface {
	Scope

	// Snapshot returns a copy of all values since the last report execution,
	// this is an expensive operation and should only be use for testing purposes
	Snapshot() Snapshot
}

TestScope is a metrics collector that has no reporting, ensuring that all emitted values have a given prefix or set of tags

func NewTestScope

func NewTestScope(
	prefix string,
	tags map[string]string,
) TestScope

NewTestScope creates a new Scope without a stats reporter with the given prefix and adds the ability to take snapshots of metrics emitted to it.

type Timer

type Timer = ubertally.Timer

Timer is the interface for emitting timer metrics.

type TimerSnapshot

type TimerSnapshot interface {
	// Name returns the name
	Name() string

	// Tags returns the tags
	Tags() map[string]string

	// Values returns the values
	Values() []time.Duration
}

TimerSnapshot is a snapshot of a timer

type ValidCharacters

type ValidCharacters struct {
	Ranges     []SanitizeRange
	Characters []rune
}

ValidCharacters is a collection of valid characters.

type ValueBuckets

type ValueBuckets []float64

ValueBuckets is a set of float64 values that implements Buckets.

func ExponentialValueBuckets

func ExponentialValueBuckets(start, factor float64, n int) (ValueBuckets, error)

ExponentialValueBuckets creates a set of exponential value buckets.

func LinearValueBuckets

func LinearValueBuckets(start, width float64, n int) (ValueBuckets, error)

LinearValueBuckets creates a set of linear value buckets.

func MustMakeExponentialValueBuckets

func MustMakeExponentialValueBuckets(start, factor float64, n int) ValueBuckets

MustMakeExponentialValueBuckets creates a set of exponential value buckets or panics.

func MustMakeLinearValueBuckets

func MustMakeLinearValueBuckets(start, width float64, n int) ValueBuckets

MustMakeLinearValueBuckets creates a set of linear value buckets or panics.

func (ValueBuckets) AsDurations

func (v ValueBuckets) AsDurations() []time.Duration

AsDurations implements Buckets and returns time.Duration representations of the float64 values divided by time.Second.

func (ValueBuckets) AsValues

func (v ValueBuckets) AsValues() []float64

AsValues implements Buckets.

func (ValueBuckets) Len

func (v ValueBuckets) Len() int

Implements sort.Interface

func (ValueBuckets) Less

func (v ValueBuckets) Less(i, j int) bool

Implements sort.Interface

func (ValueBuckets) String

func (v ValueBuckets) String() string

func (ValueBuckets) Swap

func (v ValueBuckets) Swap(i, j int)

Implements sort.Interface

Directories

Path Synopsis
internal

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL