tally

package module
v4.0.0-...-1bded78 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 1, 2023 License: MIT Imports: 15 Imported by: 0

README

✔ tally GoDoc Build Status Coverage Status

Fast, buffered, hierarchical stats collection in Go.

Installation

go get -u github.com/sripad11/tally

Abstract

Tally provides a common interface for emitting metrics, while letting you not worry about the velocity of metrics emission.

By default it buffers counters, gauges and histograms at a specified interval but does not buffer timer values. This is primarily so timer values can have all their values sampled if desired and if not they can be sampled as summaries or histograms independently by a reporter.

Structure

  • Scope: Keeps track of metrics, and their common metadata.
  • Metrics: Counters, Gauges, Timers and Histograms.
  • Reporter: Implemented by you. Accepts aggregated values from the scope. Forwards the aggregated values to your metrics ingestion pipeline.
    • The reporters already available listed alphabetically are:
      • github.com/sripad11/tally/m3: Report m3 metrics, timers are not sampled and forwarded directly.
      • github.com/sripad11/tally/multi: Report to multiple reporters, you can multi-write metrics to other reporters simply.
      • github.com/sripad11/tally/prometheus: Report prometheus metrics, timers by default are made summaries with an option to make them histograms instead.
      • github.com/sripad11/tally/statsd: Report statsd metrics, no support for tags.
Basics
  • Scopes created with tally provide race-safe registration and use of all metric types Counter, Gauge, Timer, Histogram.
  • NewRootScope(...) returns a Scope and io.Closer, the second return value is used to stop the scope's goroutine reporting values from the scope to it's reporter. This is to reduce the footprint of Scope from the public API for those implementing it themselves to use in Go packages that take a tally Scope.
Acquire a Scope
reporter = NewMyStatsReporter()  // Implement as you will
tags := map[string]string{
	"dc": "east-1",
	"type": "master",
}
reportEvery := time.Second

scope := tally.NewRootScope(tally.ScopeOptions{
	Tags: tags,
	Reporter: reporter,
}, reportEvery)
Get/Create a metric, use it
// Get a counter, increment a counter
reqCounter := scope.Counter("requests")  // cache me
reqCounter.Inc(1)

queueGauge := scope.Gauge("queue_length")  // cache me
queueGauge.Update(42)
Report your metrics

Use the inbuilt statsd reporter:

import (
	"io"
	"github.com/cactus/go-statsd-client/v5/statsd"
	"github.com/sripad11/tally"
	tallystatsd "github.com/sripad11/tally/statsd"
	// ...
)

func newScope() (tally.Scope, io.Closer) {
	statter, _ := statsd.NewBufferedClient("127.0.0.1:8125",
		"stats", 100*time.Millisecond, 1440)

	reporter := tallystatsd.NewReporter(statter, tallystatsd.Options{
		SampleRate: 1.0,
	})

	scope, closer := tally.NewRootScope(tally.ScopeOptions{
		Prefix:   "my-service",
		Tags:     map[string]string{},
		Reporter: reporter,
	}, time.Second)

	return scope, closer
}

Implement your own reporter using the StatsReporter interface:


// BaseStatsReporter implements the shared reporter methods.
type BaseStatsReporter interface {
	Capabilities() Capabilities
	Flush()
}

// StatsReporter is a backend for Scopes to report metrics to.
type StatsReporter interface {
	BaseStatsReporter

	// ReportCounter reports a counter value
	ReportCounter(
		name string,
		tags map[string]string,
		value int64,
	)

	// ReportGauge reports a gauge value
	ReportGauge(
		name string,
		tags map[string]string,
		value float64,
	)

	// ReportTimer reports a timer value
	ReportTimer(
		name string,
		tags map[string]string,
		interval time.Duration,
	)

	// ReportHistogramValueSamples reports histogram samples for a bucket
	ReportHistogramValueSamples(
		name string,
		tags map[string]string,
		buckets Buckets,
		bucketLowerBound,
		bucketUpperBound float64,
		samples int64,
	)

	// ReportHistogramDurationSamples reports histogram samples for a bucket
	ReportHistogramDurationSamples(
		name string,
		tags map[string]string,
		buckets Buckets,
		bucketLowerBound,
		bucketUpperBound time.Duration,
		samples int64,
	)
}

Or implement your own metrics implementation that matches the tally Scope interface to use different buffering semantics:

type Scope interface {
	// Counter returns the Counter object corresponding to the name.
	Counter(name string) Counter

	// Gauge returns the Gauge object corresponding to the name.
	Gauge(name string) Gauge

	// Timer returns the Timer object corresponding to the name.
	Timer(name string) Timer

	// Histogram returns the Histogram object corresponding to the name.
	// To use default value and duration buckets configured for the scope
	// simply pass tally.DefaultBuckets or nil.
	// You can use tally.ValueBuckets{x, y, ...} for value buckets.
	// You can use tally.DurationBuckets{x, y, ...} for duration buckets.
	// You can use tally.MustMakeLinearValueBuckets(start, width, count) for linear values.
	// You can use tally.MustMakeLinearDurationBuckets(start, width, count) for linear durations.
	// You can use tally.MustMakeExponentialValueBuckets(start, factor, count) for exponential values.
	// You can use tally.MustMakeExponentialDurationBuckets(start, factor, count) for exponential durations.
	Histogram(name string, buckets Buckets) Histogram

	// Tagged returns a new child scope with the given tags and current tags.
	Tagged(tags map[string]string) Scope

	// SubScope returns a new child scope appending a further name prefix.
	SubScope(name string) Scope

	// Capabilities returns a description of metrics reporting capabilities.
	Capabilities() Capabilities
}

// Capabilities is a description of metrics reporting capabilities.
type Capabilities interface {
	// Reporting returns whether the reporter has the ability to actively report.
	Reporting() bool

	// Tagging returns whether the reporter has the capability for tagged metrics.
	Tagging() bool
}

Performance

This stuff needs to be fast. With that in mind, we avoid locks and unnecessary memory allocations.

BenchmarkCounterInc-8               	200000000	         7.68 ns/op
BenchmarkReportCounterNoData-8      	300000000	         4.88 ns/op
BenchmarkReportCounterWithData-8    	100000000	        21.6 ns/op
BenchmarkGaugeSet-8                 	100000000	        16.0 ns/op
BenchmarkReportGaugeNoData-8        	100000000	        10.4 ns/op
BenchmarkReportGaugeWithData-8      	50000000	        27.6 ns/op
BenchmarkTimerInterval-8            	50000000	        37.7 ns/op
BenchmarkTimerReport-8              	300000000	         5.69 ns/op

Released under the MIT License.

Documentation

Index

Constants

View Source 
const Version = "4.1.8"

Version is the current version of the library.

Variables

View Source 
var (
	// DefaultReplacementCharacter is the default character used for
	// replacements.
	DefaultReplacementCharacter = '_'

	// AlphanumericRange is the range of alphanumeric characters.
	AlphanumericRange = []SanitizeRange{
		{rune('a'), rune('z')},
		{rune('A'), rune('Z')},
		{rune('0'), rune('9')}}

	// UnderscoreCharacters is just an underscore character.
	UnderscoreCharacters = []rune{
		'_'}

	// UnderscoreDashCharacters is a slice of underscore, and
	// dash characters.
	UnderscoreDashCharacters = []rune{
		'-',
		'_'}

	// UnderscoreDashDotCharacters is a slice of underscore,
	// dash, and dot characters.
	UnderscoreDashDotCharacters = []rune{
		'.',
		'-',
		'_'}
)
View Source 
var (
	// NoopScope is a scope that does nothing
	NoopScope, _ = NewRootScope(ScopeOptions{Reporter: NullStatsReporter}, 0)
	// DefaultSeparator is the default separator used to join nested scopes
	DefaultSeparator = "."
)
View Source 
var NewStopwatch = ubertally.NewStopwatch

Functions

func KeyForPrefixedStringMap 

func KeyForPrefixedStringMap(
	prefix string,
	stringMap map[string]string,
) string

KeyForPrefixedStringMap generates a unique key for a a prefix and a map string set combination.

func KeyForStringMap 

func KeyForStringMap(
	stringMap map[string]string,
) string

KeyForStringMap generates a unique key for a map string set combination.

func NoOpSanitizeFn 

func NoOpSanitizeFn(v string) string

NoOpSanitizeFn returns the input un-touched.

Types

type BaseStatsReporter 

type BaseStatsReporter interface {
	// Capabilities returns the capabilities description of the reporter.
	Capabilities() Capabilities

	// Flush asks the reporter to flush all reported values.
	Flush()
}

BaseStatsReporter implements the shared reporter methods.

type BucketPair 

type BucketPair = ubertally.BucketPair

BucketPair describes the lower and upper bounds for a derived bucket from a buckets set.

func BucketPairs 

func BucketPairs(buckets Buckets) []BucketPair

BucketPairs creates a set of bucket pairs from a set of buckets describing the lower and upper bounds for each derived bucket.

type Buckets 

type Buckets = ubertally.Buckets

Buckets is an interface that can represent a set of buckets either as float64s or as durations. 

var (
	// DefaultBuckets can be passed to specify to default buckets.
	DefaultBuckets Buckets
)

type CachedCount 

type CachedCount interface {
	ReportCount(value int64)
}

CachedCount interface for reporting an individual counter

type CachedGauge 

type CachedGauge interface {
	ReportGauge(value float64)
}

CachedGauge interface for reporting an individual gauge

type CachedHistogram 

type CachedHistogram interface {
	ValueBucket(
		bucketLowerBound, bucketUpperBound float64,
	) CachedHistogramBucket
	DurationBucket(
		bucketLowerBound, bucketUpperBound time.Duration,
	) CachedHistogramBucket
}

CachedHistogram interface for reporting histogram samples to buckets

type CachedHistogramBucket 

type CachedHistogramBucket interface {
	ReportSamples(value int64)
}

CachedHistogramBucket interface for reporting histogram samples to a specific bucket

type CachedStatsReporter 

type CachedStatsReporter interface {
	BaseStatsReporter

	// AllocateCounter pre allocates a counter data structure with name & tags.
	AllocateCounter(
		name string,
		tags map[string]string,
	) CachedCount

	// AllocateGauge pre allocates a gauge data structure with name & tags.
	AllocateGauge(
		name string,
		tags map[string]string,
	) CachedGauge

	// AllocateTimer pre allocates a timer data structure with name & tags.
	AllocateTimer(
		name string,
		tags map[string]string,
	) CachedTimer

	// AllocateHistogram pre allocates a histogram data structure with name, tags,
	// value buckets and duration buckets.
	AllocateHistogram(
		name string,
		tags map[string]string,
		buckets Buckets,
	) CachedHistogram
}

CachedStatsReporter is a backend for Scopes that pre allocates all counter, gauges, timers & histograms. This is harder to implement but more performant.

type CachedTimer 

type CachedTimer interface {
	ReportTimer(interval time.Duration)
}

CachedTimer interface for reporting an individual timer

type Capabilities 

type Capabilities = ubertally.Capabilities

Capabilities is a description of metrics reporting capabilities.

type Counter 

type Counter = ubertally.Counter

Counter is the interface for emitting counter type metrics.

type CounterSnapshot 

type CounterSnapshot interface {
	// Name returns the name
	Name() string

	// Tags returns the tags
	Tags() map[string]string

	// Value returns the value
	Value() int64
}

CounterSnapshot is a snapshot of a counter

type DurationBuckets 

type DurationBuckets []time.Duration

DurationBuckets is a set of time.Duration values that implements Buckets.

func ExponentialDurationBuckets 

func ExponentialDurationBuckets(start time.Duration, factor float64, n int) (DurationBuckets, error)

ExponentialDurationBuckets creates a set of exponential duration buckets.

func LinearDurationBuckets 

func LinearDurationBuckets(start, width time.Duration, n int) (DurationBuckets, error)

LinearDurationBuckets creates a set of linear duration buckets.

func MustMakeExponentialDurationBuckets 

func MustMakeExponentialDurationBuckets(start time.Duration, factor float64, n int) DurationBuckets

MustMakeExponentialDurationBuckets creates a set of exponential value buckets or panics.

func MustMakeLinearDurationBuckets 

func MustMakeLinearDurationBuckets(start, width time.Duration, n int) DurationBuckets

MustMakeLinearDurationBuckets creates a set of linear duration buckets. or panics.

func (DurationBuckets) AsDurations 

func (v DurationBuckets) AsDurations() []time.Duration

AsDurations implements Buckets.

func (DurationBuckets) AsValues 

func (v DurationBuckets) AsValues() []float64

AsValues implements Buckets and returns float64 representations of the time.Duration values divided by time.Second.

func (DurationBuckets) Len 

func (v DurationBuckets) Len() int

Implements sort.Interface

func (DurationBuckets) Less 

func (v DurationBuckets) Less(i, j int) bool

Implements sort.Interface

func (DurationBuckets) String 

func (v DurationBuckets) String() string

func (DurationBuckets) Swap 

func (v DurationBuckets) Swap(i, j int)

Implements sort.Interface

type Gauge 

type Gauge = ubertally.Gauge

Gauge is the interface for emitting gauge metrics.

type GaugeSnapshot 

type GaugeSnapshot interface {
	// Name returns the name
	Name() string

	// Tags returns the tags
	Tags() map[string]string

	// Value returns the value
	Value() float64
}

GaugeSnapshot is a snapshot of a gauge

type Histogram 

type Histogram = ubertally.Histogram

Histogram is the interface for emitting histogram metrics

type HistogramSnapshot 

type HistogramSnapshot interface {
	// Name returns the name
	Name() string

	// Tags returns the tags
	Tags() map[string]string

	// Values returns the sample values by upper bound for a valueHistogram
	Values() map[float64]int64

	// Durations returns the sample values by upper bound for a durationHistogram
	Durations() map[time.Duration]int64
}

HistogramSnapshot is a snapshot of a histogram

type InternalMetricOption 

type InternalMetricOption int

InternalMetricOption is used to configure internal metrics. 

const (
	// Unset is the "no-op" config, which turns off internal metrics.
	Unset InternalMetricOption = iota
	// SendInternalMetrics turns on internal metrics submission.
	SendInternalMetrics
	// OmitInternalMetrics turns off internal metrics submission.
	OmitInternalMetrics
)

type ObjectPool 

type ObjectPool struct {
	// contains filtered or unexported fields
}

ObjectPool is an minimalistic object pool to avoid any circular dependencies on any other object pool.

func NewObjectPool 

func NewObjectPool(size int) *ObjectPool

NewObjectPool creates a new pool.

func (*ObjectPool) Get 

func (p *ObjectPool) Get() interface{}

Get gets an object from the pool.

func (*ObjectPool) Init 

func (p *ObjectPool) Init(alloc func() interface{})

Init initializes the object pool.

func (*ObjectPool) Put 

func (p *ObjectPool) Put(obj interface{})

Put puts an object back to the pool.

type SanitizeFn 

type SanitizeFn func(string) string

SanitizeFn returns a sanitized version of the input string.

type SanitizeOptions 

type SanitizeOptions struct {
	NameCharacters       ValidCharacters
	KeyCharacters        ValidCharacters
	ValueCharacters      ValidCharacters
	ReplacementCharacter rune
}

SanitizeOptions are the set of configurable options for sanitisation.

type SanitizeRange 

type SanitizeRange [2]rune

SanitizeRange is a range of characters (inclusive on both ends).

type Sanitizer 

type Sanitizer interface {
	// Name sanitizes the provided `name` string.
	Name(n string) string

	// Key sanitizes the provided `key` string.
	Key(k string) string

	// Value sanitizes the provided `value` string.
	Value(v string) string
}

Sanitizer sanitizes the provided input based on the function executed.

func NewNoOpSanitizer 

func NewNoOpSanitizer() Sanitizer

NewNoOpSanitizer returns a sanitizer which returns all inputs un-touched.

func NewSanitizer 

func NewSanitizer(opts SanitizeOptions) Sanitizer

NewSanitizer returns a new sanitizer based on provided options.

type Scope 

type Scope = ubertally.Scope

Scope is a namespace wrapper around a stats reporter, ensuring that all emitted values have a given prefix or set of tags.

IMPORTANT: When using Prometheus reporters, users must take care to 

not create metrics from both parent scopes and subscopes
that have the same metric name but different tag keys,
as metric allocation will panic.

func NewRootScope 

func NewRootScope(opts ScopeOptions, interval time.Duration) (Scope, io.Closer)

NewRootScope creates a new root Scope with a set of options and a reporting interval. Must provide either a StatsReporter or a CachedStatsReporter.

func NewRootScopeWithDefaultInterval 

func NewRootScopeWithDefaultInterval(opts ScopeOptions) (Scope, io.Closer)

NewRootScopeWithDefaultInterval invokes NewRootScope with the default reporting interval of 2s.

type ScopeOptions 

type ScopeOptions struct {
	Tags            map[string]string
	Prefix          string
	Reporter        StatsReporter
	CachedReporter  CachedStatsReporter
	Separator       string
	DefaultBuckets  Buckets
	SanitizeOptions *SanitizeOptions

	MetricsOption InternalMetricOption
	// contains filtered or unexported fields
}

ScopeOptions is a set of options to construct a scope.

type Snapshot 

type Snapshot interface {
	// Counters returns a snapshot of all counter summations since last report execution
	Counters() map[string]CounterSnapshot

	// Gauges returns a snapshot of gauge last values since last report execution
	Gauges() map[string]GaugeSnapshot

	// Timers returns a snapshot of timer values since last report execution
	Timers() map[string]TimerSnapshot

	// Histograms returns a snapshot of histogram samples since last report execution
	Histograms() map[string]HistogramSnapshot
}

Snapshot is a snapshot of values since last report execution

type StatsReporter 

type StatsReporter interface {
	BaseStatsReporter

	// ReportCounter reports a counter value
	ReportCounter(
		name string,
		tags map[string]string,
		value int64,
	)

	// ReportGauge reports a gauge value
	ReportGauge(
		name string,
		tags map[string]string,
		value float64,
	)

	// ReportTimer reports a timer value
	ReportTimer(
		name string,
		tags map[string]string,
		interval time.Duration,
	)

	// ReportHistogramValueSamples reports histogram samples for a bucket
	ReportHistogramValueSamples(
		name string,
		tags map[string]string,
		buckets Buckets,
		bucketLowerBound,
		bucketUpperBound float64,
		samples int64,
	)

	// ReportHistogramDurationSamples reports histogram samples for a bucket
	ReportHistogramDurationSamples(
		name string,
		tags map[string]string,
		buckets Buckets,
		bucketLowerBound,
		bucketUpperBound time.Duration,
		samples int64,
	)
}

StatsReporter is a backend for Scopes to report metrics to. 

var NullStatsReporter StatsReporter = nullStatsReporter{}

NullStatsReporter is an implementation of StatsReporter than simply does nothing.

type Stopwatch 

type Stopwatch = ubertally.Stopwatch

Stopwatch is a helper for simpler tracking of elapsed time, use the Stop() method to report time elapsed since its created back to the timer or histogram.

type StopwatchRecorder 

type StopwatchRecorder = ubertally.StopwatchRecorder

StopwatchRecorder is a recorder that is called when a stopwatch is stopped with Stop().

type TestScope 

type TestScope interface {
	Scope

	// Snapshot returns a copy of all values since the last report execution,
	// this is an expensive operation and should only be use for testing purposes
	Snapshot() Snapshot
}

TestScope is a metrics collector that has no reporting, ensuring that all emitted values have a given prefix or set of tags

func NewTestScope 

func NewTestScope(
	prefix string,
	tags map[string]string,
) TestScope

NewTestScope creates a new Scope without a stats reporter with the given prefix and adds the ability to take snapshots of metrics emitted to it.

type Timer 

type Timer = ubertally.Timer

Timer is the interface for emitting timer metrics.

type TimerSnapshot 

type TimerSnapshot interface {
	// Name returns the name
	Name() string

	// Tags returns the tags
	Tags() map[string]string

	// Values returns the values
	Values() []time.Duration
}

TimerSnapshot is a snapshot of a timer

type ValidCharacters 

type ValidCharacters struct {
	Ranges     []SanitizeRange
	Characters []rune
}

ValidCharacters is a collection of valid characters.

type ValueBuckets 

type ValueBuckets []float64

ValueBuckets is a set of float64 values that implements Buckets.

func ExponentialValueBuckets 

func ExponentialValueBuckets(start, factor float64, n int) (ValueBuckets, error)

ExponentialValueBuckets creates a set of exponential value buckets.

func LinearValueBuckets 

func LinearValueBuckets(start, width float64, n int) (ValueBuckets, error)

LinearValueBuckets creates a set of linear value buckets.

func MustMakeExponentialValueBuckets 

func MustMakeExponentialValueBuckets(start, factor float64, n int) ValueBuckets

MustMakeExponentialValueBuckets creates a set of exponential value buckets or panics.

func MustMakeLinearValueBuckets 

func MustMakeLinearValueBuckets(start, width float64, n int) ValueBuckets

MustMakeLinearValueBuckets creates a set of linear value buckets or panics.

func (ValueBuckets) AsDurations 

func (v ValueBuckets) AsDurations() []time.Duration

AsDurations implements Buckets and returns time.Duration representations of the float64 values divided by time.Second.

func (ValueBuckets) AsValues 

func (v ValueBuckets) AsValues() []float64

AsValues implements Buckets.

func (ValueBuckets) Len 

func (v ValueBuckets) Len() int

Implements sort.Interface

func (ValueBuckets) Less 

func (v ValueBuckets) Less(i, j int) bool

Implements sort.Interface

func (ValueBuckets) String 

func (v ValueBuckets) String() string

func (ValueBuckets) Swap 

func (v ValueBuckets) Swap(i, j int)

Implements sort.Interface

Source Files

View all Source files

Directories

Path Synopsis
internal
cache
identity
example
example

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.