topk

README

go-prometheus-topk

A custom Prometheus metric exporter wrapping the github.com/dgryski/go-topk library.

By using top-K metric collection, you can estimate the outliers of high-cardinality data without storing the entire dataset. (If you need something more precise than "estimates", then structured log processing is the way to go.)

Status

This is not an officially supported Google product.

Documentation

Overview

Package topk provides a Metric/Collector implementation of a top-K streaming summary algorithm for use with high cardinality data.

The github.com/dgryski/go-topk package is used to implement the calculations.

Index

• type TopK
  • func NewTopK(opts TopKOpts, labelNames []string) TopK
• type TopKBucket
• type TopKOpts

Types

type TopK

type TopK interface {
	prometheus.Collector

	CurryWith(prometheus.Labels) (TopK, error)
	MustCurryWith(prometheus.Labels) TopK
	GetMetricWith(prometheus.Labels) (TopKBucket, error)
	GetMetricWithLabelValues(lvs ...string) (TopKBucket, error)
	With(prometheus.Labels) TopKBucket
	WithLabelValues(lvs ...string) TopKBucket
}

TopK is a metric package for estimating the top keys of a high-cardinality set.

On every collection, the top "K" label pairs (where K is the value of opts.Buckets) are exported as Counters with variable labels, plus a parallel set of Gauges for the error bars.

Usage: call one of the With() methods to receive a TopKBucket, and call the Observe method to record an observation. If any NaN values are passed to Observe, they are treated as 0 so as to not pollute the storage.

func NewTopK

func NewTopK(opts TopKOpts, labelNames []string) TopK

NewTopK constructs a new TopK metric container.

type TopKBucket

type TopKBucket interface {
	Observe(float64)
	Inc()
}

type TopKOpts

type TopKOpts struct {
	// Namespace, Subsystem, and Name are components of the fully-qualified
	// name of the TopK (created by joining these components with "_").
	// Only Name is mandatory, the others merely help structuring the name.
	// Note that the fully-qualified name of the TopK must be a valid
	// Prometheus metric name.
	Namespace string
	Subsystem string
	Name      string

	// Help provides information about this Histogram.
	//
	// Metrics with the same fully-qualified name must have the same Help
	// string.
	Help string

	// ConstLabels are used to attach fixed labels to this metric. Metrics
	// with the same fully-qualified name must have the same label names in
	// their ConstLabels.
	//
	// ConstLabels are only used rarely. In particular, do not use them to
	// attach the same labels to all your metrics. Those use cases are
	// better covered by target labels set by the scraping Prometheus
	// server, or by one specific metric (e.g. a build_info or a
	// machine_role metric). See also
	// https://prometheus.io/docs/instrumenting/writing_exporters/#target-labels,-not-static-scraped-labels
	ConstLabels prometheus.Labels

	// Buckets provides the number of metric streams that this metric is
	// expected to keep an accurate count for (the "K" in top-K).
	Buckets uint64

	// Values under the ReportingThreshold are tracked but not exported.
	ReportingThreshold float64
}