dmetrics

package module
v0.0.0-...-524a5c5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 14, 2024 License: Apache-2.0 Imports: 16 Imported by: 91

README

StreamingFast Metrics Library

reference License

For now, this library contains simple wrapping structure around Prometheus metrics. This improves usage and developer experience of defining metrics for StreamingFast services. It is part of StreamingFast.

This library should be kept as SMALL as possible, as it is a dependency we want to sprinkle around widely.

Usage

All metrics must be created from a metrics Set. The idea is that each library and micro-services defines a single set of metrics.

All metrics to be published by Prometheus must be registered at some point.

Example metrics.go in an example bstream project:

// Exported, so it can be registered by a `metrics.go` in main packages.
var MetricsSet = dmetrics.NewSet()

// HitCount is exported if used by other packages elsewhere, use like `bstream.HitCount.Inc()`
var HitCount = MetricsSet.NewGauge("bstream_hit_count", "hello %s")

// myCount is not exported, because used only in here.  Use as: `myCount.Inc()`
var myCount = MetricsSet.NewGauge("bstream_my_count", "hello %s")

In a main package, in a metrics.go (similar to logging.go):

func init() {
    dmetrics.Register(
        bstream.MetricsSet,
        dauth.MetricsSet,
        blockmeta.MetricsSet,
    )
}

Background

Initially, we had defined our metrics directly as the Prometheus type giving a definition of metrics in the form:

var mapSize = newGauge(
	"map_size",
	"size of live blocks map",
)

func IncMapSize() {
	mapSize.Inc()
}

func DecMapSize() {
	mapSize.Dec()
}

The usage of this was then like this:

    metrics.IncMapSize()

    ...

    metrics.DecMapSize()

This was repeated for all metrics then defined. This is problematic as when there is multiple metrics, the source file for definitions becomes bloated with lots of repeated stuff and duplicated stuff.

To overcome this, this library wraps different Prometheus metrics to clean down the definitions file. and offer a nicer API around them , and also usage. The previous example can now be turned into:

var MapSize = dmetrics.NewGauge("map_size", "size of live blocks map")

And the usage is now like:

    metrics.MapSize.Inc()

    ...

    metrics.MapSize.Dec()

An incredible improvement in the definitions of the metrics themselves.

Contributing

Issues and PR in this repo related strictly to the dmetrics library.

Report any protocol-specific issues in their respective repositories

Please first refer to the general StreamingFast contribution guide, if you wish to contribute to this code base.

License

Apache 2.0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var InferUnit = time.Duration(0)
View Source 
var NoOpPrometheusRegister = func(c ...prometheus.Collector) {}
View Source 
var PrometheusRegister = prometheus.MustRegister

Functions

func Register 

func Register(sets ...*Set)

Register from `metrics.go`

func Serve 

func Serve(addr string)

Types

type AppReadiness 

type AppReadiness struct {
	// contains filtered or unexported fields
}

func (*AppReadiness) SetNotReady 

func (a *AppReadiness) SetNotReady()

func (*AppReadiness) SetReady 

func (a *AppReadiness) SetReady()

type AvgCounter 

type AvgCounter struct {
	// contains filtered or unexported fields
}

func NewAvgCounter 

func NewAvgCounter(samplingWindow time.Duration, eventType string) *AvgCounter

NewAvgCounter allows you to get the average of an event over the period of time. For example, if you want to know the average cache hits in a given time

Over 1 second, you will increment the average by the number of cache hits

``` counter := NewAvgCounter(1*time.Second, "cache hits") counter.IncBy(283) counter.IncBy(23) counter.IncBy(192) counter.IncBy(392)

counter.String() == avg 222.5 cache hits (over 1s) [12321 total] ```

func (*AvgCounter) Average 

func (c *AvgCounter) Average() float64

func (*AvgCounter) AverageString 

func (c *AvgCounter) AverageString() string

func (*AvgCounter) IncBy 

func (c *AvgCounter) IncBy(value int64)

IncBy adds multiple events (useful for debouncing event counts)

func (*AvgCounter) String 

func (c *AvgCounter) String() string

func (*AvgCounter) Total 

func (c *AvgCounter) Total() uint64

type AvgDurationCounter 

type AvgDurationCounter struct {
	// contains filtered or unexported fields
}

func NewAvgDurationCounter 

func NewAvgDurationCounter(samplingWindow time.Duration, unit time.Duration, description string) *AvgDurationCounter

NewAvgDurationCounter allows you to get teh average elapsed time of a given process As an example, if you want to know the average block process time.

Example: if over 1 second you process 3 blocks where the processing time respectively takes 2s, 5s, 300ms. The average will yield a result of 2.43s per block.

``` counter := NewAvgDurationCounter(30*time.Second, time.Second, "per block") counter.AddDuration(2 * time.Second) counter.AddDuration(5 * time.Second) counter.AddDuration(300 * time.Millisecond)

counter.String() == 2.43s per block (avg over 30s) ```

The `unit` parameter can be 0, in which case the unit will be inferred based on the actual duration, e.g. if the average is 1.5s, the unit will be 1s while if the average is 10us, the unit will be 10us.

func (*AvgDurationCounter) AddDuration 

func (c *AvgDurationCounter) AddDuration(dur time.Duration)

func (*AvgDurationCounter) AddElapsedTime 

func (c *AvgDurationCounter) AddElapsedTime(start time.Time)

func (*AvgDurationCounter) Average 

func (c *AvgDurationCounter) Average() time.Duration

func (*AvgDurationCounter) AverageString 

func (c *AvgDurationCounter) AverageString() string

func (*AvgDurationCounter) String 

func (c *AvgDurationCounter) String() string

func (*AvgDurationCounter) Total 

func (c *AvgDurationCounter) Total() time.Duration

type AvgRateCounter 

type AvgRateCounter struct {
	// contains filtered or unexported fields
}

func MustNewAvgRateCounter 

func MustNewAvgRateCounter(samplingWindow time.Duration, period time.Duration, unit string) *AvgRateCounter

func NewAvgRateCounter 

func NewAvgRateCounter(samplingWindow time.Duration, period time.Duration, unit string) (*AvgRateCounter, error)

NewAvgRateCounter Tracks the average rate over a period of time. The rate is computed by accumulating the total count at the <samplingWindow> interval and averaging them our ove the number of <period> defined. Suppose there is a block count that increments as follows 

0s to 1s -> 10 blocks
1s to 2s -> 3 blocks
2s to 3s -> 0 blocks
3s to 4s -> 7 blocks

If your samplingWindow = 1s and your period = 4s, the rate will be computed as 

(10 + 3 + 0 + 7)/4 = 5 blocks/sec

If your samplingWindow = 1s and your period = 3s, the rate will be computed as 

(10 + 3 + 0)/4 = 4.33 blocks/sec

then when the "window moves" you would get 

(3 + 0 + 7)/4 = 3.333 blocks/sec

func (*AvgRateCounter) Add 

func (a *AvgRateCounter) Add(v uint64)

Add tracks a number of events, to be used to compute the rage

func (AvgRateCounter) Rate 

func (c AvgRateCounter) Rate() float64

func (AvgRateCounter) RateString 

func (c AvgRateCounter) RateString() string

func (*AvgRateCounter) Stop 

func (c *AvgRateCounter) Stop()

func (AvgRateCounter) String 

func (c AvgRateCounter) String() string

func (AvgRateCounter) SyncNow 

func (c AvgRateCounter) SyncNow()

SyncNow forces a sync to retrieve the value(s) of the counter(s) and update the average rate.

This call is blocking until the sync is performed.

func (AvgRateCounter) Total 

func (c AvgRateCounter) Total() uint64

type AvgRatePromCounter 

type AvgRatePromCounter struct {
	// contains filtered or unexported fields
}

func MustNewAvgRateFromPromCounter 

func MustNewAvgRateFromPromCounter(promCollector prometheus.Collector, samplingWindow time.Duration, period time.Duration, unit string) *AvgRatePromCounter

MustNewAvgRateFromPromCounter acts like NewAvgRateFromPromCounter but panics if an error occurs. Refers to NewAvgRateFromPromCounter for more information.

func NewAvgRateFromPromCounter 

func NewAvgRateFromPromCounter(promCollector prometheus.Collector, samplingWindow time.Duration, period time.Duration, unit string) (*AvgRatePromCounter, error)

NewAvgRateFromPromCounter Extracts the average rate of a Prom Collector over a period of time. The rate is computed by accumulating the total count at the <samplingWindow> interval and averaging them our ove the number of <period> defined. Suppose there is a block count that increments as follows 

0s to 1s -> 10 blocks
1s to 2s -> 3 blocks
2s to 3s -> 0 blocks
3s to 4s -> 7 blocks

If your samplingWindow = 1s and your period = 4s, the rate will be computed as 

(10 + 3 + 0 + 7)/4 = 5 blocks/sec

If your samplingWindow = 1s and your period = 3s, the rate will be computed as 

(10 + 3 + 0)/4 = 4.33 blocks/sec

then when the "window moves" you would get 

(3 + 0 + 7)/4 = 3.333 blocks/sec

func (AvgRatePromCounter) Stop 

func (c AvgRatePromCounter) Stop()

type AvgRatePromGauge 

type AvgRatePromGauge struct {
	// contains filtered or unexported fields
}

func MustNewAvgRateFromPromGauge 

func MustNewAvgRateFromPromGauge(promCollector prometheus.Collector, samplingWindow time.Duration, period time.Duration, unit string) *AvgRatePromGauge

MustNewAvgRateFromPromGauge acts like NewAvgRateFromPromGauge but panics if an error occurs. Refers to NewAvgRateFromPromGauge for more information.

func NewAvgRateFromPromGauge 

func NewAvgRateFromPromGauge(promCollector prometheus.Collector, samplingWindow time.Duration, period time.Duration, unit string) (*AvgRatePromGauge, error)

NewAvgRateFromPromGauge extracts the average rate of a Promtheus Gauge over a period of time. The rate is computed by accumulating the total count at the <samplingWindow> interval and averaging them our ove the number of <period> defined. Suppose there is a block count that increments as follows 

0s to 1s -> 10 blocks
1s to 2s -> 3 blocks
2s to 3s -> 0 blocks
3s to 4s -> 7 blocks

If your samplingWindow = 1s and your period = 4s, the rate will be computed as 

(10 + 3 + 0 + 7)/4 = 5 blocks/sec

If your samplingWindow = 1s and your period = 3s, the rate will be computed as 

(10 + 3 + 0)/4 = 4.33 blocks/sec

then when the "window moves" you would get 

(3 + 0 + 7)/4 = 3.333 blocks/sec

**Important** Your Gauge should be ever increasing. If it's not, the results will be incorrect.

func (AvgRatePromGauge) Stop 

func (c AvgRatePromGauge) Stop()

type CountableFunc 

type CountableFunc = func() uint64

type Counter 

type Counter struct {
	// contains filtered or unexported fields
}

func (*Counter) AddFloat64 

func (c *Counter) AddFloat64(value float64)

func (*Counter) AddInt 

func (c *Counter) AddInt(value int)

func (*Counter) AddInt64 

func (c *Counter) AddInt64(value int64)

func (*Counter) AddUint64 

func (c *Counter) AddUint64(value uint64)

func (*Counter) Collect 

func (g *Counter) Collect(in chan<- prometheus.Metric)

func (*Counter) Describe 

func (g *Counter) Describe(in chan<- *prometheus.Desc)

func (*Counter) Inc 

func (c *Counter) Inc()

func (*Counter) Native 

func (c *Counter) Native() prometheus.Counter

type CounterVec 

type CounterVec struct {
	// contains filtered or unexported fields
}

func (*CounterVec) AddFloat64 

func (c *CounterVec) AddFloat64(value float64, labels ...string)

func (*CounterVec) AddInt 

func (c *CounterVec) AddInt(value int, labels ...string)

func (*CounterVec) AddInt64 

func (c *CounterVec) AddInt64(value int64, labels ...string)

func (*CounterVec) AddUint64 

func (c *CounterVec) AddUint64(value uint64, labels ...string)

func (*CounterVec) Collect 

func (g *CounterVec) Collect(in chan<- prometheus.Metric)

func (*CounterVec) DeleteLabelValues 

func (c *CounterVec) DeleteLabelValues(labels ...string)

func (*CounterVec) Describe 

func (g *CounterVec) Describe(in chan<- *prometheus.Desc)

func (*CounterVec) Inc 

func (c *CounterVec) Inc(labels ...string)

func (*CounterVec) Native 

func (g *CounterVec) Native() *prometheus.CounterVec

type Gauge 

type Gauge struct {
	// contains filtered or unexported fields
}

func (*Gauge) Collect 

func (g *Gauge) Collect(in chan<- prometheus.Metric)

func (*Gauge) Dec 

func (g *Gauge) Dec()

func (*Gauge) Describe 

func (g *Gauge) Describe(in chan<- *prometheus.Desc)

func (*Gauge) Inc 

func (g *Gauge) Inc()

func (*Gauge) Native 

func (g *Gauge) Native() prometheus.Gauge

func (*Gauge) SetFloat64 

func (g *Gauge) SetFloat64(value float64)

func (*Gauge) SetUint64 

func (g *Gauge) SetUint64(value uint64)

type GaugeVec 

type GaugeVec struct {
	// contains filtered or unexported fields
}

func (*GaugeVec) Collect 

func (g *GaugeVec) Collect(in chan<- prometheus.Metric)

func (*GaugeVec) Dec 

func (g *GaugeVec) Dec(labels ...string)

func (*GaugeVec) DeleteLabelValues 

func (g *GaugeVec) DeleteLabelValues(labels ...string)

func (*GaugeVec) Describe 

func (g *GaugeVec) Describe(in chan<- *prometheus.Desc)

func (*GaugeVec) Inc 

func (g *GaugeVec) Inc(labels ...string)

func (*GaugeVec) Native 

func (g *GaugeVec) Native() *prometheus.GaugeVec

func (*GaugeVec) SetFloat64 

func (g *GaugeVec) SetFloat64(value float64, labels ...string)

func (*GaugeVec) SetInt 

func (g *GaugeVec) SetInt(value int, labels ...string)

func (*GaugeVec) SetInt64 

func (g *GaugeVec) SetInt64(value int64, labels ...string)

func (*GaugeVec) SetUint64 

func (g *GaugeVec) SetUint64(value uint64, labels ...string)

type HeadBlockNum 

type HeadBlockNum struct {
	// contains filtered or unexported fields
}

func (*HeadBlockNum) SetUint64 

func (h *HeadBlockNum) SetUint64(blockNum uint64)

type HeadTimeDrift 

type HeadTimeDrift struct {
	// contains filtered or unexported fields
}

func (*HeadTimeDrift) SetBlockTime 

func (h *HeadTimeDrift) SetBlockTime(blockTime time.Time)

type Histogram 

type Histogram struct {
	// contains filtered or unexported fields
}

func (*Histogram) Collect 

func (h *Histogram) Collect(in chan<- prometheus.Metric)

func (*Histogram) Describe 

func (h *Histogram) Describe(in chan<- *prometheus.Desc)

func (*Histogram) Native 

func (h *Histogram) Native() prometheus.Histogram

func (*Histogram) ObserveDuration 

func (h *Histogram) ObserveDuration(value time.Duration)

func (*Histogram) ObserveFloat64 

func (h *Histogram) ObserveFloat64(value float64)

func (*Histogram) ObserveInt 

func (h *Histogram) ObserveInt(value int64)

func (*Histogram) ObserveInt64 

func (h *Histogram) ObserveInt64(value int64)

func (*Histogram) ObserveSince 

func (h *Histogram) ObserveSince(value time.Time)

func (*Histogram) ObserveUint64 

func (h *Histogram) ObserveUint64(value int64)

type HistogramVec 

type HistogramVec struct {
	// contains filtered or unexported fields
}

func (*HistogramVec) Collect 

func (h *HistogramVec) Collect(in chan<- prometheus.Metric)

func (*HistogramVec) DeleteLabelValues 

func (h *HistogramVec) DeleteLabelValues(labels ...string)

func (*HistogramVec) Describe 

func (h *HistogramVec) Describe(in chan<- *prometheus.Desc)

func (*HistogramVec) Native 

func (h *HistogramVec) Native() *prometheus.HistogramVec

func (*HistogramVec) ObserveDuration 

func (h *HistogramVec) ObserveDuration(value time.Duration, labels ...string)

func (*HistogramVec) ObserveFloat64 

func (h *HistogramVec) ObserveFloat64(value float64, labels ...string)

func (*HistogramVec) ObserveInt 

func (h *HistogramVec) ObserveInt(value int64, labels ...string)

func (*HistogramVec) ObserveInt64 

func (h *HistogramVec) ObserveInt64(value int64, labels ...string)

func (*HistogramVec) ObserveSince 

func (h *HistogramVec) ObserveSince(value time.Time, labels ...string)

func (*HistogramVec) ObserveUint64 

func (h *HistogramVec) ObserveUint64(value int64, labels ...string)

type Metric 

type Metric interface {
	prometheus.Collector
}

type Option 

type Option func(s *Set)

func PrefixNameWith 

func PrefixNameWith(prefix string) Option

PrefixNameWith will prefix all metric of this given set using the following prefix.

type RateCounter 

type RateCounter struct {
	// contains filtered or unexported fields
}

func NewPerMinuteLocalRateCounter 

func NewPerMinuteLocalRateCounter(unit string) *RateCounter

func NewPerSecondLocalRateCounter 

func NewPerSecondLocalRateCounter(unit string) *RateCounter

func NewRateCounter 

func NewRateCounter(interval time.Duration, unit string) *RateCounter

NewRateCounter allows you to know how many times an event happen over a fixed period of time

For example, if over 1 second you process 20 blocks, then querying the counter within this 1s samplingWindow will yield a result of 20 blocks/s. The rate change as the time moves.

``` counter := NewRateCounter(1*time.Second, "s", "blocks") counter.Incr() counter.Incr() counter.Incr()

counter.String() == 3 blocks/s (over 1s)

IMPORTANT: The rate is calculated by the number of events that occurred in the last sampling window thus if from time 0s to 1s you have 3 events the rate is 3 events/sec; subsequently if you wait another 2 seconds without any event occurring, then query the rate you will get 0 event/sec. If you want an Avg rate over multiple sampling window Use AvgRateCounter ```

func (*RateCounter) Inc 

func (c *RateCounter) Inc()

Incr add 1 event into the RateCounter

func (*RateCounter) IncBy 

func (c *RateCounter) IncBy(value int64)

IncrBy adds multiple events inot the RateCounter

func (*RateCounter) Rate 

func (c *RateCounter) Rate() int64

func (*RateCounter) RateString 

func (c *RateCounter) RateString() string

func (*RateCounter) String 

func (c *RateCounter) String() string

func (*RateCounter) Total 

func (c *RateCounter) Total() uint64

type Set 

type Set struct {
	// contains filtered or unexported fields
}

func NewSet 

func NewSet(options ...Option) *Set

NewSet creates a set of metrics that can then be used to create a varieties of specific metrics (Gauge, Counter, Histogram).

func (*Set) NewAppReadiness 

func (s *Set) NewAppReadiness(service string) *AppReadiness

func (*Set) NewCounter 

func (s *Set) NewCounter(name string, helpChunks ...string) *Counter

func (*Set) NewCounterVec 

func (s *Set) NewCounterVec(name string, labels []string, helpChunks ...string) *CounterVec

func (*Set) NewGauge 

func (s *Set) NewGauge(name string, helpChunks ...string) *Gauge

func (*Set) NewGaugeVec 

func (s *Set) NewGaugeVec(name string, labels []string, helpChunks ...string) *GaugeVec

func (*Set) NewHeadBlockNumber 

func (s *Set) NewHeadBlockNumber(service string) *HeadBlockNum

func (*Set) NewHeadTimeDrift 

func (s *Set) NewHeadTimeDrift(service string) *HeadTimeDrift

func (*Set) NewHistogram 

func (s *Set) NewHistogram(name string, helpChunks ...string) *Histogram

func (*Set) NewHistogramVec 

func (s *Set) NewHistogramVec(name string, labels []string, helpChunks ...string) *HistogramVec

func (*Set) Register 

func (s *Set) Register()

type ValueFromMetric 

type ValueFromMetric struct {
	// contains filtered or unexported fields
}

ValueFromMetric can be used to extract the value of a Prometheus Metric object.

*Important* This for now does not correctly handles `Vec` like metrics since the actual logic is to return the first ever value encountered while in a `Vec` metric, there is usually N values, one per label.

func NewValueFromMetric 

func NewValueFromMetric(metric prometheus.Collector, unit string) *ValueFromMetric

func (*ValueFromMetric) ValueFloat 

func (c *ValueFromMetric) ValueFloat() float64

func (*ValueFromMetric) ValueUint 

func (c *ValueFromMetric) ValueUint() uint64

type ValuesFromMetric 

type ValuesFromMetric struct {
	// contains filtered or unexported fields
}

ValuesFromMetric can be used to extract the values of a Prometheus Metric 'Vec' object, i.e. with a 'label' dimension.

func NewValuesFromMetric 

func NewValuesFromMetric(metric prometheus.Collector) *ValuesFromMetric

func (*ValuesFromMetric) Floats 

func (c *ValuesFromMetric) Floats(label string) map[string]float64

Floats(label string) gets you the float64 values for each value of the given label. Values without that label or with a nil value for that label are discarded

func (*ValuesFromMetric) Uints 

func (c *ValuesFromMetric) Uints(label string) map[string]uint64

Uints(label string) converts values to uint64 of the Floats(label string) function

Source Files

View all Source files

Directories

Path Synopsis
Package ring implements operations on circular lists, copied over to make it generic
 Package ring implements operations on circular lists, copied over to make it generic

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.