crow

Documentation

Overview

Package crow implements a correctness checker tool similar to Loki Canary. Inspired by Cortex test-exporter.

Index

• Variables
• type Config
  • func (c *Config) RegisterFlags(f *flag.FlagSet)
  • func (c *Config) RegisterFlagsWithPrefix(f *flag.FlagSet, prefix string)
• type Crow
  • func New(cfg Config) (*Crow, error)
  • func (c *Crow) StateMetrics() prometheus.Collector
  • func (c *Crow) Stop()
  • func (c *Crow) TestMetrics() prometheus.Collector

Variables

var DefaultConfig = Config{
	MaxValidations: 5,
	NumSamples:     10,

	QueryTimeout:  150 * time.Millisecond,
	QueryDuration: 2 * time.Second,
	QueryStep:     100 * time.Millisecond,

	MaxTimestampDelta: 750 * time.Millisecond,
	ValueEpsilon:      0.0001,
}

DefaultConfig holds defaults for Crow settings.

Types

type Config

type Config struct {
	PrometheusAddr string // Base URL of Prometheus server
	NumSamples     int    // Number of samples to generate
	UserID         string // User ID to use for auth when querying.
	PasswordFile   string // Password File for auth when querying.
	ExtraSelectors string // Extra selectors for queries, i.e., cluster="prod"
	OrgID          string // Org ID to inject in X-Org-ScopeID header when querying.

	QueryTimeout  time.Duration // Timeout for querying
	QueryDuration time.Duration // Time before and after sample to search
	QueryStep     time.Duration // Step between samples in search

	MaxValidations    int           // Maximum amount of times to search for a sample
	MaxTimestampDelta time.Duration // Maximum timestamp delta to use for validating.
	ValueEpsilon      float64       // Maximum epsilon to use for validating.

	// Logger to use. If nil, logs will be discarded.
	Log log.Logger
}

Config for the Crow metrics checker.

func (*Config) RegisterFlags

func (c *Config) RegisterFlags(f *flag.FlagSet)

RegisterFlags registers flags for the config to the given FlagSet.

func (*Config) RegisterFlagsWithPrefix

func (c *Config) RegisterFlagsWithPrefix(f *flag.FlagSet, prefix string)

RegisterFlagsWithPrefix registers flags for the config to the given FlagSet and prefixing each flag with the given prefix. prefix, if non-empty, should end in `.`.

type Crow

type Crow struct {
	// contains filtered or unexported fields
}

Crow is a correctness checker that validates scraped metrics reach a Prometheus-compatible server with the same values and roughly the same timestamp.

Crow exposes two sets of metrics:

1. Test metrics, where each scrape generates a validation job. 2. State metrics, exposing state of the Crow checker itself.

These two metrics should be exposed via different endpoints, and only state metrics are safe to be manually collected from.

Collecting from the set of test metrics generates a validation job, where Crow will query the Prometheus API to ensure the metrics that were scraped were written with (approximately) the same timestamp as the scrape time and with (approximately) the same floating point values exposed in the scrape.

If a set of test metrics were not found and retries have been exhausted, or if the metrics were found but the values did not match, the error counter will increase.

func New

func New(cfg Config) (*Crow, error)

New creates a new Crow.

func (*Crow) StateMetrics

func (c *Crow) StateMetrics() prometheus.Collector

StateMetrics exposes metrics of Crow itself. These metrics are not validated for presence in the remote system.

func (*Crow) Stop

func (c *Crow) Stop()

Stop stops crow. Panics if Stop is called more than once.

func (*Crow) TestMetrics

func (c *Crow) TestMetrics() prometheus.Collector

TestMetrics exposes a collector of test metrics. Each collection will schedule a validation job.