registry

package
v0.0.0-...-69690f0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 24, 2024 License: Apache-2.0 Imports: 37 Imported by: 0

Documentation

Overview

Package registry package implement a dynamic collection of metrics sources

It support both pushed metrics (using AddMetricPointFunction) and pulled metrics thought Collector or Gatherer

Index

Constants

View Source 
const (
	// NoProbe is the default value. Probes can be very costly as it involves network calls, hence it is disabled by default.
	NoProbe queryType = iota
	// OnlyProbes specifies we only want data from the probes.
	OnlyProbes
	// All specifies we want all the data, including probes.
	All
	// FromStore use the store (queryable) to get recent stored data.
	FromStore
)

Variables

View Source 
var (
	ErrBadArgument = errors.New("bad argument")
)

Functions

func DefaultSNMPRules 

func DefaultSNMPRules(resolution time.Duration) []types.SimpleRule

func JitterForTime 

func JitterForTime(align time.Time, interval time.Duration) uint64

JitterForTime return a Jitter such as scrape run at align + N * interval.

func WaitForSecrets 

func WaitForSecrets(ctx context.Context, secretsGate *gate.Gate, slotsNeeded int) (func(), error)

WaitForSecrets hold the current goroutine until the given number of slots are taken. This is to ensure that too many inputs with secrets don't run at the same time, which would result in exceeding the locked memory limit. WaitForSecrets returns a callback to release all the slots taken once the gathering is done, or an error if the given context expired.

Types

type AppenderCallback 

type AppenderCallback interface {
	// Collect collects point and write them into Appender. The appender must not be used once Collect returned.
	// If you omit to Commit() on the appender, it will be automatically done when Collect return without error.
	CollectWithState(ctx context.Context, state GatherState, app storage.Appender) error
}

type AppenderFunc 

type AppenderFunc func(ctx context.Context, state GatherState, app storage.Appender) error

func (AppenderFunc) CollectWithState 

func (af AppenderFunc) CollectWithState(ctx context.Context, state GatherState, app storage.Appender) error

type GatherState 

type GatherState struct {
	QueryType queryType
	// FromScrapeLoop tells whether the gathering is done by the periodic scrape loop or for /metrics endpoint
	FromScrapeLoop bool
	T0             time.Time
	NoFilter       bool
	// HintMetricFilter is an optional filter that gather could use to skip metrics that would be filtered later.
	// Nothing is mandatory: the HintMetricFilter could be nil and the gatherer could ignore HintMetricFilter even if non-nil.
	// If used the filter should be applied after any label alteration.
	HintMetricFilter func(lbls labels.Labels) bool
}

GatherState is an argument given to gatherers that support it. It allows us to give extra information to gatherers. Due to the way such objects are constructed when no argument is supplied (when calling Gather() on a GathererWithState, most of the time Gather() will directly call GatherWithState(GatherState{}), please make sure that default values are sensible. For example, NoProbe *must* be the default queryType, as we do not want queries on /metrics to always probe the collectors by default).

func GatherStateFromMap 

func GatherStateFromMap(params map[string][]string) GatherState

GatherStateFromMap creates a GatherState from a state passed as a map.

func (GatherState) Now 

func (s GatherState) Now() time.Time

type GathererWithOrWithoutState 

type GathererWithOrWithoutState interface {
	Gather() ([]*dto.MetricFamily, error)
	GatherWithState(ctx context.Context, state GatherState) ([]*dto.MetricFamily, error)
}

func WithPastPointFilter 

func WithPastPointFilter(gatherer GathererWithOrWithoutState, purgeInterval time.Duration) GathererWithOrWithoutState

WithPastPointFilter wraps the given gatherer with a filter that will ensure that every emitted point will not have a timestamp before the latest sent. The filter will purge its cache at the given purgeInterval.

type GathererWithScheduleUpdate 

type GathererWithScheduleUpdate interface {
	SetScheduleUpdate(scheduleUpdate func(runAt time.Time))
}

GathererWithScheduleUpdate is a Gatherer that had a ScheduleUpdate (like Probe gatherer). The ScheduleUpdate could be used to trigger an additional gather earlier than default scrape interval.

type GathererWithState 

type GathererWithState interface {
	GatherWithState(ctx context.Context, state GatherState) ([]*dto.MetricFamily, error)
}

GathererWithState is a generalization of prometheus.Gather.

type GathererWithStateWrapper 

type GathererWithStateWrapper struct {
	// contains filtered or unexported fields
}

GathererWithStateWrapper is a wrapper around GathererWithState that allows to specify a state to forward to the wrapped gatherer when the caller does not know about GathererWithState and uses raw Gather(). The main use case is the /metrics HTTP endpoint, where we want to be able to gather() only some metrics (e.g. all metrics/only probes/no probes). In the prometheus exporter endpoint, when receiving an request, the (user-provided) HTTP handler will: - create a new wrapper instance, generate a GatherState accordingly, and call wrapper.setState(newState). - pass the wrapper to a new prometheus HTTP handler. - when Gather() is called upon the wrapper by prometheus, the wrapper calls GathererWithState(newState) on its internal gatherer. GatherWithState also contains the metrics allow/deny list in order to sync the metrics on /metric with the metrics sent to the bleemeo platform.

func NewGathererWithStateWrapper 

func NewGathererWithStateWrapper(ctx context.Context, g GathererWithState, filter metricFilter) *GathererWithStateWrapper

NewGathererWithStateWrapper creates a new wrapper around GathererWithState.

func (*GathererWithStateWrapper) Gather 

func (w *GathererWithStateWrapper) Gather() ([]*dto.MetricFamily, error)

Gather implements prometheus.Gatherer for GathererWithStateWrapper.

func (*GathererWithStateWrapper) SetState 

func (w *GathererWithStateWrapper) SetState(state GatherState)

SetState updates the state the wrapper will provide to its internal gatherer when called.

type Option 

type Option struct {
	PushPoint             types.PointPusher
	ThresholdHandler      ThresholdHandler
	Queryable             storage.Queryable
	FQDN                  string
	GloutonPort           string
	MetricFormat          types.MetricFormat
	BlackboxSendScraperID bool
	Filter                metricFilter
	SecretInputsGate      *gate.Gate
	ShutdownDeadline      time.Duration
}

type PointBuffer 

type PointBuffer struct {
	// contains filtered or unexported fields
}

PointBuffer add points received from PushPoints to a buffer.

func (*PointBuffer) Points 

func (p *PointBuffer) Points() []types.MetricPoint

Points returns the buffer and resets it.

func (*PointBuffer) PushPoints 

func (p *PointBuffer) PushPoints(_ context.Context, points []types.MetricPoint)

PushPoints adds points to the buffer.

type ProbeGatherer 

type ProbeGatherer struct {
	// contains filtered or unexported fields
}

ProbeGatherer is a specific gatherer that wraps probes, choose when to Gather() depending on the GatherState argument.

func NewProbeGatherer 

func NewProbeGatherer(gatherer prometheus.Gatherer, fastUpdateOnFailure bool) *ProbeGatherer

NewProbeGatherer creates a new ProbeGatherer with the prometheus gatherer specified.

func (*ProbeGatherer) Gather 

func (p *ProbeGatherer) Gather() ([]*dto.MetricFamily, error)

Gather some metrics with an empty gatherer state. While not a critical error, this function should never be called, as callers should know about GatherWithState().

func (*ProbeGatherer) GatherWithState 

func (p *ProbeGatherer) GatherWithState(ctx context.Context, state GatherState) ([]*dto.MetricFamily, error)

GatherWithState uses the specified gather state along the gatherer to retrieve a set of metrics.

func (*ProbeGatherer) SetScheduleUpdate 

func (p *ProbeGatherer) SetScheduleUpdate(fun func(runAt time.Time))

SetScheduleUpdate is called by Registry because this gathered will be used.

type RegistrationOption 

type RegistrationOption struct {
	Description  string
	JitterSeed   uint64
	MinInterval  time.Duration
	Timeout      time.Duration
	StopCallback func() `json:"-"`
	// ExtraLabels are labels added. If a labels already exists, extraLabels takes precedence.
	ExtraLabels map[string]string
	// NoLabelsAlteration disable (most) alteration of labels. It doesn't apply to PushPoints.
	// Meta labels (starting with __) are still dropped and (if applicable) converted to annotations.
	NoLabelsAlteration bool
	// CompatibilityNameItem enable renaming metrics labels to just name + item (from Annotations.BleemeoItem).
	// This should eventually be dropped once all metrics are produced with right name + item directly rather than using
	// Annotations.BleemeoItem. This compatibility was mostly needed when Bleemeo didn't supported labels and only had
	// name + item. If dropped, we should be careful to don't change existing metrics.
	// Note: it doesn't impact labels from ExtraLabels (including relabeling processing), it only drop labels
	// from each points from the gatherer itself, the ExtraLabels after relabeling (a.k.a "LabelUsed" in diagnostic) are still
	// added.
	CompatibilityNameItem bool
	// DisablePeriodicGather skip the periodic calls which forward gathered points to r.PushPoint.
	// The periodic call uses the MinInterval. When MinInterval is 0, the dynamic interval set by UpdateDelay is used.
	DisablePeriodicGather bool
	// ApplyDynamicRelabel controls whether the metrics should go through the relabel hook.
	ApplyDynamicRelabel bool
	Rules               []types.SimpleRule
	// GatherModifier is a function that can modify the gather result (add/modify/delete). It is called after Rules
	// are applied, just before ExtraLabels is done.
	// It could be nil to skip this step.
	GatherModifier gatherModifier `json:"-"`
	// IsEssential tells whether the corresponding gatherer is considered as 'essential' regarding the agent dashboard.
	// When all 'essentials' gatherers are stuck, Glouton kills himself (see Registry.HealthCheck).
	IsEssential bool
	// AcceptAllowedMetricsOnly will only keep metrics allowed at ends of Gather(), so the
	// metric not allowed by allow_list (or metric denied) will be dropped. Metrics that are
	// needed by SimpleRule will still be allowed.
	// Currently (until Registry.renamer is dropped), this shouldn't be activated on SNMP gatherer.
	AcceptAllowedMetricsOnly bool
	// HonorTimestamp indicate whether timestamp associated with each metric point is used or if a timestamp
	// decided by the Registry is used. Using the timestamp of the registry is preferred as its more stable.
	// If you need mixed timestamp decided by the Registry and timestamp associated with some points, use a
	// zero time (time.Time{}, Unix epoc or nil) on point that need to use the Registry's timestamp.
	// This is currently only implemented for AppenderCallback.
	HonorTimestamp bool
	// CallForMetricsEndpoint indicate whether the callback must be called for /metrics or
	// cached result from last periodic collection is used.
	CallForMetricsEndpoint bool
	// If labels has a container name (in the meta-labels), use it when building the "instance" label
	InstanceUseContainerName bool
	// contains filtered or unexported fields
}

func (*RegistrationOption) String 

func (opt *RegistrationOption) String() string

type Registry 

type Registry struct {
	// contains filtered or unexported fields
}

Registry is a dynamic collection of metrics sources.

For the Prometheus metrics source, it mostly a wrapper around prometheus.Gatherers, but it allow to attach labels to each Gatherers. It also support pushed metrics.

It is used by Glouton for two main purpose:

  • provide metrics on /metrics endpoints. For this gather of metrics is (mostly) done when an HTTP query reach /metrics.
  • provide metrics to be sent to stored in local store + sent to Bleemeo. Here gather of metrics is done periodically.

It may contains two kind of metrics source:

  • Prometheus Gatherer. When adding a Gatherer additional labels could be added
  • "push" callback, which are function that use PushPoints() to add points to the registry buffer. Points in this buffer are send when /metrics is queried. Push callbacks are only called periodically by RunCollection, they are NOT called on query to /metrics.

Points send to local store (which forward them to Bleemeo) are:

  • any point pushed using PushPoints()
  • any points returned by a registered Gatherer when pushPoint option was set when gatherer was registered.

func New 

func New(opt Option) (*Registry, error)

func (*Registry) AddDefaultCollector 

func (r *Registry) AddDefaultCollector()

AddDefaultCollector adds the following collectors: GoCollector and ProcessCollector like the prometheus.DefaultRegisterer Internal registry which contains all glouton metrics.

func (*Registry) AddNodeExporter 

func (r *Registry) AddNodeExporter(option node.Option, vethProvider *veth.Provider) error

AddNodeExporter add a node_exporter to collector.

func (*Registry) DiagnosticArchive 

func (r *Registry) DiagnosticArchive(ctx context.Context, archive types.ArchiveWriter) error

func (*Registry) Exporter 

func (r *Registry) Exporter() http.Handler

Exporter return an HTTP exporter.

func (*Registry) Gather 

func (r *Registry) Gather() ([]*dto.MetricFamily, error)

Gather implements prometheus.Gatherer.

func (*Registry) GatherWithState 

func (r *Registry) GatherWithState(ctx context.Context, state GatherState) ([]*dto.MetricFamily, error)

GatherWithState implements GathererGatherWithState.

func (*Registry) HealthCheck 

func (r *Registry) HealthCheck()

HealthCheck perform some health check and log any issue found. This method could panic when health conditions are bad for too long in order to cause a Glouton restart.

func (*Registry) InternalRunScrape 

func (r *Registry) InternalRunScrape(ctx context.Context, loopCtx context.Context, t0 time.Time, id int)

InternalRunScrape run a scrape/gathering on given registration id (from RegisterGatherer & co). Points gatherer are processed at if a periodic gather occurred. This should only be used in test.

func (*Registry) RegisterAppenderCallback 

func (r *Registry) RegisterAppenderCallback(
	opt RegistrationOption,
	cb AppenderCallback,
) (int, error)

RegisterAppenderCallback add a callback that use an Appender to write points to the registry.

func (*Registry) RegisterGatherer 

func (r *Registry) RegisterGatherer(opt RegistrationOption, gatherer prometheus.Gatherer) (int, error)

RegisterGatherer add a new gatherer to the list of metric sources.

func (*Registry) RegisterInput 

func (r *Registry) RegisterInput(
	opt RegistrationOption,
	input telegraf.Input,
) (int, error)

RegisterInput uses a Telegraph input to write points to the registry.

func (*Registry) RegisterPushPointsCallback 

func (r *Registry) RegisterPushPointsCallback(opt RegistrationOption, f func(context.Context, time.Time)) (int, error)

RegisterPushPointsCallback add a callback that should push points to the registry. This callback will be called for each collection period. It's mostly used to add Telegraf input (using glouton/collector). Deprecated: use RegisterAppenderCallback. Note: before being able to drop pushpoint & registerpushpoint, we likely need:

  • support for "GathererWithScheduleUpdate-like" on RegisterAppenderCallback (needed by service check, when they trigger check on TCP close)
  • support for conversion of all annotation to meta-label and vise-vera (model/convert.go)
  • support for TTL ?

func (*Registry) Run 

func (r *Registry) Run(ctx context.Context) error

func (*Registry) ScheduleScrape 

func (r *Registry) ScheduleScrape(id int, runAt time.Time)

func (*Registry) Unregister 

func (r *Registry) Unregister(id int) bool

Unregister remove a Gatherer or PushPointCallback from the list of metric sources.

func (*Registry) UpdateRegistrationHooks 

func (r *Registry) UpdateRegistrationHooks(relabelHook RelabelHook, updateDelayHook UpdateDelayHook)

UpdateRegistrationHooks change the hook used just before relabeling and wait for all pending metrics emission. When this function return, it's guaratee that all call to Option.PushPoint will use new labels. The hook is assumed to be idempotent, that is for a given labels input the result is the same. If the hook want break this idempotence, UpdateRegistrationHooks() should be re-called to force update of existings Gatherer.

func (*Registry) WithTTL 

func (r *Registry) WithTTL(ttl time.Duration) types.PointPusher

WithTTL return a AddMetricPointFunction with TTL on pushed points.

type RelabelHook 

type RelabelHook func(ctx context.Context, labels map[string]string) (newLabel map[string]string, retryLater bool)

RelabelHook is a hook called just before applying relabeling. This hook receive the full labels (including meta labels) and is allowed to modify/add/delete them. The result (which should still include meta labels) is processed by relabeling rules. If the hook return retryLater, it means that hook can not processed the labels currently and it registry should retry later. Points or gatherer associated will be dropped.

type ThresholdHandler 

type ThresholdHandler interface {
	ApplyThresholds(points []types.MetricPoint) (newPoints []types.MetricPoint, statusPoints []types.MetricPoint)
}

type UpdateDelayHook 

type UpdateDelayHook func(labels map[string]string) (minInterval time.Duration, retryLater bool)

UpdateDelayHook returns the minimal gathering interval for the given labels. For example, according to the Bleemeo plan.

Source Files

View all Source files

Directories

Path Synopsis
internal
ruler

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.