collector

Documentation

Index

• type AggregateMetricsCollector
  • func (c *AggregateMetricsCollector) Collect(ch chan<- prometheus.Metric)
  • func (c *AggregateMetricsCollector) Describe(ch chan<- *prometheus.Desc)
• type CapacityPluginMetricsCollector
  • func (c *CapacityPluginMetricsCollector) Collect(ch chan<- prometheus.Metric)
  • func (c *CapacityPluginMetricsCollector) Describe(ch chan<- *prometheus.Desc)
• type CapacityPluginMetricsInstance
• type Collector
  • func NewCollector(cluster *core.Cluster, dbm *gorp.DbMap) *Collector
  • func (c *Collector) CapacityScrapeJob(registerer prometheus.Registerer) jobloop.Job
  • func (c *Collector) CheckConsistencyJob(registerer prometheus.Registerer) jobloop.Job
  • func (c *Collector) CleanupOldCommitmentsJob(registerer prometheus.Registerer) jobloop.Job
  • func (c *Collector) RateScrapeJob(registerer prometheus.Registerer) jobloop.Job
  • func (c *Collector) ResourceScrapeJob(registerer prometheus.Registerer) jobloop.Job
  • func (c *Collector) ScanDomains(opts ScanDomainsOpts) (result []string, resultErr error)
  • func (c *Collector) ScanDomainsAndProjectsJob(registerer prometheus.Registerer) jobloop.Job
  • func (c *Collector) ScanProjects(domain *db.Domain) (result []string, resultErr error)
  • func (c *Collector) SyncQuotaToBackendJob(registerer prometheus.Registerer) jobloop.Job
• type DataMetricsReporter
  • func (d *DataMetricsReporter) ServeHTTP(w http.ResponseWriter, r *http.Request)
• type QuotaPluginMetricsCollector
  • func (c *QuotaPluginMetricsCollector) Collect(ch chan<- prometheus.Metric)
  • func (c *QuotaPluginMetricsCollector) Describe(ch chan<- *prometheus.Desc)
• type QuotaPluginMetricsInstance
• type ScanDomainsOpts
• type TaskTiming
  • func (t TaskTiming) Duration() time.Duration

Types

type AggregateMetricsCollector

type AggregateMetricsCollector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
}

AggregateMetricsCollector is a prometheus.Collector that submits dynamically-calculated aggregate metrics about scraping progress.

func (*AggregateMetricsCollector) Collect

func (c *AggregateMetricsCollector) Collect(ch chan<- prometheus.Metric)

Collect implements the prometheus.Collector interface.

func (*AggregateMetricsCollector) Describe

func (c *AggregateMetricsCollector) Describe(ch chan<- *prometheus.Desc)

Describe implements the prometheus.Collector interface.

type CapacityPluginMetricsCollector

type CapacityPluginMetricsCollector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
	// When .Override is set, the DB is bypassed and only the given
	// CapacityPluginMetricsInstances are considered. This is used for testing only.
	Override []CapacityPluginMetricsInstance
}

CapacityPluginMetricsCollector is a prometheus.Collector that submits metrics which are specific to the selected capacity plugins.

func (*CapacityPluginMetricsCollector) Collect

func (c *CapacityPluginMetricsCollector) Collect(ch chan<- prometheus.Metric)

Collect implements the prometheus.Collector interface.

func (*CapacityPluginMetricsCollector) Describe

func (c *CapacityPluginMetricsCollector) Describe(ch chan<- *prometheus.Desc)

Describe implements the prometheus.Collector interface.

type CapacityPluginMetricsInstance

type CapacityPluginMetricsInstance struct {
	CapacitorID       string
	SerializedMetrics string
}

CapacityPluginMetricsInstance describes a single project service for which plugin metrics are submitted. It appears in type CapacityPluginMetricsCollector.

type Collector

type Collector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
	// Usually logg.Error, but can be changed inside unit tests.
	LogError func(msg string, args ...any)
	// Usually time.Now, but can be changed inside unit tests.
	// MeasureTimeAtEnd behaves slightly differently in unit tests: It will advance
	// the mock.Clock before reading it to simulate time passing during the previous task.
	MeasureTime      func() time.Time
	MeasureTimeAtEnd func() time.Time
	// Usually addJitter, but can be changed inside unit tests.
	AddJitter func(time.Duration) time.Duration
}

Collector provides methods that implement the collection jobs performed by limes-collect. The struct contains references to the driver used, the plugin (which defines the service type to be targeted), and a few other things; basically everything that needs to be replaced by a mock implementation for the collector's unit tests.

func NewCollector

func NewCollector(cluster *core.Cluster, dbm *gorp.DbMap) *Collector

NewCollector creates a Collector instance.

func (*Collector) CapacityScrapeJob

func (c *Collector) CapacityScrapeJob(registerer prometheus.Registerer) jobloop.Job

CapacityScrapeJob is a jobloop.Job. Each task scrapes one capacitor. Cluster resources managed by this capacitor are added, updated and deleted as necessary.

func (*Collector) CheckConsistencyJob

func (c *Collector) CheckConsistencyJob(registerer prometheus.Registerer) jobloop.Job

func (*Collector) CleanupOldCommitmentsJob

func (c *Collector) CleanupOldCommitmentsJob(registerer prometheus.Registerer) jobloop.Job

CleanupOldCommitmentsJob is a jobloop.CronJob.

It moves expired commitments to state "expired" and cleans up old expired commitments that do not have any non-expired predecessors.

func (*Collector) RateScrapeJob

func (c *Collector) RateScrapeJob(registerer prometheus.Registerer) jobloop.Job

RateScrapeJob looks at one specific project service per task, checks the database for outdated or missing rate records for the given service, and updates them by querying the backend service.

This job is not ConcurrencySafe, but multiple instances can safely be run in parallel if they act on separate service types. The job can only be run if a target service type is specified using the `jobloop.WithLabel("service_type", serviceType)` option.

func (*Collector) ResourceScrapeJob

func (c *Collector) ResourceScrapeJob(registerer prometheus.Registerer) jobloop.Job

ResourceScrapeJob looks at one specific project service per task, collects quota and usage information from the backend service, and adjusts the backend quota if it differs from the desired values.

This job is not ConcurrencySafe, but multiple instances can safely be run in parallel if they act on separate service types. The job can only be run if a target service type is specified using the `jobloop.WithLabel("service_type", serviceType)` option.

func (*Collector) ScanDomains

func (c *Collector) ScanDomains(opts ScanDomainsOpts) (result []string, resultErr error)

ScanDomains queries Keystone to discover new domains, and returns a list of UUIDs for the newly discovered domains.

func (*Collector) ScanDomainsAndProjectsJob

func (c *Collector) ScanDomainsAndProjectsJob(registerer prometheus.Registerer) jobloop.Job

ScanDomainsAndProjectsJob is a jobloop.CronJob. It syncs domains and projects from Keystone into the Limes database.

func (*Collector) ScanProjects

func (c *Collector) ScanProjects(domain *db.Domain) (result []string, resultErr error)

ScanProjects queries Keystone to discover new projects in the given domain.

func (*Collector) SyncQuotaToBackendJob

func (c *Collector) SyncQuotaToBackendJob(registerer prometheus.Registerer) jobloop.Job

SyncQuotaToBackendJob looks for project services that need to have their quota applied to the backend, and runs SetQuota for those services.

This job is not ConcurrencySafe, but multiple instances can safely be run in parallel if they act on separate service types. The job can only be run if a target service type is specified using the `jobloop.WithLabel("service_type", serviceType)` option.

type DataMetricsReporter

type DataMetricsReporter struct {
	Cluster      *core.Cluster
	DB           *gorp.DbMap
	ReportZeroes bool
}

DataMetricsReporter renders Prometheus metrics for data attributes (quota, usage, etc.) for all projects known to Limes.

It is an http.Handler, instead of implementing the prometheus.Collector interface (like all the other Collector types in this package) and going through the normal promhttp facility.

We are not going through promhttp here because promhttp insists on holding all metrics in memory before rendering them out (in order to sort them). Given the extremely high cardinality of these metrics, this results in unreasonably high memory usage spikes.

This implementation also holds all the metrics in memory (because ORDER BY on database level turned out to be prohibitively expensive), but we hold their rendered forms (i.e. something like `{bar="bar",foo="foo"} 42` instead of a dozen allocations for each label name, label value, label pair, a map of label pairs, and so on) in order to save memory.

func (*DataMetricsReporter) ServeHTTP

func (d *DataMetricsReporter) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements the http.Handler interface.

type QuotaPluginMetricsCollector

type QuotaPluginMetricsCollector struct {
	Cluster *core.Cluster
	DB      *gorp.DbMap
	// When .Override is set, the DB is bypassed and only the given
	// QuotaPluginMetricsInstances are considered. This is used for testing only.
	Override []QuotaPluginMetricsInstance
}

QuotaPluginMetricsCollector is a prometheus.Collector that submits metrics which are specific to the selected quota plugins.

func (*QuotaPluginMetricsCollector) Collect

func (c *QuotaPluginMetricsCollector) Collect(ch chan<- prometheus.Metric)

Collect implements the prometheus.Collector interface.

func (*QuotaPluginMetricsCollector) Describe

func (c *QuotaPluginMetricsCollector) Describe(ch chan<- *prometheus.Desc)

Describe implements the prometheus.Collector interface.

type QuotaPluginMetricsInstance

type QuotaPluginMetricsInstance struct {
	Project           core.KeystoneProject
	ServiceType       limes.ServiceType
	SerializedMetrics string
}

QuotaPluginMetricsInstance describes a single project service for which plugin metrics are submitted. It appears in type QuotaPluginMetricsCollector.

type ScanDomainsOpts

type ScanDomainsOpts struct {
	// Recurse into ScanProjects for all domains in the selected cluster,
	// rather than just for new domains.
	ScanAllProjects bool
}

ScanDomainsOpts contains additional options for ScanDomains().

type TaskTiming

type TaskTiming struct {
	StartedAt  time.Time // filled during DiscoverTask
	FinishedAt time.Time // filled during ProcessTask
}

TaskTiming appears in the task types of our ProducerConsumerJobs.

func (TaskTiming) Duration

func (t TaskTiming) Duration() time.Duration

Duration measures the duration of the main portion of a task.