providers

README

package providers

Providers implement the ConfigProvider interface and are responsible for scanning different sources like files on disk, environment variables, databases or containers and objects metadata, searching for integration configurations. Every configuration, regardless of the format, must specify at least one check instance. Providers dump every configuration they find into a CheckConfig struct containing an array of configuration instances. Configuration instances are converted in YAML format so that a check object will be eventually able to convert them into the appropriate data structure.

Usage example:

var configs []loader.CheckConfig
for _, provider := range configProviders {
  c, _ := provider.Collect(ctx)
  configs = append(configs, c...)
}

Config Providers

FileConfigProvider

The FileConfigProvider is a file-based config provider. By default it only scans files once at startup but can configured to poll regularly.

KubeletConfigProvider

The KubeletConfigProvider relies on the Kubelet API to detect check configs defined on pod annotations.

ContainerConfigProvider

The ContainerConfigProvider detects check configs defined on container labels.

ECSConfigProvider

The ECSConfigProvider relies on the ECS API to detect check configs defined on container labels. This config provider is enabled on ECS Fargate only, on ECS EC2 we use the docker config provider.

KubeServiceConfigProvider

The KubeServiceConfigProvider relies on the Kubernetes API server to detect the cluster check configs defined on service annotations. The Datadog Cluster Agent runs this ConfigProvider.

ClusterChecksConfigProvider

The ClusterChecksConfigProvider queries the Datadog Cluster Agent API to consume the exposed cluster check configs. The node Agent or the cluster check runner can run this config provider.

KubeEndpointsConfigProvider

The KubeEndpointsConfigProvider relies on the Kubernetes API server to detect the endpoints check configs defined on service annotations. The Datadog Cluster Agent runs this ConfigProvider.

EndpointChecksConfigProvider

The EndpointChecksConfigProvider queries the Datadog Cluster Agent API to consume the exposed endpoints check configs.

PrometheusPodsConfigProvider

The PrometheusPodsConfigProvider relies on the Kubelet API to detect Prometheus pod annotations and generate a corresponding Openmetrics config.

PrometheusServicesConfigProvider

The PrometheusServicesConfigProvider relies on the Kubernetes API server to watch Prometheus service annotations and generate a corresponding Openmetrics config. The Datadog Cluster Agent runs this ConfigProvider.

CloudFoundryConfigProvider

The CloudFoundryConfigProvider relies on the CloudFoundry BBS API to detect check configs defined in LRP environment variables.

ConsulConfigProvider

The ConsulConfigProvider reads the check configs from consul.

ETCDConfigProvider

The ETCDConfigProvider reads the check configs from etcd.

ZookeeperConfigProvider

The ZookeeperConfigProvider reads the check configs from zookeeper.

RemoteConfigProvider

The RemoteConfigProvider reads the check configs from remote-config.

Documentation

Index

• Variables
• func GetIntegrationConfigFromFile(name, fpath string) (integration.Config, error)
• func GetPollInterval(cp pkgconfigsetup.ConfigurationProviders) time.Duration
• func InitConfigFilesReader(paths []string)
• func ReadConfigFiles(keep FilterFunc) ([]integration.Config, map[string]string, error)
• func RegisterProvider(name string, ...)
• func RegisterProviderWithComponents(name string, factory ConfigProviderFactory, ...)
• func RegisterProviders(providerCatalog map[string]ConfigProviderFactory)
• func ResetReader(paths []string)
• type ClusterChecksConfigProvider
  • func (c *ClusterChecksConfigProvider) Collect(ctx context.Context) ([]integration.Config, error)
  • func (c *ClusterChecksConfigProvider) GetConfigErrors() map[string]ErrorMsgSet
  • func (c *ClusterChecksConfigProvider) IsUpToDate(ctx context.Context) (bool, error)
  • func (c *ClusterChecksConfigProvider) String() string
• type CollectingConfigProvider
• type ConfigProvider
  • func NewClusterChecksConfigProvider(providerConfig *pkgconfigsetup.ConfigurationProviders, _ *telemetry.Store) (ConfigProvider, error)
  • func NewContainerConfigProvider(_ *pkgconfigsetup.ConfigurationProviders, wmeta workloadmeta.Component, ...) (ConfigProvider, error)
• type ConfigProviderFactory
• type ContainerConfigProvider
  • func (k *ContainerConfigProvider) GetConfigErrors() map[string]ErrorMsgSet
  • func (k *ContainerConfigProvider) Stream(ctx context.Context) <-chan integration.ConfigChanges
  • func (k *ContainerConfigProvider) String() string
• type ErrorMsgSet
• type FileConfigProvider
  • func NewFileConfigProvider(telemetryStore *telemetry.Store) *FileConfigProvider
  • func (c *FileConfigProvider) Collect(_ context.Context) ([]integration.Config, error)
  • func (c *FileConfigProvider) GetConfigErrors() map[string]ErrorMsgSet
  • func (c *FileConfigProvider) IsUpToDate(_ context.Context) (bool, error)
  • func (c *FileConfigProvider) String() string
• type FilterFunc
• type RemoteConfigProvider
  • func NewRemoteConfigProvider() *RemoteConfigProvider
  • func (rc *RemoteConfigProvider) Collect(_ context.Context) ([]integration.Config, error)
  • func (rc *RemoteConfigProvider) GetConfigErrors() map[string]ErrorMsgSet
  • func (rc *RemoteConfigProvider) IntegrationScheduleCallback(updates map[string]state.RawConfig, ...)
  • func (rc *RemoteConfigProvider) IsUpToDate(_ context.Context) (bool, error)
  • func (rc *RemoteConfigProvider) String() string
• type StreamingConfigProvider

Variables

var NewCloudFoundryConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewCloudFoundryConfigProvider instantiates a new CloudFoundryConfigProvider from given config

var NewConsulConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewConsulConfigProvider creates a client connection to consul and create a new ConsulConfigProvider

var NewEndpointsChecksConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewEndpointsChecksConfigProvider returns a new ConfigProvider collecting endpoints check configurations from the cluster-agent. Connectivity is not checked at this stage to allow for retries, Collect will do it.

var NewEtcdConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewEtcdConfigProvider creates a client connection to etcd and create a new EtcdConfigProvider

var NewKubeEndpointsConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewKubeEndpointsConfigProvider returns a new ConfigProvider connected to apiserver. Connectivity is not checked at this stage to allow for retries, Collect will do it.

var NewKubeEndpointsFileConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewKubeEndpointsFileConfigProvider returns a new KubeEndpointsFileConfigProvider

var NewKubeServiceConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewKubeServiceConfigProvider returns a new ConfigProvider connected to apiserver. Connectivity is not checked at this stage to allow for retries, Collect will do it.

var NewKubeServiceFileConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewKubeServiceFileConfigProvider returns a new KubeServiceFileConfigProvider

var NewPrometheusPodsConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewPrometheusPodsConfigProvider returns a new Prometheus ConfigProvider connected to kubelet. Connectivity is not checked at this stage to allow for retries, Collect will do it.

var NewPrometheusServicesConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewPrometheusServicesConfigProvider returns a new Prometheus ConfigProvider connected to kube apiserver

var NewZookeeperConfigProvider func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewZookeeperConfigProvider returns a new Client connected to a Zookeeper backend.

Functions

func GetIntegrationConfigFromFile

func GetIntegrationConfigFromFile(name, fpath string) (integration.Config, error)

GetIntegrationConfigFromFile returns an instance of integration.Config if `fpath` points to a valid config file

func GetPollInterval

func GetPollInterval(cp pkgconfigsetup.ConfigurationProviders) time.Duration

GetPollInterval computes the poll interval from the config

func InitConfigFilesReader

func InitConfigFilesReader(paths []string)

InitConfigFilesReader initializes the config files reader. It reads all configs and caches them in memory for 5 minutes. InitConfigFilesReader should be called at agent startup.

func ReadConfigFiles

func ReadConfigFiles(keep FilterFunc) ([]integration.Config, map[string]string, error)

ReadConfigFiles returns integration configs read from config files, a mapping integration config error strings and an error. The filter argument allows returing a subset of configs depending on the caller preferences. InitConfigFilesReader should be called at agent startup before this function to setup the config paths and cache the configs.

func RegisterProvider

func RegisterProvider(name string,
	factory func(providerConfig *pkgconfigsetup.ConfigurationProviders, telemetryStore *telemetry.Store) (ConfigProvider, error),
	providerCatalog map[string]ConfigProviderFactory)

RegisterProvider adds a loader to the providers catalog

func RegisterProviderWithComponents

func RegisterProviderWithComponents(name string, factory ConfigProviderFactory, providerCatalog map[string]ConfigProviderFactory)

RegisterProviderWithComponents adds a loader to the providers catalog

func RegisterProviders

func RegisterProviders(providerCatalog map[string]ConfigProviderFactory)

RegisterProviders adds all the default providers to the catalog

func ResetReader

func ResetReader(paths []string)

ResetReader is only for unit tests

Types

type ClusterChecksConfigProvider

type ClusterChecksConfigProvider struct {
	// contains filtered or unexported fields
}

ClusterChecksConfigProvider implements the ConfigProvider interface for the cluster check feature.

func (*ClusterChecksConfigProvider) Collect

func (c *ClusterChecksConfigProvider) Collect(ctx context.Context) ([]integration.Config, error)

Collect retrieves configurations the cluster-agent dispatched to this agent

func (*ClusterChecksConfigProvider) GetConfigErrors

func (c *ClusterChecksConfigProvider) GetConfigErrors() map[string]ErrorMsgSet

GetConfigErrors is not implemented for the ClusterChecksConfigProvider

func (*ClusterChecksConfigProvider) IsUpToDate

func (c *ClusterChecksConfigProvider) IsUpToDate(ctx context.Context) (bool, error)

IsUpToDate queries the cluster-agent to update its status and query if new configurations are available

func (*ClusterChecksConfigProvider) String

func (c *ClusterChecksConfigProvider) String() string

String returns a string representation of the ClusterChecksConfigProvider

type CollectingConfigProvider

type CollectingConfigProvider interface {
	// Collect is responsible of populating a list of Config instances by
	// retrieving configuration patterns from external resources.
	Collect(context.Context) ([]integration.Config, error)

	// IsUpToDate determines whether the information returned from the last
	// call to Collect is still correct.  If not, Collect will be called again.
	IsUpToDate(context.Context) (bool, error)
}

CollectingConfigProvider is an interface used together with ConfigProvider. ConfigProviders that are NOT able to use streaming, and therefore need external reconciliation, should implement it.

type ConfigProvider

type ConfigProvider interface {
	// String returns the name of the provider.  All Config instances produced
	// by this provider will have this value in their Provider field.
	String() string

	// GetConfigErrors returns a map of errors that occurred on the last Collect
	// call, indexed by a description of the resource that generated the error.
	// The result is displayed in diagnostic tools such as `agent status`.
	GetConfigErrors() map[string]ErrorMsgSet
}

ConfigProvider represents a source of `integration.Config` values that can either be applied immediately or resolved for a service and applied.

These Config values may come from files on disk, databases, environment variables, container labels, etc.

Any type implementing the interface will take care of any dependency or data needed to access the resource providing the configuration.

func NewClusterChecksConfigProvider

func NewClusterChecksConfigProvider(providerConfig *pkgconfigsetup.ConfigurationProviders, _ *telemetry.Store) (ConfigProvider, error)

NewClusterChecksConfigProvider returns a new ConfigProvider collecting cluster check configurations from the cluster-agent. Connectivity is not checked at this stage to allow for retries, Collect will do it.

func NewContainerConfigProvider

func NewContainerConfigProvider(_ *pkgconfigsetup.ConfigurationProviders, wmeta workloadmeta.Component, telemetryStore *telemetry.Store) (ConfigProvider, error)

NewContainerConfigProvider returns a new ConfigProvider subscribed to both container and pods

type ConfigProviderFactory

type ConfigProviderFactory func(providerConfig *pkgconfigsetup.ConfigurationProviders, wmeta workloadmeta.Component, telemetryStore *telemetry.Store) (ConfigProvider, error)

ConfigProviderFactory is any function capable to create a ConfigProvider instance

type ContainerConfigProvider

type ContainerConfigProvider struct {
	// contains filtered or unexported fields
}

ContainerConfigProvider implements the ConfigProvider interface for both pods and containers

func (*ContainerConfigProvider) GetConfigErrors

func (k *ContainerConfigProvider) GetConfigErrors() map[string]ErrorMsgSet

GetConfigErrors returns a map of configuration errors for each namespace/pod

func (*ContainerConfigProvider) Stream

func (k *ContainerConfigProvider) Stream(ctx context.Context) <-chan integration.ConfigChanges

Stream starts listening to workloadmeta to generate configs as they come instead of relying on a periodic call to Collect.

func (*ContainerConfigProvider) String

func (k *ContainerConfigProvider) String() string

String returns a string representation of the ContainerConfigProvider

type ErrorMsgSet

type ErrorMsgSet map[string]struct{}

ErrorMsgSet contains a list of unique configuration errors for a provider

type FileConfigProvider

type FileConfigProvider struct {
	Errors map[string]string
	// contains filtered or unexported fields
}

FileConfigProvider collect configuration files from disk

func NewFileConfigProvider

func NewFileConfigProvider(telemetryStore *telemetry.Store) *FileConfigProvider

NewFileConfigProvider creates a new FileConfigProvider.

func (*FileConfigProvider) Collect

func (c *FileConfigProvider) Collect(_ context.Context) ([]integration.Config, error)

Collect returns the check configurations defined in Yaml files. Configs with advanced AD identifiers are filtered-out. They're handled by other file-based config providers.

func (*FileConfigProvider) GetConfigErrors

func (c *FileConfigProvider) GetConfigErrors() map[string]ErrorMsgSet

GetConfigErrors is not implemented for the FileConfigProvider

func (*FileConfigProvider) IsUpToDate

func (c *FileConfigProvider) IsUpToDate(_ context.Context) (bool, error)

IsUpToDate is not implemented for the file Providers as the files are not meant to change very often.

func (*FileConfigProvider) String

func (c *FileConfigProvider) String() string

String returns a string representation of the FileConfigProvider

type FilterFunc

type FilterFunc func(integration.Config) bool

FilterFunc is used by ReadConfigFiles to filter integration configs.

var GetAll FilterFunc = func(_ integration.Config) bool { return true }

GetAll makes ReadConfigFiles return all the configurations found.

var WithAdvancedADOnly FilterFunc = func(c integration.Config) bool { return len(c.AdvancedADIdentifiers) > 0 }

WithAdvancedADOnly makes ReadConfigFiles return the configurations with AdvancedADIdentifiers only.

var WithoutAdvancedAD FilterFunc = func(c integration.Config) bool { return len(c.AdvancedADIdentifiers) == 0 }

WithoutAdvancedAD makes ReadConfigFiles return the all configurations except the ones with AdvancedADIdentifiers.

type RemoteConfigProvider

type RemoteConfigProvider struct {
	// contains filtered or unexported fields
}

RemoteConfigProvider receives configuration from remote-config

func NewRemoteConfigProvider

func NewRemoteConfigProvider() *RemoteConfigProvider

NewRemoteConfigProvider creates a new RemoteConfigProvider.

func (*RemoteConfigProvider) Collect

func (rc *RemoteConfigProvider) Collect(_ context.Context) ([]integration.Config, error)

Collect retrieves integrations from the remote-config, builds Config objects and returns them

func (*RemoteConfigProvider) GetConfigErrors

func (rc *RemoteConfigProvider) GetConfigErrors() map[string]ErrorMsgSet

GetConfigErrors returns a map of configuration errors for each configuration path

func (*RemoteConfigProvider) IntegrationScheduleCallback

func (rc *RemoteConfigProvider) IntegrationScheduleCallback(updates map[string]state.RawConfig, applyStateCallback func(string, state.ApplyStatus))

IntegrationScheduleCallback is called at every AGENT_INTEGRATIONS to schedule/unschedule integrations

func (*RemoteConfigProvider) IsUpToDate

func (rc *RemoteConfigProvider) IsUpToDate(_ context.Context) (bool, error)

IsUpToDate allows to cache configs as long as no changes are detected in remote-config

func (*RemoteConfigProvider) String

func (rc *RemoteConfigProvider) String() string

String returns a string representation of the RemoteConfigProvider

type StreamingConfigProvider

type StreamingConfigProvider interface {
	// Stream starts the streaming config provider until the provided
	// context is cancelled. Config changes are sent on the return channel.
	Stream(context.Context) <-chan integration.ConfigChanges
}

StreamingConfigProvider is an interface used together with ConfigProvider. ConfigProviders that are able to use streaming should implement it, and the config poller will use Stream instead of Collect to collect config changes.