configsource

Documentation

Index

• func BuildConfigSources(ctx context.Context, configSourcesSettings map[string]Settings, ...) (map[string]ConfigSource, error)
• func BuildConfigSourcesFromConf(ctx context.Context, confToFurtherResolve *confmap.Conf, logger *zap.Logger, ...) (map[string]ConfigSource, *confmap.Conf, error)
• func MergeCloseFuncs(closeFuncs []confmap.CloseFunc) confmap.CloseFunc
• func ResolveWithConfigSources(ctx context.Context, configSources map[string]ConfigSource, ...) (*confmap.Conf, confmap.CloseFunc, error)
• func SettingsFromConf(ctx context.Context, conf *confmap.Conf, factories Factories, ...) (map[string]Settings, *confmap.Conf, error)
• type ConfigSource
• type Factories
• type Factory
• type Settings
• type SourceSettings
  • func NewSourceSettings(id component.ID) SourceSettings
  • func (s *SourceSettings) ID() component.ID
  • func (s *SourceSettings) SetIDName(idName string)

Functions

func BuildConfigSources

func BuildConfigSources(ctx context.Context, configSourcesSettings map[string]Settings, logger *zap.Logger, factories Factories) (map[string]ConfigSource, error)

func BuildConfigSourcesFromConf

func BuildConfigSourcesFromConf(ctx context.Context, confToFurtherResolve *confmap.Conf, logger *zap.Logger, factories Factories, confmapProviders map[string]confmap.Provider) (map[string]ConfigSource, *confmap.Conf, error)

BuildConfigSourcesFromConf inspects the given confmap.Conf and builds all config sources referenced in the configuration intended to be used with ResolveWithConfigSources().

func MergeCloseFuncs

func MergeCloseFuncs(closeFuncs []confmap.CloseFunc) confmap.CloseFunc

func ResolveWithConfigSources

func ResolveWithConfigSources(ctx context.Context, configSources map[string]ConfigSource, confmapProviders map[string]confmap.Provider, conf *confmap.Conf, watcher confmap.WatcherFunc) (*confmap.Conf, confmap.CloseFunc, error)

ResolveWithConfigSources returns a confmap.Conf in which all env vars and config sources on the given input config map are resolved to actual literal values of the env vars or config sources.

1. ResolveWithConfigSources to inject the data from config sources into a configuration; 2. Wait for an update on "watcher" func. 3. Close the confmap.Retrieved instance;

The current syntax to reference a config source in a YAML is provisional. Currently single-line:

param_to_be_retrieved: $<cfgSrcName>:<selector>[?<params_url_query_format>]

bracketed single-line:

param_to_be_retrieved: ${<cfgSrcName>:<selector>[?<params_url_query_format>]}

and multi-line are supported:

param_to_be_retrieved: |
  $<cfgSrcName>: <selector>
  [<params_multi_line_YAML>]

The <cfgSrcName> is a name string used to identify the config source instance to be used to retrieve the value.

The <selector> is the mandatory parameter required when retrieving data from a config source.

Not all config sources need the optional parameters, they are used to provide extra control when retrieving and preparing the data to be injected into the configuration.

For single-line format <params_url_query_format> uses the same syntax as URL query parameters. Hypothetical example in a YAML file:

component:

config_field: $file:/etc/secret.bin?binary=true

For multi-line format <params_multi_line_YAML> uses syntax as a YAML inside YAML. Possible usage example in a YAML file:

component:

config_field: |
  $yamltemplate: /etc/log_template.yaml
  logs_path: /var/logs/
  timeout: 10s

Not all config sources need these optional parameters, they are used to provide extra control when retrieving and data to be injected into the configuration.

Assuming a config source named "env" that retrieve environment variables and one named "file" that retrieves contents from individual files, here are some examples:

component:
  # Retrieves the value of the environment variable LOGS_DIR.
  logs_dir: $env:LOGS_DIR

  # Retrieves the value from the file /etc/secret.bin and injects its contents as a []byte.
  bytes_from_file: $file:/etc/secret.bin?binary=true

  # Retrieves the value from the file /etc/text.txt and injects its contents as a string.
  # Hypothetically the "file" config source by default tries to inject the file contents
  # as a string if params doesn't specify that "binary" is true.
  text_from_file: $file:/etc/text.txt

Bracketed single-line should be used when concatenating a suffix to the value retrieved by the config source. Example:

component:
  # Retrieves the value of the environment variable LOGS_DIR and appends /component.log to it.
  log_file_fullname: ${env:LOGS_DIR}/component.log

Environment variables are expanded before passed to the config source when used in the selector or the optional parameters. Example:

component:
  # Retrieves the value from the file text.txt located on the path specified by the environment
  # variable DATA_PATH. The name of the environment variable is the string after the delimiter
  # until the first character different than '_' and non-alpha-numeric.
  text_from_file: $file:$DATA_PATH/text.txt

Since environment variables and config sources both use the '$', with or without brackets, as a prefix for their expansion it is necessary to have a way to distinguish between them. For the non-bracketed syntax the code will peek at the first character other than alpha-numeric and '_' after the '$'. If that character is a ':' it will treat it as a config source and as environment variable otherwise. For example:

component:
  field_0: $PATH:/etc/logs # Injects the data from a config sourced named "PATH" using the selector "/etc/logs".
  field_1: $PATH/etc/logs  # Expands the environment variable "PATH" and adds the suffix "/etc/logs" to it.

So if you need to include an environment followed by ':' the bracketed syntax must be used instead:

component:
  field_0: ${PATH}:/etc/logs # Expands the environment variable "PATH" and adds the suffix ":/etc/logs" to it.

For the bracketed syntax the presence of ':' inside the brackets indicates that code will treat the bracketed contents as a config source. For example:

component:
  field_0: ${file:/var/secret.txt} # Injects the data from a config sourced named "file" using the selector "/var/secret.txt".
  field_1: ${file}:/var/secret.txt # Expands the environment variable "file" and adds the suffix ":/var/secret.txt" to it.

If the character following the '$' is in the set {'*', '#', '$', '@', '!', '?', '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9'} the code will consider it to be the name of an environment variable to expand, or config source if followed by ':'. Do not use any of these characters as the first char on the name of a config source or an environment variable (even if allowed by the system) to avoid unexpected results.

For an overview about the internals of the Manager refer to the package README.md.

func SettingsFromConf

func SettingsFromConf(ctx context.Context, conf *confmap.Conf, factories Factories, confmapProviders map[string]confmap.Provider) (map[string]Settings, *confmap.Conf, error)

SettingsFromConf reads the configuration for ConfigSource objects from the given confmap.Conf and returns a map of Settings and the remaining Conf without the `config_sources` mapping

Types

type ConfigSource

type ConfigSource interface {
	// Retrieve goes to the configuration source and retrieves the selected data which
	// contains the value to be injected in the configuration and the corresponding watcher that
	// will be used to monitor for updates of the retrieved value. The retrieved value is selected
	// according to the selector and the params arguments.
	//
	// The selector is a string that is required on all invocations, the params are optional. Each
	// implementation handles the generic params according to their requirements.
	Retrieve(ctx context.Context, selector string, params *confmap.Conf, watcher confmap.WatcherFunc) (*confmap.Retrieved, error)
}

type Factories

type Factories map[component.Type]Factory

Factories maps the type of a ConfigSource to the respective factory object.

type Factory

type Factory interface {
	// CreateDefaultConfig creates the default configuration settings for the ConfigSource.
	// This method can be called multiple times depending on the pipeline
	// configuration and should not cause side-effects that prevent the creation
	// of multiple instances of the ConfigSource.
	// The object returned by this method needs to pass the checks implemented by
	// 'configcheck.ValidateConfig'. It is recommended to have such check in the
	// tests of any implementation of the Factory interface.
	CreateDefaultConfig() Settings

	// CreateConfigSource creates a configuration source based on the given config.
	CreateConfigSource(context.Context, Settings, *zap.Logger) (ConfigSource, error)

	// Type gets the type of the component created by this factory.
	Type() component.Type
}

Factory is a factory interface for configuration sources. Given it's not an accepted component and because of the direct Factory usage restriction from https://github.com/open-telemetry/opentelemetry-collector/commit/9631ceabb7dc4ca5cc187bab26d8319783bcc562 it's not a proper Collector config.Factory.

type Settings

type Settings interface {
	// ID returns the ID of the component that this configuration belongs to.
	ID() component.ID
	// SetIDName updates the name part of the ID for the component that this configuration belongs to.
	SetIDName(idName string)
}

Settings is the configuration of a config source. Specific config sources must implement this interface and will typically embed SourceSettings struct or a struct that extends it.

type SourceSettings

type SourceSettings struct {
	// contains filtered or unexported fields
}

SourceSettings defines common settings of a Settings configuration. Specific config sources can embed this struct and extend it with more fields if needed. When embedded it must be with `mapstructure:",squash"` tag.

func NewSourceSettings

func NewSourceSettings(id component.ID) SourceSettings

NewSourceSettings return a new config.SourceSettings struct with the given ComponentID.

func (*SourceSettings) ID

func (s *SourceSettings) ID() component.ID

ID returns the ID of the component that this configuration belongs to.

func (*SourceSettings) SetIDName

func (s *SourceSettings) SetIDName(idName string)

SetIDName updates the name part of the ID for the component that this configuration belongs to.