config

Documentation

Overview

Package config defines the data models for entities. This file defines the models for configuration format. The defined entities are: Config (the top-level structure), Receivers, Exporters, Processors, Pipelines.

Receivers, Exporters and Processors typically have common configuration settings, however sometimes specific implementations will have extra configuration settings. This requires the configuration data for these entities to be polymorphic.

To satisfy these requirements we declare interfaces Receiver, Exporter, Processor, which define the behavior. We also provide helper structs ReceiverSettings, ExporterSettings, ProcessorSettings, which define the common settings and unmarshaling from config files.

Specific Receivers/Exporters/Processors are expected to at the minimum implement the corresponding interface and if they have additional settings they must also extend the corresponding common settings struct (the easiest approach is to embed the common struct).

Index

• Constants
• func UnmarshalExporter(cfgMap *Map, cfg Exporter) error
• func UnmarshalExtension(cfgMap *Map, cfg Extension) error
• func UnmarshalProcessor(cfgMap *Map, cfg Processor) error
• func UnmarshalReceiver(cfgMap *Map, cfg Receiver) error
• type ChangeEvent
• type CloseFunc
• type ComponentID
  • func NewComponentID(typeVal Type) ComponentID
  • func NewComponentIDFromString(idStr string) (ComponentID, error)
  • func NewComponentIDWithName(typeVal Type, nameVal string) ComponentID
  • func (id ComponentID) Name() string
  • func (id ComponentID) String() string
  • func (id ComponentID) Type() Type
  • func (id *ComponentID) UnmarshalText(text []byte) error
• type Config
  • func (cfg *Config) Validate() error
• type DataType
• type Exporter
• type ExporterSettings
  • func NewExporterSettings(id ComponentID) ExporterSettings
  • func (es *ExporterSettings) ID() ComponentID
  • func (es *ExporterSettings) SetIDName(idName string)
  • func (es *ExporterSettings) Validate() error
• type Extension
• type ExtensionSettings
  • func NewExtensionSettings(id ComponentID) ExtensionSettings
  • func (es *ExtensionSettings) ID() ComponentID
  • func (es *ExtensionSettings) SetIDName(idName string)
  • func (es *ExtensionSettings) Validate() error
• type Map
  • func NewMap() *Map
  • func NewMapFromStringMap(data map[string]interface{}) *Map
  • func (l *Map) AllKeys() []string
  • func (l *Map) Get(key string) interface{}
  • func (l *Map) IsSet(key string) bool
  • func (l *Map) Merge(in *Map) error
  • func (l *Map) Set(key string, value interface{})
  • func (l *Map) Sub(key string) (*Map, error)
  • func (l *Map) ToStringMap() map[string]interface{}
  • func (l *Map) Unmarshal(rawVal interface{}) error
  • func (l *Map) UnmarshalExact(rawVal interface{}) error
• type MapConverterFunc
• type MapProvider
• type Pipeline
• type Pipelines
• type Processor
• type ProcessorSettings
  • func NewProcessorSettings(id ComponentID) ProcessorSettings
  • func (ps *ProcessorSettings) ID() ComponentID
  • func (ps *ProcessorSettings) SetIDName(idName string)
  • func (ps *ProcessorSettings) Validate() error
• type Receiver
• type ReceiverSettings
  • func NewReceiverSettings(id ComponentID) ReceiverSettings
  • func (rs *ReceiverSettings) ID() ComponentID
  • func (rs *ReceiverSettings) SetIDName(idName string)
  • func (rs *ReceiverSettings) Validate() error
• type Retrieved
  • func NewRetrievedFromMap(cfgMap *Map, opts ...RetrievedOption) Retrieved
  • func (r Retrieved) AsMap() (*Map, error)
  • func (r Retrieved) Close(ctx context.Context) error
• type RetrievedOption
  • func WithRetrievedClose(closeFunc CloseFunc) RetrievedOption
• type Service
• type ServiceTelemetry
• type ServiceTelemetryLogs
• type ServiceTelemetryMetrics
• type Type
• type Unmarshallable
• type WatcherFunc

Constants

const (
	// KeyDelimiter is used as the default key delimiter in the default koanf instance.
	KeyDelimiter = "::"
)

Functions

func UnmarshalExporter

func UnmarshalExporter(cfgMap *Map, cfg Exporter) error

UnmarshalExporter helper function to unmarshal an Exporter config. It checks if the config implements Unmarshallable and uses that if available, otherwise uses Map.UnmarshalExact, erroring if a field is nonexistent.

func UnmarshalExtension

func UnmarshalExtension(cfgMap *Map, cfg Extension) error

UnmarshalExtension helper function to unmarshal an Extension config. It checks if the config implements Unmarshallable and uses that if available, otherwise uses Map.UnmarshalExact, erroring if a field is nonexistent.

func UnmarshalProcessor

func UnmarshalProcessor(cfgMap *Map, cfg Processor) error

UnmarshalProcessor helper function to unmarshal a Processor config. It checks if the config implements Unmarshallable and uses that if available, otherwise uses Map.UnmarshalExact, erroring if a field is nonexistent.

func UnmarshalReceiver

func UnmarshalReceiver(cfgMap *Map, cfg Receiver) error

UnmarshalReceiver helper function to unmarshal a Receiver config. It checks if the config implements Unmarshallable and uses that if available, otherwise uses Map.UnmarshalExact, erroring if a field is nonexistent.

Types

type ChangeEvent

type ChangeEvent struct {
	// Error is nil if the config is changed and needs to be re-fetched.
	// Any non-nil error indicates that there was a problem with watching the config changes.
	Error error
}

ChangeEvent describes the particular change event that happened with the config. TODO: see if this can be eliminated.

type CloseFunc

type CloseFunc func(context.Context) error

CloseFunc a function equivalent to Retrieved.Close.

type ComponentID

type ComponentID struct {
	// contains filtered or unexported fields
}

ComponentID represents the identity for a component. It combines two values: * type - the Type of the component. * name - the name of that component. The component ComponentID (combination type + name) is unique for a given component.Kind.

func NewComponentID

func NewComponentID(typeVal Type) ComponentID

NewComponentID returns a new ComponentID with the given Type and empty name.

func NewComponentIDFromString

func NewComponentIDFromString(idStr string) (ComponentID, error)

NewComponentIDFromString decodes a string in type[/name] format into ComponentID. The type and name components will have spaces trimmed, the "type" part must be present, the forward slash and "name" are optional. The returned ComponentID will be invalid if err is not-nil.

func NewComponentIDWithName

func NewComponentIDWithName(typeVal Type, nameVal string) ComponentID

NewComponentIDWithName returns a new ComponentID with the given Type and name.

func (ComponentID) Name

func (id ComponentID) Name() string

Name returns the custom name of the component.

func (ComponentID) String

func (id ComponentID) String() string

String returns the ComponentID string representation as "type[/name]" format.

func (ComponentID) Type

func (id ComponentID) Type() Type

Type returns the type of the component.

func (*ComponentID) UnmarshalText

func (id *ComponentID) UnmarshalText(text []byte) error

UnmarshalText implements the encoding.TextUnmarshaler interface.

type Config

type Config struct {
	// Receivers is a map of ComponentID to Receivers.
	Receivers map[ComponentID]Receiver

	// Exporters is a map of ComponentID to Exporters.
	Exporters map[ComponentID]Exporter

	// Processors is a map of ComponentID to Processors.
	Processors map[ComponentID]Processor

	// Extensions is a map of ComponentID to extensions.
	Extensions map[ComponentID]Extension

	Service
}

Config defines the configuration for the various elements of collector or agent.

func (*Config) Validate

func (cfg *Config) Validate() error

Validate returns an error if the config is invalid.

This function performs basic validation of configuration. There may be more subtle invalid cases that we currently don't check for but which we may want to add in the future (e.g. disallowing receiving and exporting on the same endpoint).

type DataType

type DataType = Type

DataType is a special Type that represents the data types supported by the collector. We currently support collecting metrics, traces and logs, this can expand in the future.

const (
	// TracesDataType is the data type tag for traces.
	TracesDataType DataType = "traces"

	// MetricsDataType is the data type tag for metrics.
	MetricsDataType DataType = "metrics"

	// LogsDataType is the data type tag for logs.
	LogsDataType DataType = "logs"
)

Currently supported data types. Add new data types here when new types are supported in the future.

type Exporter

type Exporter interface {
	// contains filtered or unexported methods
}

Exporter is the configuration of a component.Exporter. Specific extensions must implement this interface and must embed ExporterSettings struct or a struct that extends it.

type ExporterSettings

type ExporterSettings struct {
	// contains filtered or unexported fields
}

ExporterSettings defines common settings for a component.Exporter configuration. Specific exporters can embed this struct and extend it with more fields if needed.

It is highly recommended to "override" the Validate() function.

When embedded in the exporter config, it must be with `mapstructure:",squash"` tag.

func NewExporterSettings

func NewExporterSettings(id ComponentID) ExporterSettings

NewExporterSettings return a new ExporterSettings with the given ComponentID.

func (*ExporterSettings) ID

func (es *ExporterSettings) ID() ComponentID

ID returns the receiver ComponentID.

func (*ExporterSettings) SetIDName

func (es *ExporterSettings) SetIDName(idName string)

SetIDName sets the receiver name.

func (*ExporterSettings) Validate

func (es *ExporterSettings) Validate() error

Validate validates the configuration and returns an error if invalid.

type Extension

type Extension interface {
	// contains filtered or unexported methods
}

Extension is the configuration of a component.Extension. Specific extensions must implement this interface and must embed ExtensionSettings struct or a struct that extends it.

type ExtensionSettings

type ExtensionSettings struct {
	// contains filtered or unexported fields
}

ExtensionSettings defines common settings for a component.Extension configuration. Specific processors can embed this struct and extend it with more fields if needed.

It is highly recommended to "override" the Validate() function.

When embedded in the extension config, it must be with `mapstructure:",squash"` tag.

func NewExtensionSettings

func NewExtensionSettings(id ComponentID) ExtensionSettings

NewExtensionSettings return a new ExtensionSettings with the given ComponentID.

func (*ExtensionSettings) ID

func (es *ExtensionSettings) ID() ComponentID

ID returns the receiver ComponentID.

func (*ExtensionSettings) SetIDName

func (es *ExtensionSettings) SetIDName(idName string)

SetIDName sets the receiver name.

func (*ExtensionSettings) Validate

func (es *ExtensionSettings) Validate() error

Validate validates the configuration and returns an error if invalid.

type Map

type Map struct {
	// contains filtered or unexported fields
}

Map represents the raw configuration map for the OpenTelemetry Collector. The config.Map can be unmarshalled into the Collector's config using the "configunmarshaler" package.

func NewMap

func NewMap() *Map

NewMap creates a new empty config.Map instance.

func NewMapFromStringMap

func NewMapFromStringMap(data map[string]interface{}) *Map

NewMapFromStringMap creates a config.Map from a map[string]interface{}.

func (*Map) AllKeys

func (l *Map) AllKeys() []string

AllKeys returns all keys holding a value, regardless of where they are set. Nested keys are returned with a KeyDelimiter separator.

func (*Map) Get

func (l *Map) Get(key string) interface{}

Get can retrieve any value given the key to use.

func (*Map) IsSet

func (l *Map) IsSet(key string) bool

IsSet checks to see if the key has been set in any of the data locations. IsSet is case-insensitive for a key.

func (*Map) Merge

func (l *Map) Merge(in *Map) error

Merge merges the input given configuration into the existing config. Note that the given map may be modified.

func (*Map) Set

func (l *Map) Set(key string, value interface{})

Set sets the value for the key.

func (*Map) Sub

func (l *Map) Sub(key string) (*Map, error)

Sub returns new Map instance representing a sub-config of this instance. It returns an error is the sub-config is not a map[string]interface{} (use Get()), and an empty Map if none exists.

func (*Map) ToStringMap

func (l *Map) ToStringMap() map[string]interface{}

ToStringMap creates a map[string]interface{} from a Parser.

func (*Map) Unmarshal

func (l *Map) Unmarshal(rawVal interface{}) error

Unmarshal unmarshalls the config into a struct. Tags on the fields of the structure must be properly set.

func (*Map) UnmarshalExact

func (l *Map) UnmarshalExact(rawVal interface{}) error

UnmarshalExact unmarshalls the config into a struct, erroring if a field is nonexistent.

type MapConverterFunc

type MapConverterFunc func(context.Context, *Map) error

MapConverterFunc is a converter function for the config.Map that allows distributions (in the future components as well) to build backwards compatible config converters.

type MapProvider

type MapProvider interface {
	// Retrieve goes to the configuration source and retrieves the selected data which
	// contains the value to be injected in the configuration and the corresponding watcher that
	// will be used to monitor for updates of the retrieved value.
	//
	// `uri` must follow the "<scheme>:<opaque_data>" format. This format is compatible
	// with the URI definition (see https://datatracker.ietf.org/doc/html/rfc3986). The "<scheme>"
	// must be always included in the `uri`. The scheme supported by any provider MUST be at
	// least 2 characters long to avoid conflicting with a driver-letter identifier as specified
	// in https://tools.ietf.org/id/draft-kerwin-file-scheme-07.html#syntax.
	//
	// `watcher` callback is called when the config changes. watcher may be called from
	// a different go routine. After watcher is called Retrieved.Get should be called
	// to get the new config. See description of Retrieved for more details.
	// watcher may be nil, which indicates that the caller is not interested in
	// knowing about the changes.
	//
	// If ctx is cancelled should return immediately with an error.
	// Should never be called concurrently with itself or with Shutdown.
	Retrieve(ctx context.Context, uri string, watcher WatcherFunc) (Retrieved, error)

	// Scheme returns the location scheme used by Retrieve.
	Scheme() string

	// Shutdown signals that the configuration for which this Provider was used to
	// retrieve values is no longer in use and the Provider should close and release
	// any resources that it may have created.
	//
	// This method must be called when the Collector service ends, either in case of
	// success or error. Retrieve cannot be called after Shutdown.
	//
	// Should never be called concurrently with itself or with Retrieve.
	// If ctx is cancelled should return immediately with an error.
	Shutdown(ctx context.Context) error
}

MapProvider is an interface that helps to retrieve a config map and watch for any changes to the config map. Implementations may load the config from a file, a database or any other source.

The typical usage is the following:

r, err := mapProvider.Retrieve("file:/path/to/config")
// Use r.Map; wait for watcher to be called.
r.Close()
r, err = mapProvider.Retrieve("file:/path/to/config")
// Use r.Map; wait for watcher to be called.
r.Close()
// repeat retrieve/wait/close cycle until it is time to shut down the Collector process.
// ...
mapProvider.Shutdown()

type Pipeline

type Pipeline struct {
	Receivers  []ComponentID `mapstructure:"receivers"`
	Processors []ComponentID `mapstructure:"processors"`
	Exporters  []ComponentID `mapstructure:"exporters"`
}

Pipeline defines a single pipeline.

type Pipelines

type Pipelines map[ComponentID]*Pipeline

Pipelines is a map of names to Pipelines.

type Processor

type Processor interface {
	// contains filtered or unexported methods
}

Processor is the configuration of a component.Processor. Specific extensions must implement this interface and must embed ProcessorSettings struct or a struct that extends it.

type ProcessorSettings

type ProcessorSettings struct {
	// contains filtered or unexported fields
}

ProcessorSettings defines common settings for a component.Processor configuration. Specific processors can embed this struct and extend it with more fields if needed.

It is highly recommended to "override" the Validate() function.

When embedded in the processor config it must be with `mapstructure:",squash"` tag.

func NewProcessorSettings

func NewProcessorSettings(id ComponentID) ProcessorSettings

NewProcessorSettings return a new ProcessorSettings with the given ComponentID.

func (*ProcessorSettings) ID

func (ps *ProcessorSettings) ID() ComponentID

ID returns the receiver ComponentID.

func (*ProcessorSettings) SetIDName

func (ps *ProcessorSettings) SetIDName(idName string)

SetIDName sets the receiver name.

func (*ProcessorSettings) Validate

func (ps *ProcessorSettings) Validate() error

Validate validates the configuration and returns an error if invalid.

type Receiver

type Receiver interface {
	// contains filtered or unexported methods
}

Receiver is the configuration of a component.Receiver. Specific extensions must implement this interface and must embed ReceiverSettings struct or a struct that extends it.

type ReceiverSettings

type ReceiverSettings struct {
	// contains filtered or unexported fields
}

ReceiverSettings defines common settings for a component.Receiver configuration. Specific receivers can embed this struct and extend it with more fields if needed.

It is highly recommended to "override" the Validate() function.

When embedded in the receiver config it must be with `mapstructure:",squash"` tag.

func NewReceiverSettings

func NewReceiverSettings(id ComponentID) ReceiverSettings

NewReceiverSettings return a new ReceiverSettings with the given ComponentID.

func (*ReceiverSettings) ID

func (rs *ReceiverSettings) ID() ComponentID

ID returns the receiver ComponentID.

func (*ReceiverSettings) SetIDName

func (rs *ReceiverSettings) SetIDName(idName string)

SetIDName sets the receiver name.

func (*ReceiverSettings) Validate

func (rs *ReceiverSettings) Validate() error

Validate validates the configuration and returns an error if invalid.

type Retrieved

type Retrieved struct {
	// Deprecated: [v0.50.0] Use NewRetrievedFromMap to initialize, and Retrieved.AsMap to access.
	*Map
	// Deprecated: [v0.50.0] Use NewRetrievedFromMap to initialize, and Retrieved.Close to access.
	CloseFunc
}

Retrieved holds the result of a call to the Retrieve method of a Provider object.

func NewRetrievedFromMap

func NewRetrievedFromMap(cfgMap *Map, opts ...RetrievedOption) Retrieved

NewRetrievedFromMap returns a new Retrieved instance that contains a Map data.

• cfgMap the Map that will be merged to the given map in the MergeTo.
• CloseFunc specifies a function to be invoked when the configuration for which it was used to retrieve values is no longer in use and should close and release any watchers that it may have created.

func (Retrieved) AsMap

func (r Retrieved) AsMap() (*Map, error)

AsMap returns the retrieved configuration parsed as a Map.

func (Retrieved) Close

func (r Retrieved) Close(ctx context.Context) error

Close and release any watchers that MapProvider.Retrieve may have created.

Should block until all resources are closed, and guarantee that `onChange` is not going to be called after it returns except when `ctx` is cancelled.

Should never be called concurrently with itself.

type RetrievedOption

type RetrievedOption func(*retrievedSettings)

RetrievedOption options to customize Retrieved values.

func WithRetrievedClose

func WithRetrievedClose(closeFunc CloseFunc) RetrievedOption

WithRetrievedClose overrides the default Retrieved.Close function. The default Retrieved.Close function does nothing and always returns nil.

type Service

type Service struct {
	// Telemetry is the configuration for collector's own telemetry.
	Telemetry ServiceTelemetry `mapstructure:"telemetry"`

	// Extensions are the ordered list of extensions configured for the service.
	Extensions []ComponentID `mapstructure:"extensions"`

	// Pipelines are the set of data pipelines configured for the service.
	Pipelines Pipelines `mapstructure:"pipelines"`
}

Service defines the configurable components of the service.

type ServiceTelemetry

type ServiceTelemetry struct {
	Logs    ServiceTelemetryLogs    `mapstructure:"logs"`
	Metrics ServiceTelemetryMetrics `mapstructure:"metrics"`
}

ServiceTelemetry defines the configurable settings for service telemetry.

type ServiceTelemetryLogs

type ServiceTelemetryLogs struct {
	// Level is the minimum enabled logging level.
	// (default = "INFO")
	Level zapcore.Level `mapstructure:"level"`

	// Development puts the logger in development mode, which changes the
	// behavior of DPanicLevel and takes stacktraces more liberally.
	// (default = false)
	Development bool `mapstructure:"development"`

	// Encoding sets the logger's encoding.
	// Example values are "json", "console".
	Encoding string `mapstructure:"encoding"`

	// DisableCaller stops annotating logs with the calling function's file
	// name and line number. By default, all logs are annotated.
	// (default = false)
	DisableCaller bool `mapstructure:"disable_caller"`

	// DisableStacktrace completely disables automatic stacktrace capturing. By
	// default, stacktraces are captured for WarnLevel and above logs in
	// development and ErrorLevel and above in production.
	// (default = false)
	DisableStacktrace bool `mapstructure:"disable_stacktrace"`

	// OutputPaths is a list of URLs or file paths to write logging output to.
	// The URLs could only be with "file" schema or without schema.
	// The URLs with "file" schema must be an absolute path.
	// The URLs without schema are treated as local file paths.
	// "stdout" and "stderr" are interpreted as os.Stdout and os.Stderr.
	// see details at Open in zap/writer.go.
	// (default = ["stderr"])
	OutputPaths []string `mapstructure:"output_paths"`

	// ErrorOutputPaths is a list of URLs or file paths to write zap internal logger errors to.
	// The URLs could only be with "file" schema or without schema.
	// The URLs with "file" schema must use absolute paths.
	// The URLs without schema are treated as local file paths.
	// "stdout" and "stderr" are interpreted as os.Stdout and os.Stderr.
	// see details at Open in zap/writer.go.
	//
	// Note that this setting only affects the zap internal logger errors.
	// (default = ["stderr"])
	ErrorOutputPaths []string `mapstructure:"error_output_paths"`

	// InitialFields is a collection of fields to add to the root logger.
	// Example:
	//
	// 		initial_fields:
	//	   		foo: "bar"
	//
	// By default, there is no initial field.
	InitialFields map[string]interface{} `mapstructure:"initial_fields"`
}

ServiceTelemetryLogs defines the configurable settings for service telemetry logs. This MUST be compatible with zap.Config. Cannot use directly zap.Config because the collector uses mapstructure and not yaml tags.

type ServiceTelemetryMetrics

type ServiceTelemetryMetrics struct {
	// Level is the level of telemetry metrics, the possible values are:
	//  - "none" indicates that no telemetry data should be collected;
	//  - "basic" is the recommended and covers the basics of the service telemetry.
	//  - "normal" adds some other indicators on top of basic.
	//  - "detailed" adds dimensions and views to the previous levels.
	Level configtelemetry.Level `mapstructure:"level"`

	// Address is the [address]:port that metrics exposition should be bound to.
	Address string `mapstructure:"address"`
}

ServiceTelemetryMetrics exposes the common Telemetry configuration for one component. Experimental: *NOTE* this structure is subject to change or removal in the future.

type Type

type Type string

Type is the component type as it is used in the config.

type Unmarshallable

type Unmarshallable interface {
	// Unmarshal is a function that unmarshals a config.Map into the struct in a custom way.
	// The config.Map for this specific component may be nil or empty if no config available.
	Unmarshal(component *Map) error
}

Unmarshallable defines an optional interface for custom configuration unmarshalling. A configuration struct can implement this interface to override the default unmarshalling.

type WatcherFunc

type WatcherFunc func(*ChangeEvent)