connector

package
v0.0.0-...-1543348 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 7, 2023 License: Apache-2.0 Imports: 5 Imported by: 0

README

Connectors

A connector is both an exporter and receiver. It consumes data as an exporter in one pipeline and emits data as a receiver in another pipeline. It may consume and emit data of the same data type, or of different data types. A connector may generate and emit data to summarize the consumed data, or it may simply replicate or route data.

Status

Connectors are currently in Alpha stage. To enable connectors, run the collector with the service.connectors feature gate. e.g. ./otelcol --config config.yaml --feature-gates connectors.

Supported Data Types

Each type of connector is designed to work with one or more pairs of data types and may only be used to connect pipelines accordingly. (Recall that every pipeline is associated with a single data type, either traces, metrics, or logs.)

For example, the count connector counts traces, metrics, and logs, and reports the counts as a metric. Therefore, it may be used to connect the following types of pipelines.

Exporter Pipeline Type Receiver Pipeline Type
traces metrics
metrics metrics
logs metrics

Another example, the router connector, is useful for routing data onto the appropriate pipeline so that it may be processed in distinct ways and/or exported to an appropriate backend. It does not alter the data it consumes in any ways, nor does it produce any additional data. Therefore, it may be used to connect the following types of pipelines.

Exporter Pipeline Type Receiver Pipeline Type
traces traces
metrics metrics
logs logs

Configuration

Declaration

Connectors are defined within a dedicated connectors section at the top level of the collector config.

The count connector may be used with default settings.

receivers:
  foo:
exporters:
  bar:
connectors:
  count:
  router:
Usage

Recall that a connector is an exporter and a receiver and that each connector MUST be used as both, in separate pipelines.

receivers:
  foo:
exporters:
  bar:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo]
      exporters: [count]
    metrics:
      receivers: [count]
      exporters: [bar]

Connectors can be used alongside traditional exporters.

receivers:
  foo:
exporters:
  bar/traces_backend:
  bar/metrics_backend:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo]
      exporters: [bar/traces_backend, count]
    metrics:
      receivers: [count]
      exporters: [bar/metrics_backend]

Connectors can be used alongside traditional receivers.

receivers:
  foo/traces:
  foo/metrics:
exporters:
  bar:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo/traces]
      exporters: [count]
    metrics:
      receivers: [foo/metrics, count]
      exporters: [bar]

A connector can be an exporter in multiple pipelines.

receivers:
  foo/traces:
  foo/metrics:
  foo/logs:
exporters:
  bar/traces_backend:
  bar/metrics_backend:
  bar/logs_backend:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo/traces]
      exporters: [bar/traces_backend, count]
    metrics:
      receivers: [foo/metrics]
      exporters: [bar/metrics_backend, count]
    logs:
      receivers: [foo/logs]
      exporters: [bar/logs_backend, count]
    metrics/counts:
      receivers: [count]
      exporters: [bar/metrics_backend]

A connector can be a receiver in multiple pipelines.

receivers:
  foo/traces:
  foo/metrics:
exporters:
  bar/traces_backend:
  bar/metrics_backend:
  bar/metrics_backend/2:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo/traces]
      exporters: [bar/traces_backend, count]
    metrics:
      receivers: [count]
      exporters: [bar/metrics_backend]
    metrics/2:
      receivers: [count]
      exporters: [bar/metrics_backend/2]

Multiple connectors can be used in sequence.

receivers:
  foo:
exporters:
  bar:
connectors:
  count:
  count/the_counts:
service:
  pipelines:
    traces:
      receivers: [foo]
      exporters: [count]
    metrics:
      receivers: [count]
      exporters: [bar/metrics_backend, count/the_counts]
    metrics/count_the_counts:
      receivers: [count/the_counts]
      exporters: [bar]

A connector can only be used in a pair of pipelines when it supports the combination of Exporter Pipeline Type and Receiver Pipeline Type.

receivers:
  foo:
exporters:
  bar:
connectors:
  count:
service:
  pipelines:
    traces:
      receivers: [foo]
      exporters: [count]
    logs:
      receivers: [count] # Invalid. The count connector does not support traces -> logs.
      exporters: [bar]
Exporter Pipeline Type

The type of pipeline in which a connector is used as an exporter.

Receiver Pipeline Type

The type of pipeline in which the connector is used as a receiver.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MakeFactoryMap 

func MakeFactoryMap(factories ...Factory) (map[component.Type]Factory, error)

MakeFactoryMap takes a list of connector factories and returns a map with factory type as keys. It returns a non-nil error when there are factories with duplicate type.

Types

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder processor is a helper struct that given a set of Configs and Factories helps with creating processors.

func NewBuilder 

func NewBuilder(cfgs map[component.ID]component.Config, factories map[component.Type]Factory) *Builder

NewBuilder creates a new connector.Builder to help with creating components form a set of configs and factories.

func (*Builder) CreateLogsToLogs 

func (b *Builder) CreateLogsToLogs(ctx context.Context, set CreateSettings, next consumer.Logs) (Logs, error)

CreateLogsToLogs creates a Logs connector based on the settings and config.

func (*Builder) CreateLogsToMetrics 

func (b *Builder) CreateLogsToMetrics(ctx context.Context, set CreateSettings, next consumer.Metrics) (Logs, error)

CreateLogsToMetrics creates a Logs connector based on the settings and config.

func (*Builder) CreateLogsToTraces 

func (b *Builder) CreateLogsToTraces(ctx context.Context, set CreateSettings, next consumer.Traces) (Logs, error)

CreateLogsToTraces creates a Logs connector based on the settings and config.

func (*Builder) CreateMetricsToLogs 

func (b *Builder) CreateMetricsToLogs(ctx context.Context, set CreateSettings, next consumer.Logs) (Metrics, error)

CreateMetricsToLogs creates a Metrics connector based on the settings and config.

func (*Builder) CreateMetricsToMetrics 

func (b *Builder) CreateMetricsToMetrics(ctx context.Context, set CreateSettings, next consumer.Metrics) (Metrics, error)

CreateMetricsToMetrics creates a Metrics connector based on the settings and config.

func (*Builder) CreateMetricsToTraces 

func (b *Builder) CreateMetricsToTraces(ctx context.Context, set CreateSettings, next consumer.Traces) (Metrics, error)

CreateMetricsToTraces creates a Metrics connector based on the settings and config.

func (*Builder) CreateTracesToLogs 

func (b *Builder) CreateTracesToLogs(ctx context.Context, set CreateSettings, next consumer.Logs) (Traces, error)

CreateTracesToLogs creates a Traces connector based on the settings and config.

func (*Builder) CreateTracesToMetrics 

func (b *Builder) CreateTracesToMetrics(ctx context.Context, set CreateSettings, next consumer.Metrics) (Traces, error)

CreateTracesToMetrics creates a Traces connector based on the settings and config.

func (*Builder) CreateTracesToTraces 

func (b *Builder) CreateTracesToTraces(ctx context.Context, set CreateSettings, next consumer.Traces) (Traces, error)

CreateTracesToTraces creates a Traces connector based on the settings and config.

func (*Builder) Factory 

func (b *Builder) Factory(componentType component.Type) component.Factory

func (*Builder) IsConfigured 

func (b *Builder) IsConfigured(componentID component.ID) bool

type CreateLogsToLogsFunc 

type CreateLogsToLogsFunc func(context.Context, CreateSettings, component.Config, consumer.Logs) (Logs, error)

CreateLogsToLogsFunc is the equivalent of Factory.CreateLogsToLogs().

func (CreateLogsToLogsFunc) CreateLogsToLogs 

func (f CreateLogsToLogsFunc) CreateLogsToLogs(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Logs,
) (Logs, error)

CreateLogsToLogs implements Factory.CreateLogsToLogs().

type CreateLogsToMetricsFunc 

type CreateLogsToMetricsFunc func(context.Context, CreateSettings, component.Config, consumer.Metrics) (Logs, error)

CreateLogsToMetricsFunc is the equivalent of Factory.CreateLogsToMetrics().

func (CreateLogsToMetricsFunc) CreateLogsToMetrics 

func (f CreateLogsToMetricsFunc) CreateLogsToMetrics(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Metrics,
) (Logs, error)

CreateLogsToMetrics implements Factory.CreateLogsToMetrics().

type CreateLogsToTracesFunc 

type CreateLogsToTracesFunc func(context.Context, CreateSettings, component.Config, consumer.Traces) (Logs, error)

CreateLogsToTracesFunc is the equivalent of Factory.CreateLogsToTraces().

func (CreateLogsToTracesFunc) CreateLogsToTraces 

func (f CreateLogsToTracesFunc) CreateLogsToTraces(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Traces,
) (Logs, error)

CreateLogsToTraces implements Factory.CreateLogsToTraces().

type CreateMetricsToLogsFunc 

type CreateMetricsToLogsFunc func(context.Context, CreateSettings, component.Config, consumer.Logs) (Metrics, error)

CreateMetricsToLogsFunc is the equivalent of Factory.CreateMetricsToLogs().

func (CreateMetricsToLogsFunc) CreateMetricsToLogs 

func (f CreateMetricsToLogsFunc) CreateMetricsToLogs(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Logs,
) (Metrics, error)

CreateMetricsToLogs implements Factory.CreateMetricsToLogs().

type CreateMetricsToMetricsFunc 

type CreateMetricsToMetricsFunc func(context.Context, CreateSettings, component.Config, consumer.Metrics) (Metrics, error)

CreateMetricsToMetricsFunc is the equivalent of Factory.CreateMetricsToTraces().

func (CreateMetricsToMetricsFunc) CreateMetricsToMetrics 

func (f CreateMetricsToMetricsFunc) CreateMetricsToMetrics(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Metrics,
) (Metrics, error)

CreateMetricsToMetrics implements Factory.CreateMetricsToTraces().

type CreateMetricsToTracesFunc 

type CreateMetricsToTracesFunc func(context.Context, CreateSettings, component.Config, consumer.Traces) (Metrics, error)

CreateMetricsToTracesFunc is the equivalent of Factory.CreateMetricsToTraces().

func (CreateMetricsToTracesFunc) CreateMetricsToTraces 

func (f CreateMetricsToTracesFunc) CreateMetricsToTraces(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Traces,
) (Metrics, error)

CreateMetricsToTraces implements Factory.CreateMetricsToTraces().

type CreateSettings 

type CreateSettings struct {
	// ID returns the ID of the component that will be created.
	ID component.ID

	component.TelemetrySettings

	// BuildInfo can be used by components for informational purposes
	BuildInfo component.BuildInfo
}

CreateSettings configures Connector creators.

type CreateTracesToLogsFunc 

type CreateTracesToLogsFunc func(context.Context, CreateSettings, component.Config, consumer.Logs) (Traces, error)

CreateTracesToLogsFunc is the equivalent of Factory.CreateTracesToLogs().

func (CreateTracesToLogsFunc) CreateTracesToLogs 

func (f CreateTracesToLogsFunc) CreateTracesToLogs(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Logs,
) (Traces, error)

CreateTracesToLogs implements Factory.CreateTracesToLogs().

type CreateTracesToMetricsFunc 

type CreateTracesToMetricsFunc func(context.Context, CreateSettings, component.Config, consumer.Metrics) (Traces, error)

CreateTracesToMetricsFunc is the equivalent of Factory.CreateTracesToMetrics().

func (CreateTracesToMetricsFunc) CreateTracesToMetrics 

func (f CreateTracesToMetricsFunc) CreateTracesToMetrics(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Metrics,
) (Traces, error)

CreateTracesToMetrics implements Factory.CreateTracesToMetrics().

type CreateTracesToTracesFunc 

type CreateTracesToTracesFunc func(context.Context, CreateSettings, component.Config, consumer.Traces) (Traces, error)

CreateTracesToTracesFunc is the equivalent of Factory.CreateTracesToTraces().

func (CreateTracesToTracesFunc) CreateTracesToTraces 

func (f CreateTracesToTracesFunc) CreateTracesToTraces(
	ctx context.Context,
	set CreateSettings,
	cfg component.Config,
	nextConsumer consumer.Traces) (Traces, error)

CreateTracesToTraces implements Factory.CreateTracesToTraces().

type Factory 

type Factory interface {
	component.Factory

	// CreateDefaultConfig creates the default configuration for the Connector.
	// This method can be called multiple times depending on the pipeline
	// configuration and should not cause side-effects that prevent the creation
	// of multiple instances of the Connector.
	// The object returned by this method needs to pass the checks implemented by
	// 'configtest.CheckConfigStruct'. It is recommended to have these checks in the
	// tests of any implementation of the Factory interface.
	CreateDefaultConfig() component.Config

	CreateTracesToTraces(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Traces) (Traces, error)
	CreateTracesToMetrics(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Metrics) (Traces, error)
	CreateTracesToLogs(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Logs) (Traces, error)

	CreateMetricsToTraces(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Traces) (Metrics, error)
	CreateMetricsToMetrics(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Metrics) (Metrics, error)
	CreateMetricsToLogs(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Logs) (Metrics, error)

	CreateLogsToTraces(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Traces) (Logs, error)
	CreateLogsToMetrics(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Metrics) (Logs, error)
	CreateLogsToLogs(ctx context.Context, set CreateSettings, cfg component.Config, nextConsumer consumer.Logs) (Logs, error)

	TracesToTracesStability() component.StabilityLevel
	TracesToMetricsStability() component.StabilityLevel
	TracesToLogsStability() component.StabilityLevel

	MetricsToTracesStability() component.StabilityLevel
	MetricsToMetricsStability() component.StabilityLevel
	MetricsToLogsStability() component.StabilityLevel

	LogsToTracesStability() component.StabilityLevel
	LogsToMetricsStability() component.StabilityLevel
	LogsToLogsStability() component.StabilityLevel
	// contains filtered or unexported methods
}

Factory is factory interface for connectors.

This interface cannot be directly implemented. Implementations must use the NewFactory to implement it.

func NewFactory 

func NewFactory(cfgType component.Type, createDefaultConfig component.CreateDefaultConfigFunc, options ...FactoryOption) Factory

NewFactory returns a Factory.

type FactoryOption 

type FactoryOption interface {
	// contains filtered or unexported methods
}

FactoryOption applies changes to Factory.

func WithLogsToLogs 

func WithLogsToLogs(createLogsToLogs CreateLogsToLogsFunc, sl component.StabilityLevel) FactoryOption

WithLogsToLogs overrides the default "error not supported" implementation for WithLogsToLogs and the default "undefined" stability level.

func WithLogsToMetrics 

func WithLogsToMetrics(createLogsToMetrics CreateLogsToMetricsFunc, sl component.StabilityLevel) FactoryOption

WithLogsToMetrics overrides the default "error not supported" implementation for WithLogsToMetrics and the default "undefined" stability level.

func WithLogsToTraces 

func WithLogsToTraces(createLogsToTraces CreateLogsToTracesFunc, sl component.StabilityLevel) FactoryOption

WithLogsToTraces overrides the default "error not supported" implementation for WithLogsToTraces and the default "undefined" stability level.

func WithMetricsToLogs 

func WithMetricsToLogs(createMetricsToLogs CreateMetricsToLogsFunc, sl component.StabilityLevel) FactoryOption

WithMetricsToLogs overrides the default "error not supported" implementation for WithMetricsToLogs and the default "undefined" stability level.

func WithMetricsToMetrics 

func WithMetricsToMetrics(createMetricsToMetrics CreateMetricsToMetricsFunc, sl component.StabilityLevel) FactoryOption

WithMetricsToMetrics overrides the default "error not supported" implementation for WithMetricsToMetrics and the default "undefined" stability level.

func WithMetricsToTraces 

func WithMetricsToTraces(createMetricsToTraces CreateMetricsToTracesFunc, sl component.StabilityLevel) FactoryOption

WithMetricsToTraces overrides the default "error not supported" implementation for WithMetricsToTraces and the default "undefined" stability level.

func WithTracesToLogs 

func WithTracesToLogs(createTracesToLogs CreateTracesToLogsFunc, sl component.StabilityLevel) FactoryOption

WithTracesToLogs overrides the default "error not supported" implementation for WithTracesToLogs and the default "undefined" stability level.

func WithTracesToMetrics 

func WithTracesToMetrics(createTracesToMetrics CreateTracesToMetricsFunc, sl component.StabilityLevel) FactoryOption

WithTracesToMetrics overrides the default "error not supported" implementation for WithTracesToMetrics and the default "undefined" stability level.

func WithTracesToTraces 

func WithTracesToTraces(createTracesToTraces CreateTracesToTracesFunc, sl component.StabilityLevel) FactoryOption

WithTracesToTraces overrides the default "error not supported" implementation for WithTracesToTraces and the default "undefined" stability level.

type Logs 

type Logs interface {
	component.Component
	consumer.Logs
}

A Logs connector acts as an exporter from a logs pipeline and a receiver to one or more traces, metrics, or logs pipelines. Logs feeds a consumer.Traces, consumer.Metrics, or consumer.Logs with data.

Examples:

  • Structured logs containing span information could be consumed and emitted as traces.
  • Metrics could be extracted from structured logs that contain numeric data.
  • Logs could be collected in one pipeline and routed to another logs pipeline based on criteria such as attributes or other content of the log. The second pipeline can then process and export the log to the appropriate backend.

type Metrics 

type Metrics interface {
	component.Component
	consumer.Metrics
}

A Metrics connector acts as an exporter from a metrics pipeline and a receiver to one or more traces, metrics, or logs pipelines. Metrics feeds a consumer.Traces, consumer.Metrics, or consumer.Logs with data.

Examples:

  • Latency between related data points could be modeled and emitted as traces.
  • Metrics could be collected in one pipeline and routed to another metrics pipeline based on criteria such as attributes or other content of the metric. The second pipeline can then process and export the metric to the appropriate backend.
  • Metrics could be analyzed by a logs connector that emits events when particular criteria are met.

type Traces 

type Traces interface {
	component.Component
	consumer.Traces
}

A Traces connector acts as an exporter from a traces pipeline and a receiver to one or more traces, metrics, or logs pipelines. Traces feeds a consumer.Traces, consumer.Metrics, or consumer.Logs with data.

Examples:

  • Traces could be collected in one pipeline and routed to another traces pipeline based on criteria such as attributes or other content of the trace. The second pipeline can then process and export the trace to the appropriate backend.
  • Traces could be summarized by a metrics connector that emits statistics describing the number of traces observed.
  • Traces could be analyzed by a logs connector that emits events when particular criteria are met.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.