routingconnector

README

Routing Connector

Status
Distributions	contrib, k8s
Issues
Code Owners	@jpkrohling, @mwear

Supported Pipeline Types

Exporter Pipeline Type	Receiver Pipeline Type	Stability Level
traces	traces	alpha
metrics	metrics	alpha
logs	logs	alpha

Routes logs, metrics or traces based on resource attributes to specific pipelines using OpenTelemetry Transformation Language (OTTL) statements as routing conditions.

Configuration

If you are not already familiar with connectors, you may find it helpful to first visit the Connectors README.

The following settings are available:

• table (required): the routing table for this connector.
• table.context (optional, default: resource): the OTTL Context in which the statement will be evaluated. Currently, only resource, span, metric, log, and request are supported.
• table.statement: the routing condition provided as the OTTL statement. Required if table.condition is not provided. May not be used for request context.
• table.condition: the routing condition provided as the OTTL condition. Required if table.statement is not provided. Required for request context.
• table.pipelines (required): the list of pipelines to use when the routing condition is met.
• default_pipelines (optional): contains the list of pipelines to use when a record does not meet any of specified conditions.
• error_mode (optional): determines how errors returned from OTTL statements are handled. Valid values are propagate, ignore and silent. If ignore or silent is used and a statement's condition has an error then the payload will be routed to the default pipelines. When silent is used the error is not logged. If not supplied, propagate is used.
• match_once (optional, default: false): determines whether the connector matches multiple statements or not. If enabled, the payload will be routed to the first pipeline in the table whose routing condition is met. May only be false when used with resource context.

Limitations

• The match_once setting is only supported when using the resource context. If any routes use span, metric, log or request context, match_once must be set to true.
• The request context requires use of the condition setting, and relies on a very limited grammar. Conditions must be in the form of request["key"] == "value" or request["key"] != "value". (In the future, this grammar may be expanded to support more complex conditions.)

Supported OTTL functions

• IsMatch
• delete_key
• delete_matching_keys

Additional Settings

The full list of settings exposed for this connector are documented here with detailed sample configuration files:

• logs
• metrics
• traces

Examples

Route traces based on an attribute:

receivers:
    otlp:

exporters:
  jaeger:
    endpoint: localhost:14250
  jaeger/acme:
    endpoint: localhost:24250
  jaeger/ecorp:
    endpoint: localhost:34250

connectors:
  routing:
    default_pipelines: [traces/jaeger]
    error_mode: ignore
    match_once: false
    table:
      - statement: route() where attributes["X-Tenant"] == "acme"
        pipelines: [traces/jaeger-acme]
      - statement: delete_key(attributes, "X-Tenant") where IsMatch(attributes["X-Tenant"], ".*corp")
        pipelines: [traces/jaeger-ecorp]

  routing/match_once:
    default_pipelines: [traces/jaeger]
    error_mode: ignore
    match_once: true
    table:
      - statement: route() where attributes["X-Tenant"] == "acme"
        pipelines: [traces/jaeger-acme]
      - statement: route() where attributes["X-Tenant"] == ".*acme"
        pipelines: [traces/jaeger-ecorp]

service:
  pipelines:
    traces/in:
      receivers: [otlp]
      exporters: [routing]
    traces/jaeger:
      receivers: [routing]
      exporters: [jaeger]
    traces/jaeger-acme:
      receivers: [routing]
      exporters: [jaeger/acme]
    traces/jaeger-ecorp:
      receivers: [routing]
      exporters: [jaeger/ecorp]

Route logs based on tenant:

receivers:
    otlp:

exporters:
  file/other:
    path: ./other.log
  file/acme:
    path: ./acme.log
  file/ecorp:
    path: ./ecorp.log

connectors:
  routing:
    match_once: true
    default_pipelines: [logs/other]
    table:
      - context: request
        condition: reqeust["X-Tenant"] == "acme"
        pipelines: [logs/acme]
      - context: request
        condition: reqeust["X-Tenant"] == "ecorp"
        pipelines: [logs/ecorp]

service:
  pipelines:
    logs/in:
      receivers: [otlp]
      exporters: [routing]
    logs/acme:
      receivers: [routing]
      exporters: [file/acme]
    logs/ecorp:
      receivers: [routing]
      exporters: [file/ecorp]
    logs/other:
      receivers: [routing]
      exporters: [file/other]

Route logs based on region:

receivers:
    otlp:

exporters:
  file/other:
    path: ./other.log
  file/east:
    path: ./east.log
  file/west:
    path: ./west.log

connectors:
  routing:
    match_once: true
    default_pipelines: [logs/other]
    table:
      - context: log
        condition: attributes["region"] == "east"
        pipelines: [logs/east]
      - context: log
        condition: attributes["region"] == "west"
        pipelines: [logs/west]

service:
  pipelines:
    logs/in:
      receivers: [otlp]
      exporters: [routing]
    logs/east:
      receivers: [routing]
      exporters: [file/east]
    logs/west:
      receivers: [routing]
      exporters: [file/west]
    logs/other:
      receivers: [routing]
      exporters: [file/other]

Route all low level logs to cheap storage. Route the remainder based on service name:

receivers:
    otlp:

exporters:
  file/cheap:
    path: ./cheap.log
  file/service1:
    path: ./service1-important.log
  file/ecorp:
    path: ./service2-important.log

connectors:
  routing:
    match_once: true
    table:
      - context: log
        condition: severity_number < SEVERITY_NUMBER_ERROR
        pipelines: [logs/cheap]
      - context: resource
        condition: attributes["service.name"] == "service1"
        pipelines: [logs/service1]
      - context: resource
        condition: attributes["service.name"] == "service2"
        pipelines: [logs/service2]

service:
  pipelines:
    logs/in:
      receivers: [otlp]
      exporters: [routing]
    logs/cheap:
      receivers: [routing]
      exporters: [file/cheap]
    logs/service1:
      receivers: [routing]
      exporters: [file/service1]
    logs/service2:
      receivers: [routing]
      exporters: [file/service2]

Route all low level logs to cheap storage. Route the remainder based on tenant:

receivers:
    otlp:

exporters:
  file/cheap:
    path: ./cheap.log
  file/acme:
    path: ./acme.log
  file/ecorp:
    path: ./ecorp.log

connectors:
  routing:
    match_once: true
    table:
      - context: log
        condition: severity_number < SEVERITY_NUMBER_ERROR
        pipelines: [logs/cheap]
      - context: request
        condition: reqeust["X-Tenant"] == "acme"
        pipelines: [logs/acme]
      - context: request
        condition: reqeust["X-Tenant"] == "ecorp"
        pipelines: [logs/ecorp]

service:
  pipelines:
    logs/in:
      receivers: [otlp]
      exporters: [routing]
    logs/cheap:
      receivers: [routing]
      exporters: [file/cheap]
    logs/acme:
      receivers: [routing]
      exporters: [file/acme]
    logs/ecorp:
      receivers: [routing]
      exporters: [file/ecorp]

Differences between the Routing Connector and Routing Processor

• The connector routes to pipelines, not exporters as the processor does.

Documentation

Index

• func NewFactory() connector.Factory
• type Config
  • func (c *Config) Validate() error
• type RoutingTableItem

Functions

func NewFactory

func NewFactory() connector.Factory

NewFactory returns a ConnectorFactory.

Types

type Config

type Config struct {
	// DefaultPipelines contains the list of pipelines to use when a more specific record can't be
	// found in the routing table.
	// Optional.
	DefaultPipelines []pipeline.ID `mapstructure:"default_pipelines"`

	// ErrorMode determines how the processor reacts to errors that occur while processing an OTTL
	// condition.
	// Valid values are `ignore` and `propagate`.
	// `ignore` means the processor ignores errors returned by conditions and continues on to the
	// next condition. This is the recommended mode. If `ignore` is used and a statement's
	// condition has an error then the payload will be routed to the default exporter. `propagate`
	// means the processor returns the error up the pipeline.  This will result in the payload being
	// dropped from the collector.
	// The default value is `propagate`.
	ErrorMode ottl.ErrorMode `mapstructure:"error_mode"`

	// Table contains the routing table for this processor.
	// Required.
	Table []RoutingTableItem `mapstructure:"table"`

	// MatchOnce determines whether the connector matches multiple statements.
	// Optional.
	MatchOnce bool `mapstructure:"match_once"`
}

Config defines configuration for the Routing processor.

func (*Config) Validate

func (c *Config) Validate() error

Validate checks if the processor configuration is valid.

type RoutingTableItem

type RoutingTableItem struct {
	// One of "request", "resource", "log" (other OTTL contexts will be added in the future)
	// Optional. Default "resource".
	Context string `mapstructure:"context"`

	// Statement is a OTTL statement used for making a routing decision.
	// One of 'Statement' or 'Condition' must be provided.
	Statement string `mapstructure:"statement"`

	// Condition is an OTTL condition used for making a routing decision.
	// One of 'Statement' or 'Condition' must be provided.
	Condition string `mapstructure:"condition"`

	// Pipelines contains the list of pipelines to use when the value from the FromAttribute field
	// matches this table item. When no pipelines are specified, the ones specified under
	// DefaultPipelines are used, if any.
	// The routing processor will fail upon the first failure from these pipelines.
	// Optional.
	Pipelines []pipeline.ID `mapstructure:"pipelines"`
}

RoutingTableItem specifies how data should be routed to the different pipelines