telegrafreceiver

README

Telegraf Receiver

Stability level: Beta

Telegraf receiver for ingesting metrics from various input plugins into otc pipeline.

Supported pipeline types: metrics

Use case: user configures telegraf input plugins in config for ingestion and otc processors and exporters for data processing and export.

Configuration

The following settings are required:

• agent_config: Telegraf config. For now it allows to provide agent and input plugins configuration. One can refer to telegraf configuration docs for full list of configuration options.

The Following settings are optional:

• separate_field (default value is false): Specify whether metric field should be added separately as data point label.
• consume_retry_delay (default value is 500ms): The retry delay for recoverable errors from the rest of the pipeline. Don't change this or the related setting below unless you know what you're doing.
• consume_max_retries (default value is 10): The maximum number of retries for recoverable errors from the rest of the pipeline.

Example:

receivers:
  telegraf:
    separate_field: false
    agent_config: |
      [agent]
        interval = "2s"
        flush_interval = "3s"
      [[inputs.mem]]

The full list of settings exposed for this receiver are documented in config.go.

Limitations

With its current implementation Telegraf receiver has the following limitations:

• only input plugins can be configured in telegraf agent confugration section (apart from agent's configuration itself). That means that metrics go straight from input plugin to the receiver for translation (into otc data model) without any processing
• only the following Telegraf metric data types are supported:
  • telegraf.Gauge that is translated to pmetric.MetricDataTypeGauge,
  • telegraf.Counter that is translated to pmetric.MetricDataTypeSum.

Migration from Telegraf

Data model

Internal OTC metric format differs from the Telegraf one and separate_field controls the conversion:

• If separate_field is false, the Open Telemetry metric name is going to be concatenated from the Telegraf metric name and the Telegraf field with _ as separator.

  The following telegraf structure:

  {
    "fields": {
      "HeapMemoryUsage.committed": 1007157248
      "HeapMemoryUsage.init": 1007157248
    },
    "name": "tomcat_jmx_jvm_memory",
    "tags": {
      "component": "webserver",
      "environment": "dev",
      "host": "32fafdb10522",
      "jolokia_agent_url": "http://tomcat:8080/jolokia",
      "webserver_system": "tomcat"
    },
    "timestamp": 1646904912
  }
  

  is going to be converted to the following OpenTelemetry structure:

  2022-03-10T07:16:34.117Z  DEBUG loggingexporter/logging_exporter.go:64
  ResourceMetrics #0
  Resource SchemaURL:
  Resource labels:
      -> component: STRING(webserver)
      -> environment: STRING(dev)
      -> host: STRING(32fafdb10522)
      -> jolokia_agent_url: STRING(http://tomcat:8080/jolokia)
      -> webserver_system: STRING(tomcat)
  InstrumentationLibraryMetrics #0
  InstrumentationLibraryMetrics SchemaURL:
  InstrumentationLibrary telegraf v0.1
  Metric #0
  Descriptor:
      -> Name: tomcat_jmx_jvm_memory_HeapMemoryUsage.committed
      -> Description:
      -> Unit:
      -> DataType: Gauge
  NumberDataPoints #0
  StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
  Timestamp: 2022-03-10 09:35:12 +0000 UTC
  Value: 1007157248.000000
  Metric #1
  Descriptor:
      -> Name: tomcat_jmx_jvm_memory_HeapMemoryUsage.init
      -> Description:
      -> Unit:
      -> DataType: Gauge
  NumberDataPoints #0
  StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
  Timestamp: 2022-03-10 09:35:12 +0000 UTC
  Value: 1007157248.000000
  

• If separate_fields is true, the Open Telemetry metric name is going to be the same as the Telegraf one, and the Telegraf field is going to be converted to the Open Telemetry data point attribute.

  The following telegraf structure:

  {
    "fields": {
      "HeapMemoryUsage.committed": 1007157248
      "HeapMemoryUsage.init": 1007157248
    },
    "name": "tomcat_jmx_jvm_memory",
    "tags": {
      "component": "webserver",
      "environment": "dev",
      "host": "32fafdb10522",
      "jolokia_agent_url": "http://tomcat:8080/jolokia",
      "webserver_system": "tomcat"
    },
    "timestamp": 1646904912
  }
  

  is going to be converted to the following OpenTelemetry structure:

  2022-03-10T11:28:30.333Z  DEBUG loggingexporter/logging_exporter.go:64
  ResourceMetrics #0
  Resource SchemaURL:
  Resource labels:
      -> component: STRING(webserver)
      -> environment: STRING(dev)
      -> host: STRING(32fafdb10522)
      -> jolokia_agent_url: STRING(http://tomcat:8080/jolokia)
      -> webserver_system: STRING(tomcat)
  InstrumentationLibraryMetrics #0
  InstrumentationLibraryMetrics SchemaURL:
  InstrumentationLibrary
  Metric #0
  Descriptor:
      -> Name: tomcat_jmx_jvm_memory
      -> Description:
      -> Unit:
      -> DataType: Gauge
  NumberDataPoints #0
  Data point attributes:
      -> field: STRING(HeapMemoryUsage.committed)
  StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
  Timestamp: 2022-03-10 09:35:12 +0000 UTC
  Value: 1007157248.000000
  Metric #1
  Descriptor:
      -> Name: tomcat_jmx_jvm_memory
      -> Description:
      -> Unit:
      -> DataType: Gauge
  NumberDataPoints #0
  Data point attributes:
      -> field: STRING(HeapMemoryUsage.init)
  StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
  Timestamp: 2022-03-10 09:35:12 +0000 UTC
  Value: 1007157248.000000
  

Keep compatibility while sending metrics to Sumo Logic

In Telegraf, metrics can be sent to Sumo Logic using Sumologic Output Plugin. It supports three formats (prometheus, carbon2, graphite), where each of them has some limitations (e.g. only specific set of chars can be used for metric name for prometheus).

OTLP doesn't have most of those limitations, so in order to keep the same metric names, some transformations should be done by processors.

Let's consider the following example. metric.with.dots is going to be sent as metric_with_dots by prometheus or metric.with.dots by OTLP. To unify it, you can use the Metrics Transform Processor:

processors:
  metricstransform:
    transforms:
      ## Replace metric.with.dots metric to metric_with_dots
      - include: metric.with.dots
        match_type: strict
        action: update
        new_name: metric_with_dots
# ...
service:
  pipelines:
    metrics:
      receivers:
        - telegraf
      processors:
        - metricstransform
      exporters:
        - sumologic
# ...

With Metrics Transform Processor and regular expressions you can also handle more complex scenarios, like in the following snippet:

processors:
  metricstransform:
    transforms:
      ## Change <part1>.<part2> metrics to to <part1>_<part2>
      - include: ^([^\.]*)\.([^\.]*)$$
        match_type: strict
        action: update
        new_name: $${1}.$${2}
      ## Change <part1>.<part2>.<part3> metrics to to <part1>_<part2>_<part3>
      - include: ^([^\.]*)\.([^\.]*)\.([^\.]*)$$
        match_type: strict
        action: update
        new_name: $${1}.$${2}.${3}
# ...
service:
  pipelines:
    metrics:
      receivers:
        - telegraf
      processors:
        - metricstransform
      exporters:
        - sumologic
# ...

Documentation

Index

• Variables
• func NewFactory() receiver.Factory
• type AttributeMapOpt
• type Config
• type MetricConverter
• type MetricOpt
  • func WithField(field string) MetricOpt
  • func WithName(name string) MetricOpt
  • func WithTag(tag *telegraf.Tag) MetricOpt
  • func WithTags(tags []*telegraf.Tag) MetricOpt
  • func WithTime(t time.Time) MetricOpt
• type TimeOpt

Variables

var (
	ErrAlreadyStarted = errors.New("component already started")
	ErrAlreadyStopped = errors.New("component already stopped")
)

Functions

func NewFactory

func NewFactory() receiver.Factory

NewFactory creates a factory for telegraf receiver.

Types

type AttributeMapOpt

type AttributeMapOpt func(attributeMap pcommon.Map)

type Config

type Config struct {
	// AgentConfig is the yaml config used as telegraf configuration.
	// Please note that only inputs should be configured as all metrics gathered
	// by them will be passed through to otc pipeline for processing and export.
	AgentConfig string `mapstructure:"agent_config"`

	// SeparateField controls whether the ingested metrics should have a field
	// concatenated with metric name like e.g. metric=mem_available or maybe rather
	// have it as a separate label like e.g. metric=mem field=available
	SeparateField bool `mapstructure:"separate_field"`

	// ConsumeRetryDelay is the retry delay for recoverable pipeline errors
	// one frequent source of these kinds of errors is the memory_limiter processor
	ConsumeRetryDelay time.Duration `mapstructure:"consume_retry_delay"`

	// ConsumeMaxRetries is the maximum number of retries for recoverable pipeline errors
	ConsumeMaxRetries uint64 `mapstructure:"consume_max_retries"`
}

Config defines configuration for the telegraf receiver.

type MetricConverter

type MetricConverter interface {
	Convert(telegraf.Metric) (pmetric.Metrics, error)
}

type MetricOpt

type MetricOpt func(m pmetric.Metric)

MetricOpt is an option func that takes in a pmetric.Metric and manipulates it.

func WithField

func WithField(field string) MetricOpt

WithField returns a MetricOpt which will set the returned metric's field tag to the specified one.

func WithName

func WithName(name string) MetricOpt

WithName returns a MetricOpt which will set the returned metric name.

func WithTag

func WithTag(tag *telegraf.Tag) MetricOpt

WithTag returns a MetricOpt which will insert a specified telegraf tag into all underlying data points' label maps.

func WithTags

func WithTags(tags []*telegraf.Tag) MetricOpt

WithTags returns a MetricOpt which will insert a list of telegraf tags into all underlying data points' label maps.

func WithTime

func WithTime(t time.Time) MetricOpt

WithTime returns a MetricOpt which will set the returned metric's timestamp.

type TimeOpt

type TimeOpt func() time.Time