resourcedetectionprocessor

README

Resource Detection Processor

Supported pipeline types: metrics, traces, logs

The resource detection processor can be used to detect resource information from the host, in a format that conforms to the OpenTelemetry resource semantic conventions, and append or override the resource value in telemetry data with this information.

Supported detectors

Environment Variable

Reads resource information from the OTEL_RESOURCE_ATTRIBUTES environment variable. This is expected to be in the format <key1>=<value1>,<key2>=<value2>,..., the details of which are currently pending confirmation in the OpenTelemetry specification.

Example:

processors:
  resourcedetection/env:
    detectors: [env]
    timeout: 2s
    override: false

System metadata

Queries the host machine to retrieve the following resource attributes:

* host.name
* os.type

By default host.name is being set to FQDN if possible, and a hostname provided by OS used as fallback. This logic can be changed with hostname_sources configuration which is set to ["dns", "os"] by default.

Use the following config to avoid getting FQDN and apply hostname provided by OS only:

processors:
  resourcedetection/system:
    detectors: ["system"]
    system:
      hostname_sources: ["os"]

• all valid options for hostname_sources:
  • "dns"
  • "os"

Note: use the Docker detector (see below) if running the Collector as a Docker container.

Docker metadata

Queries the Docker daemon to retrieve the following resource attributes from the host machine:

* host.name
* os.type

You need to mount the Docker socket (/var/run/docker.sock on Linux) to contact the Docker daemon. Docker detection does not work on macOS.

Example:

processors:
  resourcedetection/docker:
    detectors: [env, docker]
    timeout: 2s
    override: false

GCE Metadata

Uses the Google Cloud Client Libraries for Go to read resource information from the GCE metadata server to retrieve the following resource attributes:

* cloud.provider ("gcp")
* cloud.platform ("gcp_compute_engine")
* cloud.account.id
* cloud.region
* cloud.availability_zone
* host.id
* host.image.id
* host.type

Example:

processors:
  resourcedetection/gce:
    detectors: [env, gce]
    timeout: 2s
    override: false

GKE: Google Kubernetes Engine

* cloud.provider ("gcp")
* cloud.platform ("gcp_gke")
* k8s.cluster.name (name of the GKE cluster)

Example:

processors:
  resourcedetection/gke:
    detectors: [env, gke]
    timeout: 2s
    override: false

AWS EC2

Uses AWS SDK for Go to read resource information from the EC2 instance metadata API to retrieve the following resource attributes:

* cloud.provider ("aws")
* cloud.platform ("aws_ec2")
* cloud.account.id
* cloud.region
* cloud.availability_zone
* host.id
* host.image.id
* host.name
* host.type

It also can optionally gather tags for the EC2 instance that the collector is running on. Note that in order to fetch EC2 tags, the IAM role assigned to the EC2 instance must have a policy that includes the ec2:DescribeTags permission.

EC2 custom configuration example:

processors:
  resourcedetection/ec2:
    detectors: ["ec2"]
      ec2:
        # A list of regex's to match tag keys to add as resource attributes can be specified
        tags:
          - ^tag1$
          - ^tag2$
          - ^label.*$

If you are using a proxy server on your EC2 instance, it's important that you exempt requests for instance metadata as described in the AWS cli user guide. Failing to do so can result in proxied or missing instance data.

Amazon ECS

Queries the Task Metadata Endpoint (TMDE) to record information about the current ECS Task. Only TMDE V4 and V3 are supported.

* cloud.provider ("aws")
* cloud.platform ("aws_ecs")
* cloud.account.id
* cloud.region
* cloud.availability_zone
* aws.ecs.cluster.arn
* aws.ecs.task.arn
* aws.ecs.task.family
* aws.ecs.task.revision
* aws.ecs.launchtype (V4 only)
* aws.log.group.names (V4 only)
* aws.log.group.arns (V4 only)
* aws.log.stream.names (V4 only)
* aws.log.stream.arns (V4 only)

Example:

processors:
  resourcedetection/ecs:
    detectors: [env, ecs]
    timeout: 2s
    override: false

Amazon Elastic Beanstalk

Reads the AWS X-Ray configuration file available on all Beanstalk instances with X-Ray Enabled.

* cloud.provider ("aws")
* cloud.platform ("aws_elastic_beanstalk")
* deployment.environment
* service.instance.id
* service.version

Example:

processors:
  resourcedetection/elastic_beanstalk:
    detectors: [env, elastic_beanstalk]
    timeout: 2s
    override: false

Amazon EKS

* cloud.provider ("aws")
* cloud.platform ("aws_eks")

Example:

processors:
  resourcedetection/eks:
    detectors: [env, eks]
    timeout: 2s
    override: false

Azure

Queries the Azure Instance Metadata Service to retrieve the following resource attributes:

* cloud.provider ("azure")
* cloud.platform ("azure_vm")
* cloud.region
* cloud.account.id (subscription ID)
* host.id (virtual machine ID)
* host.name
* azure.vm.size (virtual machine size)
* azure.vm.scaleset.name (name of the scale set if any)
* azure.resourcegroup.name (resource group name)

Example:

processors:
  resourcedetection/azure:
    detectors: [env, azure]
    timeout: 2s
    override: false

Azure AKS

• cloud.provider ("azure")
• cloud.platform ("azure_aks")

processors:
  resourcedetection/aks:
    detectors: [env, aks]
    timeout: 2s
    override: false

Consul

Queries a consul agent and reads its' configuration endpoint to retrieve the following resource attributes:

• cloud.region (consul datacenter)
• host.id (consul node id)
• host.name (consul node name)
• exploded consul metadata - reads all key:value pairs in consul metadata into label:labelvalue pairs.

processors:
  resourcedetection/consul:
    detectors: [env, consul]
    timeout: 2s
    override: false

Configuration

# a list of resource detectors to run, valid options are: "env", "system", "gce", "gke", "ec2", "ecs", "elastic_beanstalk", "eks", "azure"
detectors: [ <string> ]
# determines if existing resource attributes should be overridden or preserved, defaults to true
override: <bool>

Ordering

Note that if multiple detectors are inserting the same attribute name, the first detector to insert wins. For example if you had detectors: [eks, ec2] then cloud.platform will be aws_eks instead of ec2. The below ordering is recommended.

GCP

• gke
• gce

AWS

• elastic_beanstalk
• eks
• ecs
• ec2

The full list of settings exposed for this extension are documented here with detailed sample configurations here.

Documentation

Overview

package resourcedetectionprocessor implements a processor for specifying resource labels to be added to OpenCensus trace data and metrics data.

Index

• func NewFactory() component.ProcessorFactory
• type Config
  • func (cfg *Config) Validate() error
• type DetectorConfig
  • func (d *DetectorConfig) GetConfigFromType(detectorType internal.DetectorType) internal.DetectorConfig

Functions

func NewFactory

func NewFactory() component.ProcessorFactory

NewFactory creates a new factory for ResourceDetection processor.

Types

type Config

type Config struct {
	config.ProcessorSettings `mapstructure:",squash"` // squash ensures fields are correctly decoded in embedded struct

	// Detectors is an ordered list of named detectors that should be
	// run to attempt to detect resource information.
	Detectors []string `mapstructure:"detectors"`
	// Override indicates whether any existing resource attributes
	// should be overridden or preserved. Defaults to true.
	Override bool `mapstructure:"override"`
	// DetectorConfig is a list of settings specific to all detectors
	DetectorConfig DetectorConfig `mapstructure:",squash"`
	// HTTP client settings for the detector
	// Timeout default is 5s
	confighttp.HTTPClientSettings `mapstructure:",squash"`
}

Config defines configuration for Resource processor.

func (*Config) Validate

func (cfg *Config) Validate() error

Validate config

type DetectorConfig

type DetectorConfig struct {
	// EC2Config contains user-specified configurations for the EC2 detector
	EC2Config ec2.Config `mapstructure:"ec2"`

	// ConsulConfig contains user-specified configurations for the Consul detector
	ConsulConfig consul.Config `mapstructure:"consul"`

	// SystemConfig contains user-specified configurations for the System detector
	SystemConfig system.Config `mapstructure:"system"`
}

DetectorConfig contains user-specified configurations unique to all individual detectors

func (*DetectorConfig) GetConfigFromType

func (d *DetectorConfig) GetConfigFromType(detectorType internal.DetectorType) internal.DetectorConfig