signaltometricsconnector

README

Signal to metrics connector

Signal to metrics connector produces metrics from all signal types (traces, logs, or metrics).

Status
Distributions	[]
Issues
Code Owners	@ChrsMark, @lahsivjar

Supported Pipeline Types

Exporter Pipeline Type	Receiver Pipeline Type	Stability Level
traces	metrics	development
logs	metrics	development
metrics	metrics	development

Configuration

The component can produce metrics from spans, datapoints (for metrics), and logs. At least one of the metrics for one signal type MUST be specified correctly for the component to work.

All signal types can be configured to produce metrics with the same configuration structure. For example, the below configuration will produce delta temporality counters for counting number of events for each of the configured signals:

signaltometrics:
  spans:
    - name: span.count
      description: Count of spans
      sum:
        value: Int(AbsoluteCount()) # Count of total spans represented by each span
  datapoints:
    - name: datapoint.count
      description: Count of datapoints
      sum:
        value: "1" # increment by 1 for each datapoint
  logs:
    - name: logrecord.count
      description: Count of log records
      sum:
        value: "1" # increment by 1 for each log record

Metrics types

signaltometrics produces a variety of metric types by utilizing OTTL to extract the relevant data for a metric type from the incoming data. The component can produce the following metric types for each signal types:

• Sum
• Histogram
• Exponential Histogram

The component does NOT perform any stateful or time based aggregations. The metric types are aggregated for the payload sent in each Consume* call. The final metric is then sent forward in the pipeline.

Sum

Sum metrics have the following configurations:

sum:
  value: <ottl_value_expression>

• [Required] value represents an OTTL expression to extract a value from the incoming data. Only OTTL expressions that return a value are accepted. The returned value determines the value type of the sum metric (int or double). OTTL converters can be used to transform the data.

Histogram

Histogram metrics have the following configurations:

histogram:
  buckets: []float64
  count: <ottl_value_expression>
  value: <ottl_value_expression>

• [Optional] buckets represents the buckets to be used for the histogram. If no buckets are configured then it defaults to:

  []float64{2, 4, 6, 8, 10, 50, 100, 200, 400, 800, 1000, 1400, 2000, 5000, 10_000, 15_000}
  

• [Optional] count represents an OTTL expression to extract the count to be recorded in the histogram from the incoming data. If no expression is provided then it defaults to the count of the signal. OTTL converters can be used to transform the data. For spans, a special converter adjusted count, is provided to help calculte the span's adjusted count.

• [Required] value represents an OTTL expression to extract the value to be recorded in the histogram from the incoming data. OTTL converters can be used to transform the data.

Exponential Histogram

Exponential histogram metrics have the following configurations:

exponential_histogram:
  max_size: <int64>
  count: <ottl_value_expression>
  value: <ottl_value_expression>

• [Optional] max_size represents the maximum number of buckets per positive or negative number range. Defaults to 160.
• [Optional] count represents an OTTL expression to extract the count to be recorded in the expoential histogram from the incoming data. If no expression is provided then it defaults to the count of the signal. OTTL converters can be used to transform the data. For spans, a special converter adjusted count, is provided to help calculte the span's adjusted count.
• [Required] value represents an OTTL expression to extract the value to be recorded in the exponential histogram from the incoming data. OTTL converters can be used to transform the data.

Attributes

The component can produce metrics categorized by the attributes (span attributes for traces, datapoint attributes for datapoints, or log record attributes for logs) from the incoming data by configuring attributes for the configured metrics.

If no attributes are configured then the metrics are produced without any attributes.

attributes:
  - key: datapoint.foo
  - key: datapoint.bar
    default_value: bar

If attributes are specified then a separate metric will be generated for each unique set of attribute values. Optionally, a default_value can be used to always include the attribute with the value of the attribute defaulting to the value specified in default_value if the incoming data is missing that attribute.

Conditions

Conditions are an optional list of OTTL conditions that are evaluated on the incoming data and are ORed together. For example:

signaltometrics:
  datapoints:
    - name: datapoint.bar.sum
      description: Count total number of datapoints as per datapoint.bar attribute
      conditions:
        - resource.attributes["foo"] != nil
        - resource.attributes["bar"] != nil
      sum:
        value: "1"

The above configuration will produce sum metrics from datapoints with either foo OR bar resource attribute defined.

Conditions can also be ANDed together, for example:

signaltometrics:
  datapoints:
    - name: gauge.to.exphistogram
      conditions:
        - metric.type == 1 AND resource.attributes["resource.foo"] != nil
      exponential_histogram:
        count: "1" # 1 count for each datapoint
        value: Double(value_int) + value_double # handle both int and double

The above configuration produces exponential histogram from gauge metrics with resource attributes resource.foo set.

Customizing resource attributes

The component allows customizing the resource attributes for the produced metrics by specifying a list of attributes that should be included in the final metrics. If no attributes are specified for include_resource_attributes then no filtering is performed i.e. all resource attributes of the incoming data is considered.

include_resource_attributes:
  - key: resource.foo # Include resource.foo attribute if present
  - key: resource.bar # Always include resource.bar attribute, default to bar
    default_value: bar

With the above configuration the produced metrics would only have the couple of resource attributes specified in the list:

• resource.foo will be present for the produced metrics if the incoming data also has the attribute defined.
• resource.bar will always be present because of the default_value. If the incoming data does not have a resource attribute with name resource.bar then the configured default_value of bar will be used.

Single writer

Metrics data streams MUST obey single-writer principle. However, since signaltometrics component produces metrics from all signal types and also allows customizing the resource attributes, there is a possibility of violating the single-writer principle. To keep the single-writer principle intact, the component adds collector instance information as resource attributes. The following resource attributes are added to each produced metrics:

signaltometrics.service.name: <service_name_of_the_otel_collector>
signaltometrics.service.namespace: <service_namespace_of_the_otel_collector>
signaltometrics.service.instance.id: <service_instance_id_of_the_otel_collector>

Custom OTTL functions

The component implements a couple of custom OTTL functions:

1. AdjustedCount: a converter capable of calculating adjusted count for a span.
2. get: a temporary solution to parse OTTL expressions with only values. This is only for internal usage and MUST NOT be used explicitly as it is a stopgap measure (see this for more details).

Documentation

Overview

Package signaltometricsconnector provides a stateless connector for generating metrics from raw signals.

Index

• func NewFactory() connector.Factory

Functions

func NewFactory

func NewFactory() connector.Factory

NewFactory returns a ConnectorFactory.