throughputmeasurementprocessor

README

Throughput Measurement Processor

This processor samples OTLP payloads and measures the protobuf size as well as number of OTLP objects in that payload. These measurements are added to the following counter metrics that can be accessed via the collectors internal telemetry service. Units for each data_size counter are in Bytes.

Counters:

• log_data_size
• metric_data_size
• trace_data_size
• log_count
• metric_count
• trace_count

Minimum agent versions

• Introduced: v1.8.0

Supported pipelines:

• Logs
• Metrics
• Traces

Configuration

Field	Type	Default	Description
enabled	bool	true	When true signals that measurements are being taken of data passing through this processor. If false this processor acts as a no-op.
sampling_ratio	float	0.5	The ratio of data payloads that are sampled. Values between 0.0 and 1.0. Values closer to 1.0 mean any individual payload is more likely to have its size measured.

Example configuration

The example configuration below shows ingesting logs and sampling the size of 50% of the OTLP log objects.

receivers:
  filelog:
    inclucde: ["/var/log/*.log"]

processors:
  throughputmeasurement:
    enabled: true
    sampling_ratio: 0.5

exporters:
  googlecloud:

service:
  pipelines:
    logs:
      receivers:
        - filelog
      processors:
        - throughputmeasurement
      exporters:
        - googlecloud

The above configuration will add metrics to the collectors internal metrics service which can be scraped via the http://localhost:8888/metrics endpoint.

More info on the internal metric service can be found here.

Documentation

Overview

Package throughputmeasurementprocessor provides a processor that measure the amount of otlp structures flowing through it

Index

• func GetThroughputRegistry(host component.Host, bindplane component.ID) (measurements.ThroughputMeasurementsRegistry, error)
• func NewFactory() processor.Factory
• type Config
  • func (cfg Config) Validate() error

Functions

func GetThroughputRegistry

func GetThroughputRegistry(host component.Host, bindplane component.ID) (measurements.ThroughputMeasurementsRegistry, error)

GetThroughputRegistry returns the throughput registry that should be registered to based on the component ID. nil, nil may be returned by this function. In this case, the processor should not register it's throughput measurements anywhere.

func NewFactory

func NewFactory() processor.Factory

NewFactory creates a new ProcessorFactory with default configuration

Types

type Config

type Config struct {
	// Enable controls whether measurements are taken or not.
	Enabled bool `mapstructure:"enabled"`

	// SamplingRatio is the ratio of payloads that are measured. Values between 0.0 and 1.0 are valid.
	SamplingRatio float64 `mapstructure:"sampling_ratio"`

	// Bindplane extension to use in order to report metrics. Optional.
	BindplaneExtension component.ID `mapstructure:"bindplane_extension"`

	// Extra labels to add to measurements and associate with emitted metrics
	ExtraLabels map[string]string `mapstructure:"extra_labels"`
}

Config is the configuration for the processor

func (Config) Validate

func (cfg Config) Validate() error

Validate validates the processor configuration