batchprocessor

README

Batch Processor

Supported pipeline types: metric, traces

The batch processor accepts spans or metrics and places them into batches. Batching helps better compress the data and reduce the number of outgoing connections required to transmit the data. This processor supports both size and time based batching.

It is highly recommended to configure the batch processor on every collector. The batch processor should be defined in the pipeline after the memory_limiter as well as any sampling processors. This is because batching should happen after any data drops such as sampling.

Please refer to config.go for the config spec.

The following configuration options can be modified:

• send_batch_size (default = 8192): Number of spans or metrics after which a batch will be sent.
• timeout (default = 200ms): Time duration after which a batch will be sent regardless of size.

Examples:

processors:
  batch:
  batch/2:
    send_batch_size: 10000
    timeout: 10s

Refer to config.yaml for detailed examples on using the processor.

Documentation

Index

• func MetricViews() []*view.View
• type Config
• type Factory
  • func (f *Factory) CreateDefaultConfig() configmodels.Processor
  • func (f *Factory) CreateMetricsProcessor(ctx context.Context, params component.ProcessorCreateParams, ...) (component.MetricsProcessor, error)
  • func (f *Factory) CreateTraceProcessor(ctx context.Context, params component.ProcessorCreateParams, ...) (component.TraceProcessor, error)
  • func (f *Factory) Type() configmodels.Type

Functions

func MetricViews

func MetricViews() []*view.View

MetricViews returns the metrics views related to batching

Types

type Config

type Config struct {
	configmodels.ProcessorSettings `mapstructure:",squash"`

	// Timeout sets the time after which a batch will be sent regardless of size.
	Timeout time.Duration `mapstructure:"timeout,omitempty"`

	// SendBatchSize is the size of a batch which after hit, will trigger it to be sent.
	SendBatchSize uint32 `mapstructure:"send_batch_size,omitempty"`
}

Config defines configuration for batch processor.

type Factory

type Factory struct {
}

Factory is the factory for batch processor.

func (*Factory) CreateDefaultConfig

func (f *Factory) CreateDefaultConfig() configmodels.Processor

CreateDefaultConfig creates the default configuration for processor.

func (*Factory) CreateMetricsProcessor

func (f *Factory) CreateMetricsProcessor(
	ctx context.Context,
	params component.ProcessorCreateParams,
	nextConsumer consumer.MetricsConsumer,
	c configmodels.Processor,
) (component.MetricsProcessor, error)

CreateMetricsProcessor creates a metrics processor based on this config.

func (*Factory) CreateTraceProcessor

func (f *Factory) CreateTraceProcessor(
	ctx context.Context,
	params component.ProcessorCreateParams,
	nextConsumer consumer.TraceConsumer,
	c configmodels.Processor,
) (component.TraceProcessor, error)

CreateTraceProcessor creates a trace processor based on this config.

func (*Factory) Type

func (f *Factory) Type() configmodels.Type

Type gets the type of the config created by this factory.