arrow_record

Documentation

Overview

Package arrow_record contains the consumer and producer for OTLP Arrow protocol.

Index

• Variables
• func NewConsoleObserver(maxRows, maxPrints int) observer.ProducerObserver
• type Config
• type Consumer
  • func NewConsumer(opts ...Option) *Consumer
  • func (c *Consumer) Close() error
  • func (c *Consumer) Consume(bar *colarspb.BatchArrowRecords) (ibes []*record_message.RecordMessage, retErr error)
  • func (c *Consumer) LogsFrom(bar *colarspb.BatchArrowRecords) ([]plog.Logs, error)
  • func (c *Consumer) MetricsFrom(bar *colarspb.BatchArrowRecords) ([]pmetric.Metrics, error)
  • func (c *Consumer) TracesFrom(bar *colarspb.BatchArrowRecords) ([]ptrace.Traces, error)
• type ConsumerAPI
• type Option
  • func WithMemoryLimit(bytes uint64) Option
  • func WithMeterProvider(p metric.MeterProvider, l configtelemetry.Level) Option
  • func WithTracesConfig(tcfg *arrow.Config) Option
• type Producer
  • func NewProducer() *Producer
  • func NewProducerWithOptions(options ...cfg.Option) *Producer
  • func (p *Producer) BatchArrowRecordsFromLogs(ls plog.Logs) (*colarspb.BatchArrowRecords, error)
  • func (p *Producer) BatchArrowRecordsFromMetrics(metrics pmetric.Metrics) (*colarspb.BatchArrowRecords, error)
  • func (p *Producer) BatchArrowRecordsFromTraces(ts ptrace.Traces) (*colarspb.BatchArrowRecords, error)
  • func (p *Producer) Close() error
  • func (p *Producer) GetAndResetStats() pstats.ProducerStats
  • func (p *Producer) LogsBuilder() *logsarrow.LogsBuilder
  • func (p *Producer) LogsRecordBuilderExt() *builder.RecordBuilderExt
  • func (p *Producer) MetricsBuilder() *metricsarrow.MetricsBuilder
  • func (p *Producer) MetricsRecordBuilderExt() *builder.RecordBuilderExt
  • func (p *Producer) Produce(rms []*record_message.RecordMessage) (*colarspb.BatchArrowRecords, error)
  • func (p *Producer) RecordSizeStats() map[string]*pstats.RecordSizeStats
  • func (p *Producer) SetObserver(observer observer.ProducerObserver)
  • func (p *Producer) ShowStats()
  • func (p *Producer) TracesBuilder() *tracesarrow.TracesBuilder
  • func (p *Producer) TracesRecordBuilderExt() *builder.RecordBuilderExt
• type ProducerAPI

Variables

var ErrConsumerMemoryLimit error = common.LimitError{}

ErrConsumerMemoryLimit is used by calling code to check errors.Is(err, ErrConsumerMemoryLimit). It is never returned.

Functions

func NewConsoleObserver

func NewConsoleObserver(maxRows, maxPrints int) observer.ProducerObserver

Types

type Config

type Config struct {
	// contains filtered or unexported fields
}

type Consumer

type Consumer struct {

	// Config embeds the configurable parameters.
	Config
	// contains filtered or unexported fields
}

Consumer is a BatchArrowRecords consumer.

func NewConsumer

func NewConsumer(opts ...Option) *Consumer

NewConsumer creates a new BatchArrowRecords consumer, i.e. a decoder consuming BatchArrowRecords and returning the corresponding OTLP representation (pmetric,Metrics, plog.Logs, ptrace.Traces).

func (*Consumer) Close

func (c *Consumer) Close() error

Close closes the consumer and all its ipc readers.

func (*Consumer) Consume

func (c *Consumer) Consume(bar *colarspb.BatchArrowRecords) (ibes []*record_message.RecordMessage, retErr error)

Consume takes a BatchArrowRecords protobuf message and returns an array of RecordMessage. Note: the records wrapped in the RecordMessage must be released after use by the caller.

func (*Consumer) LogsFrom

func (c *Consumer) LogsFrom(bar *colarspb.BatchArrowRecords) ([]plog.Logs, error)

LogsFrom produces an array of plog.Logs from a BatchArrowRecords message.

func (*Consumer) MetricsFrom

func (c *Consumer) MetricsFrom(bar *colarspb.BatchArrowRecords) ([]pmetric.Metrics, error)

MetricsFrom produces an array of pmetric.Metrics from a BatchArrowRecords message.

func (*Consumer) TracesFrom

func (c *Consumer) TracesFrom(bar *colarspb.BatchArrowRecords) ([]ptrace.Traces, error)

TracesFrom produces an array of ptrace.Traces from a BatchArrowRecords message.

type ConsumerAPI

type ConsumerAPI interface {
	LogsFrom(*colarspb.BatchArrowRecords) ([]plog.Logs, error)
	TracesFrom(*colarspb.BatchArrowRecords) ([]ptrace.Traces, error)
	MetricsFrom(*colarspb.BatchArrowRecords) ([]pmetric.Metrics, error)
	Close() error
}

ConsumerAPI is the interface of a Consumer considering all signals. This is useful for mock testing.

type Option

type Option func(*Config)

func WithMemoryLimit

func WithMemoryLimit(bytes uint64) Option

WithMemoryLimit configures the Arrow limited memory allocator.

func WithMeterProvider

func WithMeterProvider(p metric.MeterProvider, l configtelemetry.Level) Option

WithMeterProvider configures an OTel metrics provider. If none is configured, the global meter provider will be used.

func WithTracesConfig

func WithTracesConfig(tcfg *arrow.Config) Option

WithTracesConfig configures trace-specific Arrow encoding options.

type Producer

type Producer struct {
	// contains filtered or unexported fields
}

Producer is a BatchArrowRecords producer.

func NewProducer

func NewProducer() *Producer

NewProducer creates a new BatchArrowRecords producer.

The method close MUST be called when the producer is not used anymore to release the memory and avoid memory leaks.

func NewProducerWithOptions

func NewProducerWithOptions(options ...cfg.Option) *Producer

NewProducerWithOptions creates a new BatchArrowRecords producer with a set of options.

The method close MUST be called when the producer is not used anymore to release the memory and avoid memory leaks.

func (*Producer) BatchArrowRecordsFromLogs

func (p *Producer) BatchArrowRecordsFromLogs(ls plog.Logs) (*colarspb.BatchArrowRecords, error)

BatchArrowRecordsFromLogs produces a BatchArrowRecords message from a plog.Logs messages.

func (*Producer) BatchArrowRecordsFromMetrics

func (p *Producer) BatchArrowRecordsFromMetrics(metrics pmetric.Metrics) (*colarspb.BatchArrowRecords, error)

BatchArrowRecordsFromMetrics produces a BatchArrowRecords message from a pmetric.Metrics messages.

func (*Producer) BatchArrowRecordsFromTraces

func (p *Producer) BatchArrowRecordsFromTraces(ts ptrace.Traces) (*colarspb.BatchArrowRecords, error)

BatchArrowRecordsFromTraces produces a BatchArrowRecords message from a ptrace.Traces messages.

func (*Producer) Close

func (p *Producer) Close() error

Close closes all stream producers.

func (*Producer) GetAndResetStats

func (p *Producer) GetAndResetStats() pstats.ProducerStats

GetAndResetStats returns the stats and resets them.

func (*Producer) LogsBuilder

func (p *Producer) LogsBuilder() *logsarrow.LogsBuilder

func (*Producer) LogsRecordBuilderExt

func (p *Producer) LogsRecordBuilderExt() *builder.RecordBuilderExt

LogsRecordBuilderExt returns the record builder used to encode logs.

func (*Producer) MetricsBuilder

func (p *Producer) MetricsBuilder() *metricsarrow.MetricsBuilder

func (*Producer) MetricsRecordBuilderExt

func (p *Producer) MetricsRecordBuilderExt() *builder.RecordBuilderExt

MetricsRecordBuilderExt returns the record builder used to encode metrics.

func (*Producer) Produce

func (p *Producer) Produce(rms []*record_message.RecordMessage) (*colarspb.BatchArrowRecords, error)

Produce takes a slice of RecordMessage and returns the corresponding BatchArrowRecords protobuf message.

func (*Producer) RecordSizeStats

func (p *Producer) RecordSizeStats() map[string]*pstats.RecordSizeStats

RecordSizeStats returns statistics per record payload type.

func (*Producer) SetObserver

func (p *Producer) SetObserver(observer observer.ProducerObserver)

SetObserver adds an observer to the producer.

func (*Producer) ShowStats

func (p *Producer) ShowStats()

ShowStats prints the stats to the console.

func (*Producer) TracesBuilder

func (p *Producer) TracesBuilder() *tracesarrow.TracesBuilder

func (*Producer) TracesRecordBuilderExt

func (p *Producer) TracesRecordBuilderExt() *builder.RecordBuilderExt

TracesRecordBuilderExt returns the record builder used to encode traces.

type ProducerAPI

type ProducerAPI interface {
	BatchArrowRecordsFromTraces(ptrace.Traces) (*colarspb.BatchArrowRecords, error)
	BatchArrowRecordsFromLogs(plog.Logs) (*colarspb.BatchArrowRecords, error)
	BatchArrowRecordsFromMetrics(pmetric.Metrics) (*colarspb.BatchArrowRecords, error)
	Close() error
}

ProducerAPI is the interface of a Producer considering all signals. This is useful for mock testing.