config

Documentation

Index

• Variables
• type Config
  • func DefaultConfig() *Config
• type Option
  • func WithAllocator(allocator memory.Allocator) Option
  • func WithCompressionRatioStats() Option
  • func WithDictResetThreshold(dictResetThreshold float64) Option
  • func WithDumpRecordRows(payloadType string, numRows int) Option
  • func WithNoDictionary() Option
  • func WithNoZstd() Option
  • func WithObserver(observer observer.ProducerObserver) Option
  • func WithOrderAttrs16By(orderAttrs16By OrderAttrs16By) Option
  • func WithOrderAttrs32By(orderAttrs32By OrderAttrs32By) Option
  • func WithOrderSpanBy(orderSpanBy OrderSpanBy) Option
  • func WithProducerStats() Option
  • func WithRecordStats() Option
  • func WithSchemaStats() Option
  • func WithSchemaUpdates() Option
  • func WithUint16InitDictIndex() Option
  • func WithUint16LimitDictIndex() Option
  • func WithUint32LimitDictIndex() Option
  • func WithUint32LinitDictIndex() Option
  • func WithUint64InitDictIndex() Option
  • func WithUint64LimitDictIndex() Option
  • func WithUint8InitDictIndex() Option
  • func WithUint8LimitDictIndex() Option
  • func WithZstd() Option
• type OrderAttrs16By
• type OrderAttrs32By
• type OrderSpanBy

Variables

var OrderAttrs16ByVariants = map[string]OrderAttrs16By{
	"":                         OrderAttrs16ByNothing,
	"parent_id,key,value":      OrderAttrs16ByParentIdKeyValue,
	"type,key,parent_id,value": OrderAttrs16ByTypeKeyParentIdValue,
	"type,key,value,parent_id": OrderAttrs16ByTypeKeyValueParentId,
}

OrderAttrs16ByVariants is a map of string to OrderAttrs16By. It is used to iterate over the possible values of OrderAttrs16By. This map must be kept in sync with the OrderAttrs16By enumeration.

var OrderAttrs32ByVariants = map[string]OrderAttrs32By{
	"":                         OrderAttrs32ByNothing,
	"type,parent_id,key,value": OrderAttrs32ByTypeParentIdKeyValue,
	"type,key,parent_id,value": OrderAttrs32ByTypeKeyParentIdValue,
	"type,key,value,parent_id": OrderAttrs32ByTypeKeyValueParentId,
	"key,value,parent_id":      OrderAttrs32ByKeyValueParentId,
}

OrderAttrs32ByVariants is a map of string to OrderAttrs32By. It is used to iterate over the possible values of OrderAttrs32By. This map must be kept in sync with the OrderAttrs32By enumeration.

var OrderSpanByVariants = map[string]OrderSpanBy{
	"":                         OrderSpanByNothing,
	"name,trace_id":            OrderSpanByNameTraceID,
	"trace_id,name":            OrderSpanByTraceIDName,
	"name,start_time":          OrderSpanByNameStartTime,
	"name,trace_id,start_time": OrderSpanByNameTraceIdStartTime,
	"start_time,trace_id,name": OrderSpanByStartTimeTraceIDName,
	"start_time,name,trace_id": OrderSpanByStartTimeNameTraceID,
}

OrderSpanByVariants is a map of string to OrderSpanBy. It is used to iterate over the possible values of OrderSpanBy. This map must be kept in sync with the OrderSpanBy enumeration.

Types

type Config

type Config struct {
	Pool memory.Allocator

	// InitIndexSize sets the initial size of a dictionary index.
	InitIndexSize uint64
	// LimitIndexSize sets the maximum size of a dictionary index
	// before it is no longer encoded as a dictionary.
	LimitIndexSize uint64
	// DictResetThreshold specifies the ratio under which a dictionary overflow
	// is converted to a dictionary reset. This ratio is calculated as:
	//   (# of unique values in the dict) / (# of values inserted in the dict.)
	// This ratio characterizes the efficiency of the dictionary. Smaller is the
	// ratio, more efficient is the dictionary in term compression ratio because
	// it means that the dictionary entries are reused more often.
	DictResetThreshold float64

	// Zstd enables the use of ZSTD compression for IPC messages.
	Zstd bool // Use IPC ZSTD compression

	// SchemaStats enables the collection of statistics about Arrow schemas.
	SchemaStats bool
	// RecordStats enables the collection of statistics about Arrow records.
	RecordStats bool
	// Display schema updates
	SchemaUpdates bool
	// Display producer statistics
	ProducerStats bool
	// Display compression ratio statistics
	CompressionRatioStats bool
	// DumpRecordRows specifies the number of rows to dump for each record.
	// If not defined or set to 0, no rows are dumped.
	DumpRecordRows map[string]int

	// OrderSpanBy specifies how to order spans in a batch.
	OrderSpanBy OrderSpanBy
	// OrderAttrs16By specifies how to order attributes in a batch
	// (with 16bits attribute ID).
	OrderAttrs16By OrderAttrs16By
	// OrderAttrs32By specifies how to order attributes in a batch
	// (with 32bits attribute ID).
	OrderAttrs32By OrderAttrs32By

	// Observer is the optional observer to use for the producer.
	Observer observer.ProducerObserver
}

func DefaultConfig

func DefaultConfig() *Config

DefaultConfig returns a Config with the following default values:

• Pool: memory.NewGoAllocator()
• InitIndexSize: math.MaxUint16
• LimitIndexSize: math.MaxUint32
• SchemaStats: false
• Zstd: true

type Option

type Option func(*Config)

func WithAllocator

func WithAllocator(allocator memory.Allocator) Option

WithAllocator sets the allocator to use for the Producer.

func WithCompressionRatioStats

func WithCompressionRatioStats() Option

WithCompressionRatioStats enables the display of compression ratio statistics.

func WithDictResetThreshold

func WithDictResetThreshold(dictResetThreshold float64) Option

WithDictResetThreshold sets the ratio under which a dictionary overflow is converted to a dictionary reset. This ratio is calculated as:

(# of unique values in the dict) / (# of values inserted in the dict.)

func WithDumpRecordRows

func WithDumpRecordRows(payloadType string, numRows int) Option

WithDumpRecordRows specifies the number of rows to dump for a specific payload type.

func WithNoDictionary

func WithNoDictionary() Option

WithNoDictionary sets the Producer to not use dictionary encoding.

func WithNoZstd

func WithNoZstd() Option

WithNoZstd sets the Producer to not use Zstd compression at the Arrow IPC level.

func WithObserver

func WithObserver(observer observer.ProducerObserver) Option

WithObserver sets the optional observer to use for the producer.

func WithOrderAttrs16By

func WithOrderAttrs16By(orderAttrs16By OrderAttrs16By) Option

WithOrderAttrs16By specifies how to order attributes in a batch (with 16bits attribute ID).

func WithOrderAttrs32By

func WithOrderAttrs32By(orderAttrs32By OrderAttrs32By) Option

WithOrderAttrs32By specifies how to order attributes in a batch (with 32bits attribute ID).

func WithOrderSpanBy

func WithOrderSpanBy(orderSpanBy OrderSpanBy) Option

WithOrderSpanBy specifies how to order spans in a batch.

func WithProducerStats

func WithProducerStats() Option

WithProducerStats enables the display of producer statistics.

func WithRecordStats

func WithRecordStats() Option

WithRecordStats enables the collection of statistics about Arrow records.

func WithSchemaStats

func WithSchemaStats() Option

WithSchemaStats enables the collection of statistics about Arrow schemas.

func WithSchemaUpdates

func WithSchemaUpdates() Option

WithSchemaUpdates enables the display of schema updates.

func WithUint16InitDictIndex

func WithUint16InitDictIndex() Option

WithUint16InitDictIndex sets the Producer to use an uint16 index for all dictionaries.

func WithUint16LimitDictIndex

func WithUint16LimitDictIndex() Option

WithUint16LimitDictIndex sets the Producer to fall back to non dictionary encoding if the dictionary size exceeds an uint16 index.

func WithUint32LimitDictIndex

func WithUint32LimitDictIndex() Option

WithUint32LimitDictIndex sets the Producer to fall back to non dictionary encoding if the dictionary size exceeds an uint32 index.

func WithUint32LinitDictIndex

func WithUint32LinitDictIndex() Option

WithUint32LinitDictIndex sets the Producer to use an uint32 index for all dictionaries.

func WithUint64InitDictIndex

func WithUint64InitDictIndex() Option

WithUint64InitDictIndex sets the Producer to use an uint64 index for all dictionaries.

func WithUint64LimitDictIndex

func WithUint64LimitDictIndex() Option

WithUint64LimitDictIndex sets the Producer to fall back to non dictionary encoding if the dictionary size exceeds an uint64 index.

func WithUint8InitDictIndex

func WithUint8InitDictIndex() Option

WithUint8InitDictIndex sets the Producer to use an uint8 index for all dictionaries.

func WithUint8LimitDictIndex

func WithUint8LimitDictIndex() Option

WithUint8LimitDictIndex sets the Producer to fall back to non dictionary encoding if the dictionary size exceeds an uint8 index.

func WithZstd

func WithZstd() Option

WithZstd sets the Producer to use Zstd compression at the Arrow IPC level.

type OrderAttrs16By

type OrderAttrs16By int8

const (
	OrderAttrs16ByNothing OrderAttrs16By = iota
	OrderAttrs16ByParentIdKeyValue
	OrderAttrs16ByTypeKeyParentIdValue
	OrderAttrs16ByTypeKeyValueParentId
)

Enumeration defining how to order attributes in a batch (with 16bits attribute ID).

type OrderAttrs32By

type OrderAttrs32By int8

const (
	OrderAttrs32ByNothing OrderAttrs32By = iota
	OrderAttrs32ByTypeParentIdKeyValue
	OrderAttrs32ByTypeKeyParentIdValue
	OrderAttrs32ByTypeKeyValueParentId
	OrderAttrs32ByKeyValueParentId
)

Enumeration defining how to order attributes in a batch (with 32bits attribute ID).

type OrderSpanBy

type OrderSpanBy int8

const (
	OrderSpanByNothing OrderSpanBy = iota
	OrderSpanByNameTraceID
	OrderSpanByTraceIDName
	OrderSpanByNameStartTime
	OrderSpanByNameTraceIdStartTime
	OrderSpanByStartTimeTraceIDName
	OrderSpanByStartTimeNameTraceID
)

Enumeration defining how to order spans in a batch.