encoding

Documentation

Index

• Constants
• Variables
• func RangeValues(it Iterator, in metric.Interval) ([]model.SamplePair, error)
• type Batch
• type Chunk
  • func NewForEncoding(encoding Encoding) (Chunk, error)
• type Encoding
  • func (e Encoding) String() string
• type Iterator

Constants

const (

	// OpTypeLabel is the label name for chunk operation types.
	OpTypeLabel = "type"

	// CreateAndPin is the label value for create-and-pin chunk ops.
	CreateAndPin = "create" // A Desc creation with refCount=1.
	// PersistAndUnpin is the label value for persist chunk ops.
	PersistAndUnpin = "persist"
	// Pin is the label value for pin chunk ops (excludes pin on creation).
	Pin = "pin"
	// Unpin is the label value for unpin chunk ops (excludes the unpin on persisting).
	Unpin = "unpin"
	// Transcode is the label value for transcode chunk ops.
	Transcode = "transcode"
	// Drop is the label value for drop chunk ops.
	Drop = "drop"

	// Evict is the label value for evict chunk desc ops.
	Evict = "evict"
	// Load is the label value for load chunk and chunk desc ops.
	Load = "load"
)

const BatchSize = 12

BatchSize is samples per batch; this was choose by benchmarking all sizes from 1 to 128.

const (
	// ChunkLen is the length of a chunk in bytes.
	ChunkLen = 1024
)

Variables

var (
	Ops = prometheus.NewCounterVec(
		prometheus.CounterOpts{
			Namespace: namespace,
			Subsystem: subsystem,
			Name:      "chunk_ops_total",
			Help:      "The total number of chunk operations by their type.",
		},
		[]string{OpTypeLabel},
	)
	DescOps = prometheus.NewCounterVec(
		prometheus.CounterOpts{
			Namespace: namespace,
			Subsystem: subsystem,
			Name:      "chunkdesc_ops_total",
			Help:      "The total number of chunk descriptor operations by their type.",
		},
		[]string{OpTypeLabel},
	)
	NumMemDescs = prometheus.NewGauge(prometheus.GaugeOpts{
		Namespace: namespace,
		Subsystem: subsystem,
		Name:      "memory_chunkdescs",
		Help:      "The current number of chunk descriptors in memory.",
	})
)

Usually, a separate file for instrumentation is frowned upon. Metrics should be close to where they are used. However, the metrics below are set all over the place, so we go for a separate instrumentation file in this case.

var (
	// DefaultEncoding exported for use in unit tests elsewhere
	DefaultEncoding = PrometheusXorChunk
)

var NumMemChunks int64

NumMemChunks is the total number of chunks in memory. This is a global counter, also used internally, so not implemented as metrics. Collected in MemorySeriesStorage. TODO(beorn7): Having this as an exported global variable is really bad.

Functions

func RangeValues

func RangeValues(it Iterator, in metric.Interval) ([]model.SamplePair, error)

RangeValues is a utility function that retrieves all values within the given range from an Iterator.

Types

type Batch

type Batch struct {
	Timestamps [BatchSize]int64
	Values     [BatchSize]float64
	Index      int
	Length     int
}

Batch is a sorted set of (timestamp, value) pairs. They are intended to be small, and passed by value.

type Chunk

type Chunk interface {
	// Add adds a SamplePair to the chunks, performs any necessary
	// re-encoding, and creates any necessary overflow chunk.
	// The returned Chunk is the overflow chunk if it was created.
	// The returned Chunk is nil if the sample got appended to the same chunk.
	Add(sample model.SamplePair) (Chunk, error)
	// NewIterator returns an iterator for the chunks.
	// The iterator passed as argument is for re-use. Depending on implementation,
	// the iterator can be re-used or a new iterator can be allocated.
	NewIterator(Iterator) Iterator
	Marshal(io.Writer) error
	UnmarshalFromBuf([]byte) error
	Encoding() Encoding

	// Len returns the number of samples in the chunk.  Implementations may be
	// expensive.
	Len() int

	// Equals checks if this chunk holds the same data as another.
	Equals(Chunk) (bool, error)
}

Chunk is the interface for all chunks. Chunks are generally not goroutine-safe.

func NewForEncoding

func NewForEncoding(encoding Encoding) (Chunk, error)

NewForEncoding allows configuring what chunk type you want

type Encoding

type Encoding byte

Encoding defines which encoding we are using, delta, doubledelta, or varbit

const (
	// PrometheusXorChunk is a wrapper around Prometheus XOR-encoded chunk.
	// 4 is the magic value for backwards-compatibility with previous iota-based constants.
	PrometheusXorChunk Encoding = 4
)

func (Encoding) String

func (e Encoding) String() string

String implements flag.Value.

type Iterator

type Iterator interface {
	// Scans the next value in the chunk. Directly after the iterator has
	// been created, the next value is the first value in the
	// chunk. Otherwise, it is the value following the last value scanned or
	// found (by one of the Find... methods). Returns false if either the
	// end of the chunk is reached or an error has occurred.
	Scan() bool
	// Finds the oldest value at or after the provided time. Returns false
	// if either the chunk contains no value at or after the provided time,
	// or an error has occurred.
	FindAtOrAfter(model.Time) bool
	// Returns the last value scanned (by the scan method) or found (by one
	// of the find... methods). It returns model.ZeroSamplePair before any of
	// those methods were called.
	Value() model.SamplePair
	// Returns a batch of the provisded size; NB not idempotent!  Should only be called
	// once per Scan.
	Batch(size int) Batch
	// Returns the last error encountered. In general, an error signals data
	// corruption in the chunk and requires quarantining.
	Err() error
}

Iterator enables efficient access to the content of a chunk. It is generally not safe to use an Iterator concurrently with or after chunk mutation.