chunkenc

Documentation

Index

• Constants
• func IsOutOfOrderChunk(e Encoding) bool
• func IsValidEncoding(e Encoding) bool
• type Appender
• type Chunk
  • func FromData(e Encoding, d []byte) (Chunk, error)
  • func NewEmptyChunk(e Encoding) (Chunk, error)
• type CounterResetHeader
• type Encoding
  • func (e Encoding) String() string
• type HistogramAppender
  • func (a *HistogramAppender) Append(int64, float64)
  • func (a *HistogramAppender) AppendHistogram(t int64, h *histogram.Histogram)
  • func (a *HistogramAppender) Appendable(h *histogram.Histogram) (positiveInterjections, negativeInterjections []Interjection, ...)
  • func (a *HistogramAppender) Recode(positiveInterjections, negativeInterjections []Interjection, ...) (Chunk, Appender)
• type HistogramChunk
  • func NewHistogramChunk() *HistogramChunk
  • func (c *HistogramChunk) Appender() (Appender, error)
  • func (c *HistogramChunk) Bytes() []byte
  • func (c *HistogramChunk) Compact()
  • func (c *HistogramChunk) Encoding() Encoding
  • func (c *HistogramChunk) GetCounterResetHeader() CounterResetHeader
  • func (c *HistogramChunk) Iterator(it Iterator) Iterator
  • func (c *HistogramChunk) Layout() (schema int32, zeroThreshold float64, ...)
  • func (c *HistogramChunk) NumSamples() int
  • func (c *HistogramChunk) SetCounterResetHeader(h CounterResetHeader)
• type Interjection
• type Iterator
  • func MockSeriesIterator(timestamps []int64, values []float64) Iterator
  • func NewNopIterator() Iterator
• type OOOXORChunk
  • func (c *OOOXORChunk) Encoding() Encoding
• type Pool
  • func NewPool() Pool
• type ValueType
  • func (v ValueType) ChunkEncoding() Encoding
  • func (v ValueType) String() string
• type XORChunk
  • func NewXORChunk() *XORChunk
  • func (c *XORChunk) Appender() (Appender, error)
  • func (c *XORChunk) Bytes() []byte
  • func (c *XORChunk) Compact()
  • func (c *XORChunk) Encoding() Encoding
  • func (c *XORChunk) Iterator(it Iterator) Iterator
  • func (c *XORChunk) NumSamples() int

Constants

const (
	OutOfOrderMask = 0b10000000
	EncOOOXOR      = EncXOR | OutOfOrderMask
)

Chunk encodings for out-of-order chunks. These encodings must be only used by the Head block for its internal bookkeeping.

Functions

func IsOutOfOrderChunk

func IsOutOfOrderChunk(e Encoding) bool

func IsValidEncoding

func IsValidEncoding(e Encoding) bool

IsValidEncoding returns true for supported encodings.

Types

type Appender

type Appender interface {
	Append(int64, float64)
	AppendHistogram(t int64, h *histogram.Histogram)
}

Appender adds sample pairs to a chunk.

type Chunk

type Chunk interface {
	// Bytes returns the underlying byte slice of the chunk.
	Bytes() []byte

	// Encoding returns the encoding type of the chunk.
	Encoding() Encoding

	// Appender returns an appender to append samples to the chunk.
	Appender() (Appender, error)

	// The iterator passed as argument is for re-use.
	// Depending on implementation, the iterator can
	// be re-used or a new iterator can be allocated.
	Iterator(Iterator) Iterator

	// NumSamples returns the number of samples in the chunk.
	NumSamples() int

	// Compact is called whenever a chunk is expected to be complete (no more
	// samples appended) and the underlying implementation can eventually
	// optimize the chunk.
	// There's no strong guarantee that no samples will be appended once
	// Compact() is called. Implementing this function is optional.
	Compact()
}

Chunk holds a sequence of sample pairs that can be iterated over and appended to.

func FromData

func FromData(e Encoding, d []byte) (Chunk, error)

FromData returns a chunk from a byte slice of chunk data. This is there so that users of the library can easily create chunks from bytes.

func NewEmptyChunk

func NewEmptyChunk(e Encoding) (Chunk, error)

NewEmptyChunk returns an empty chunk for the given encoding.

type CounterResetHeader

type CounterResetHeader byte

CounterResetHeader defines the first 2 bits of the chunk header.

const (
	// CounterReset means there was definitely a counter reset that resulted in this chunk.
	CounterReset CounterResetHeader = 0b10000000
	// NotCounterReset means there was definitely no counter reset when cutting this chunk.
	NotCounterReset CounterResetHeader = 0b01000000
	// GaugeType means this chunk contains a gauge histogram, where counter resets do not happen.
	GaugeType CounterResetHeader = 0b11000000
	// UnknownCounterReset means we cannot say if this chunk was created due to a counter reset or not.
	// An explicit counter reset detection needs to happen during query time.
	UnknownCounterReset CounterResetHeader = 0b00000000
)

type Encoding

type Encoding uint8

Encoding is the identifier for a chunk encoding.

const (
	EncNone Encoding = iota
	EncXOR
	EncHistogram
)

The different available chunk encodings.

func (Encoding) String

func (e Encoding) String() string

type HistogramAppender

type HistogramAppender struct {
	// contains filtered or unexported fields
}

HistogramAppender is an Appender implementation for sparse histograms.

func (*HistogramAppender) Append

func (a *HistogramAppender) Append(int64, float64)

Append implements Appender. This implementation panics because normal float samples must never be appended to a histogram chunk.

func (*HistogramAppender) AppendHistogram

func (a *HistogramAppender) AppendHistogram(t int64, h *histogram.Histogram)

AppendHistogram appends a histogram to the chunk. The caller must ensure that the histogram is properly structured, e.g. the number of buckets used corresponds to the number conveyed by the span structures. First call Appendable() and act accordingly!

func (*HistogramAppender) Appendable

func (a *HistogramAppender) Appendable(h *histogram.Histogram) (
	positiveInterjections, negativeInterjections []Interjection,
	okToAppend, counterReset bool,
)

Appendable returns whether the chunk can be appended to, and if so whether any recoding needs to happen using the provided interjections (in case of any new buckets, positive or negative range, respectively).

The chunk is not appendable in the following cases:

• The schema has changed.

• The threshold for the zero bucket has changed.

• Any buckets have disappeared.

• There was a counter reset in the count of observations or in any bucket, including the zero bucket.

• The last sample in the chunk was stale while the current sample is not stale.

The method returns an additional boolean set to true if it is not appendable because of a counter reset. If the given sample is stale, it is always ok to append. If counterReset is true, okToAppend is always false.

func (*HistogramAppender) Recode

func (a *HistogramAppender) Recode(
	positiveInterjections, negativeInterjections []Interjection,
	positiveSpans, negativeSpans []histogram.Span,
) (Chunk, Appender)

Recode converts the current chunk to accommodate an expansion of the set of (positive and/or negative) buckets used, according to the provided interjections, resulting in the honoring of the provided new positive and negative spans. To continue appending, use the returned Appender rather than the receiver of this method.

type HistogramChunk

type HistogramChunk struct {
	// contains filtered or unexported fields
}

HistogramChunk holds encoded sample data for a sparse, high-resolution histogram.

Each sample has multiple "fields", stored in the following way (raw = store number directly, delta = store delta to the previous number, dod = store delta of the delta to the previous number, xor = what we do for regular sample values):

field →    ts    count zeroCount sum []posbuckets []negbuckets
sample 1   raw   raw   raw       raw []raw        []raw
sample 2   delta delta delta     xor []delta      []delta
sample >2  dod   dod   dod       xor []dod        []dod

func NewHistogramChunk

func NewHistogramChunk() *HistogramChunk

NewHistogramChunk returns a new chunk with histogram encoding of the given size.

func (*HistogramChunk) Appender

func (c *HistogramChunk) Appender() (Appender, error)

Appender implements the Chunk interface.

func (*HistogramChunk) Bytes

func (c *HistogramChunk) Bytes() []byte

Bytes returns the underlying byte slice of the chunk.

func (*HistogramChunk) Compact

func (c *HistogramChunk) Compact()

Compact implements the Chunk interface.

func (*HistogramChunk) Encoding

func (c *HistogramChunk) Encoding() Encoding

Encoding returns the encoding type.

func (*HistogramChunk) GetCounterResetHeader

func (c *HistogramChunk) GetCounterResetHeader() CounterResetHeader

GetCounterResetHeader returns the info about the first 2 bits of the chunk header.

func (*HistogramChunk) Iterator

func (c *HistogramChunk) Iterator(it Iterator) Iterator

Iterator implements the Chunk interface.

func (*HistogramChunk) Layout

func (c *HistogramChunk) Layout() (
	schema int32, zeroThreshold float64,
	negativeSpans, positiveSpans []histogram.Span,
	err error,
)

Layout returns the histogram layout. Only call this on chunks that have at least one sample.

func (*HistogramChunk) NumSamples

func (c *HistogramChunk) NumSamples() int

NumSamples returns the number of samples in the chunk.

func (*HistogramChunk) SetCounterResetHeader

func (c *HistogramChunk) SetCounterResetHeader(h CounterResetHeader)

SetCounterResetHeader sets the counter reset header.

type Interjection

type Interjection struct {
	// contains filtered or unexported fields
}

An Interjection describes how many new buckets have to be introduced before processing the pos'th delta from the original slice.

type Iterator

type Iterator interface {
	// Next advances the iterator by one and returns the type of the value
	// at the new position (or ValNone if the iterator is exhausted).
	Next() ValueType
	// Seek advances the iterator forward to the first sample with a
	// timestamp equal or greater than t. If the current sample found by a
	// previous `Next` or `Seek` operation already has this property, Seek
	// has no effect. If a sample has been found, Seek returns the type of
	// its value. Otherwise, it returns ValNone, after with the iterator is
	// exhausted.
	Seek(t int64) ValueType
	// At returns the current timestamp/value pair if the value is a float.
	// Before the iterator has advanced, the behaviour is unspecified.
	At() (int64, float64)
	// AtHistogram returns the current timestamp/value pair if the value is
	// a histogram with integer counts. Before the iterator has advanced,
	// the behaviour is unspecified.
	AtHistogram() (int64, *histogram.Histogram)
	// AtFloatHistogram returns the current timestamp/value pair if the
	// value is a histogram with floating-point counts. It also works if the
	// value is a histogram with integer counts, in which case a
	// FloatHistogram copy of the histogram is returned. Before the iterator
	// has advanced, the behaviour is unspecified.
	AtFloatHistogram() (int64, *histogram.FloatHistogram)
	// AtT returns the current timestamp.
	// Before the iterator has advanced, the behaviour is unspecified.
	AtT() int64
	// Err returns the current error. It should be used only after the
	// iterator is exhausted, i.e. `Next` or `Seek` have returned ValNone.
	Err() error
}

Iterator is a simple iterator that can only get the next value. Iterator iterates over the samples of a time series, in timestamp-increasing order.

func MockSeriesIterator

func MockSeriesIterator(timestamps []int64, values []float64) Iterator

MockSeriesIterator returns an iterator for a mock series with custom timeStamps and values.

func NewNopIterator

func NewNopIterator() Iterator

NewNopIterator returns a new chunk iterator that does not hold any data.

type OOOXORChunk

type OOOXORChunk struct {
	*XORChunk
}

OOOXORChunk holds a XORChunk and overrides the Encoding() method.

func (*OOOXORChunk) Encoding

func (c *OOOXORChunk) Encoding() Encoding

type Pool

type Pool interface {
	Put(Chunk) error
	Get(e Encoding, b []byte) (Chunk, error)
}

Pool is used to create and reuse chunk references to avoid allocations.

func NewPool

func NewPool() Pool

NewPool returns a new pool.

type ValueType

type ValueType uint8

ValueType defines the type of a value an Iterator points to.

const (
	ValNone           ValueType = iota // No value at the current position.
	ValFloat                           // A simple float, retrieved with At.
	ValHistogram                       // A histogram, retrieve with AtHistogram, but AtFloatHistogram works, too.
	ValFloatHistogram                  // A floating-point histogram, retrieve with AtFloatHistogram.
)

Possible values for ValueType.

func (ValueType) ChunkEncoding

func (v ValueType) ChunkEncoding() Encoding

func (ValueType) String

func (v ValueType) String() string

type XORChunk

type XORChunk struct {
	// contains filtered or unexported fields
}

XORChunk holds XOR encoded sample data.

func NewXORChunk

func NewXORChunk() *XORChunk

NewXORChunk returns a new chunk with XOR encoding of the given size.

func (*XORChunk) Appender

func (c *XORChunk) Appender() (Appender, error)

Appender implements the Chunk interface. It is not valid to call Appender() multiple times concurrently or to use multiple Appenders on the same chunk.

func (*XORChunk) Bytes

func (c *XORChunk) Bytes() []byte

Bytes returns the underlying byte slice of the chunk.

func (*XORChunk) Compact

func (c *XORChunk) Compact()

Compact implements the Chunk interface.

func (*XORChunk) Encoding

func (c *XORChunk) Encoding() Encoding

Encoding returns the encoding type.

func (*XORChunk) Iterator

func (c *XORChunk) Iterator(it Iterator) Iterator

Iterator implements the Chunk interface. Iterator() must not be called concurrently with any modifications to the chunk, but after it returns you can use an Iterator concurrently with an Appender or other Iterators.

func (*XORChunk) NumSamples

func (c *XORChunk) NumSamples() int

NumSamples returns the number of samples in the chunk.