chunk

Documentation

Overview

package chunk encodes timeseries in chunks of data see devdocs/chunk-format.md for more information.

Index

• Variables
• func ExtractChunkSpan(chunk []byte) uint32
• func MaxConfigurableSpan() uint32
• type Chunk
  • func New(t0 uint32) *Chunk
  • func NewFirst(t0 uint32) *Chunk
  • func (c *Chunk) Encode(span uint32) []byte
  • func (c *Chunk) Finish()
  • func (c *Chunk) Push(t uint32, v float64) error
  • func (c *Chunk) String() string
• type Format
  • func (i Format) String() string
• type IterGen
  • func NewIterGen(t0, intervalHint uint32, b []byte) (IterGen, error)
  • func (z *IterGen) DecodeMsg(dc *msgp.Reader) (err error)
  • func (z *IterGen) EncodeMsg(en *msgp.Writer) (err error)
  • func (ig IterGen) EndTs() uint32
  • func (ig IterGen) Format() Format
  • func (ig *IterGen) Get() (tsz.Iter, error)
  • func (z *IterGen) MarshalMsg(b []byte) (o []byte, err error)
  • func (z *IterGen) Msgsize() (s int)
  • func (ig *IterGen) Size() uint64
  • func (ig *IterGen) Span() uint32
  • func (z *IterGen) UnmarshalMsg(bts []byte) (o []byte, err error)
• type IterGensAsc
  • func (a IterGensAsc) Len() int
  • func (a IterGensAsc) Less(i, j int) bool
  • func (a IterGensAsc) Swap(i, j int)
• type SpanCode

Variables

var ChunkSpans = [32]uint32{
	1,
	5,
	10,
	15,
	20,
	30,
	60,
	90,
	2 * 60,
	3 * 60,
	5 * 60,
	10 * 60,
	15 * 60,
	20 * 60,
	30 * 60,
	45 * 60,
	3600,
	90 * 60,
	2 * 3600,
	150 * 60,
	3 * 3600,
	4 * 3600,
	5 * 3600,
	6 * 3600,
	7 * 3600,
	8 * 3600,
	9 * 3600,
	10 * 3600,
	12 * 3600,
	15 * 3600,
	18 * 3600,
	24 * 3600,
}

var RevChunkSpans = make(map[uint32]SpanCode, len(ChunkSpans))

Functions

func ExtractChunkSpan

func ExtractChunkSpan(chunk []byte) uint32

SpanOfChunk takes a chunk and tries to determine its span. It returns 0 if it failed to determine the span, this could fail either because the given chunk is invalid or because it has an old format

func MaxConfigurableSpan

func MaxConfigurableSpan() uint32

MaxConfigurableSpan returns the largest configurable span.

Types

type Chunk

type Chunk struct {
	Series    tsz.SeriesLong
	NumPoints uint32
	First     bool
}

Chunk is a chunk of data. not concurrency safe. last check that the methods are being called safely by Dieter on 20/11/2018 checked: String, Push, Finish, Encode and properties Series, NumPoints, First for the most part, confirming serialized access is easy by tracking all callers/references. The main exception is the ChunkWriteRequest mechanism. any CWR created is processed asynchronously, and chunk properties will be read (the series.T0, chunk.Encode(), etc) But it can be proven that we only call NewChunkWriteRequest() on chunks that are no longer being modified.

func New

func New(t0 uint32) *Chunk

func NewFirst

func NewFirst(t0 uint32) *Chunk

func (*Chunk) Encode

func (c *Chunk) Encode(span uint32) []byte

Encode encodes the chunk note: chunks don't know their own span, the caller/owner manages that, so for formats that encode it, it needs to be passed in. the returned value contains no references to the chunk. data is copied.

func (*Chunk) Finish

func (c *Chunk) Finish()

func (*Chunk) Push

func (c *Chunk) Push(t uint32, v float64) error

func (*Chunk) String

func (c *Chunk) String() string

type Format

type Format uint8

const (
	FormatStandardGoTsz Format = iota
	FormatStandardGoTszWithSpan
	FormatGoTszLongWithSpan // like FormatStandardGoTszWithSpan but using tsz.SeriesLong
)

identifier of message format

func (Format) String

func (i Format) String() string

type IterGen

type IterGen struct {
	T0           uint32
	IntervalHint uint32 // a hint wrt expected alignment of points. useful to recover delta overflows in tsz.Series4h, not used for other formats
	B            []byte
}

func NewIterGen

func NewIterGen(t0, intervalHint uint32, b []byte) (IterGen, error)

NewIterGen creates an IterGen and performs crude validation of the data note: it's ok for intervalHint to be 0 or 1 to mean unknown. it just means that series4h corruptions can't be remediated in single-point-per-chunk scenarios

func (*IterGen) DecodeMsg

func (z *IterGen) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (*IterGen) EncodeMsg

func (z *IterGen) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (IterGen) EndTs

func (ig IterGen) EndTs() uint32

end of itergen (exclusive). next t0

func (IterGen) Format

func (ig IterGen) Format() Format

func (*IterGen) Get

func (ig *IterGen) Get() (tsz.Iter, error)

func (*IterGen) MarshalMsg

func (z *IterGen) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (*IterGen) Msgsize

func (z *IterGen) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (*IterGen) Size

func (ig *IterGen) Size() uint64

func (*IterGen) Span

func (ig *IterGen) Span() uint32

func (*IterGen) UnmarshalMsg

func (z *IterGen) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler

type IterGensAsc

type IterGensAsc []IterGen

func (IterGensAsc) Len

func (a IterGensAsc) Len() int

func (IterGensAsc) Less

func (a IterGensAsc) Less(i, j int) bool

func (IterGensAsc) Swap

func (a IterGensAsc) Swap(i, j int)

type SpanCode

type SpanCode uint8