consolidation

Documentation

Overview

Package consolidation provides an abstraction for consolidators

Index

• func AggEvery(numPoints, maxPoints uint32) uint32
• func Consolidate(in []schema.Point, aggNum uint32, consolidator Consolidator) []schema.Point
• func ConsolidateContext(ctx context.Context, in []schema.Point, aggNum uint32, ...) []schema.Point
• func ConsolidateNudged(points []schema.Point, interval, maxDataPoints uint32, ...) ([]schema.Point, uint32)
• func GetAggFunc(consolidator Consolidator) batch.AggFunc
• func Validate(fn string) error
• type Consolidator
  • func FromArchive(archive schema.Method) Consolidator
  • func FromConsolidateBy(c string) Consolidator
  • func (c Consolidator) Archive() schema.Method
  • func (z *Consolidator) DecodeMsg(dc *msgp.Reader) (err error)
  • func (z Consolidator) EncodeMsg(en *msgp.Writer) (err error)
  • func (z Consolidator) MarshalMsg(b []byte) (o []byte, err error)
  • func (z Consolidator) Msgsize() (s int)
  • func (c Consolidator) String() string
  • func (z *Consolidator) UnmarshalMsg(bts []byte) (o []byte, err error)

Functions

func AggEvery

func AggEvery(numPoints, maxPoints uint32) uint32

returns how many points should be aggregated together so that you end up with as many points as possible, but never more than maxPoints

func Consolidate

func Consolidate(in []schema.Point, aggNum uint32, consolidator Consolidator) []schema.Point

Consolidate consolidates `in`, aggNum points at a time via the given function note: the returned slice repurposes in's backing array. it will always aggregate aggNum-sized groups of points together, with the timestamp of the last of them, and it always starts at the beginning, possibly having a point at the end that didn't incorporate as much data

func ConsolidateContext

func ConsolidateContext(ctx context.Context, in []schema.Point, aggNum uint32, consolidator Consolidator) []schema.Point

ConsolidateContext wraps a Consolidate() call with a context.Context condition

func ConsolidateNudged

func ConsolidateNudged(points []schema.Point, interval, maxDataPoints uint32, consolidator Consolidator) ([]schema.Point, uint32)

ConsolidateNudged consolidates points in a "mostly-stable" way, meaning if you run the same function again so that the input receives new points at the end and old points get removed at the beginning, we keep picking the same points to consolidate together, except for cases where there's a few points and a low MaxDataPoints. See nudgeMaybe() interval is the interval between the input points

func GetAggFunc

func GetAggFunc(consolidator Consolidator) batch.AggFunc

GetAggFunc returns a batch aggregation function or any valid consolidator, or nil otherwise

func Validate

func Validate(fn string) error

Types

type Consolidator

type Consolidator int

consolidator is a highlevel description of a point consolidation method mostly for use by the http api, but can also be used internally for data processing

const (
	None Consolidator = iota
	Avg
	Sum
	Lst
	Max
	Min
	Cnt // not available through http api
	Mult
	Med
	Diff
	StdDev
	Range
)

func FromArchive

func FromArchive(archive schema.Method) Consolidator

func FromConsolidateBy

func FromConsolidateBy(c string) Consolidator

FromConsolidateBy returns a consolidator based on a "consolidateBy" string for any string allowed by Validate(), or None otherwise. note that we also recognize "lst" here whereas Validate() doesn't, which does not violate the above statement.

func (Consolidator) Archive

func (c Consolidator) Archive() schema.Method

provide the name of a stored archive see aggregator.go for which archives are available

func (*Consolidator) DecodeMsg

func (z *Consolidator) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (Consolidator) EncodeMsg

func (z Consolidator) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (Consolidator) MarshalMsg

func (z Consolidator) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (Consolidator) Msgsize

func (z Consolidator) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (Consolidator) String

func (c Consolidator) String() string

String provides human friendly names

func (*Consolidator) UnmarshalMsg

func (z *Consolidator) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler