costhistory

Documentation

Index

• func SamplesToProto(in []Sample) []*roxy_v0.Sample
• type CostHistory
  • func (hist *CostHistory) BulkAppend(samples ...Sample)
  • func (hist *CostHistory) Data() Data
  • func (hist *CostHistory) Debug(w io.Writer)
  • func (hist *CostHistory) Init()
  • func (hist *CostHistory) Now() time.Time
  • func (hist *CostHistory) Reset()
  • func (hist *CostHistory) Snapshot() []Sample
  • func (hist *CostHistory) Update()
  • func (hist *CostHistory) UpdateAbsolute(counter uint64)
  • func (hist *CostHistory) UpdateRelative(counterDelta uint64)
  • func (hist *CostHistory) WindowInterval() time.Duration
• type Data
• type Packed
  • func PackSamples(epoch time.Time, in []Sample) []Packed
  • func (packed Packed) Unpack(epoch time.Time) Sample
• type Sample
  • func SampleFromProto(pb *roxy_v0.Sample) Sample
  • func SamplesFromProto(in []*roxy_v0.Sample) []Sample
  • func UnpackSamples(epoch time.Time, in []Packed) []Sample
  • func (sample Sample) Pack(epoch time.Time) Packed
  • func (sample Sample) ToProto() *roxy_v0.Sample

Functions

func SamplesToProto

func SamplesToProto(in []Sample) []*roxy_v0.Sample

SamplesToProto converts []Sample to []*roxy_v0.Sample.

Types

type CostHistory

type CostHistory struct {
	NumBuckets     uint
	BucketInterval time.Duration
	DecayFactor    float64
	NowFn          func() time.Time
	Mutex          sync.Locker
	// contains filtered or unexported fields
}

CostHistory records the history of a cost-per-second (CPS) metric, where CPS is defined as queries-per-second (QPS) times a cost factor per query.

It is designed to compute the "average" CPS over a window of time. The detailed algorithm is subject to change, but currently: "average" is defined by bucketing samples by age and computing an exponentially-decaying weighted average of the pairwise bucket-by-bucket QPS, with each QPS weighted by a function of the bucket age. In simple terms, data for the most recent sample is weighted most heavily, data from the oldest known sample is weighted least heavily, and data from before the start of the window is forgotten entirely.

CostHistory is thread-safe.

func (*CostHistory) BulkAppend

func (hist *CostHistory) BulkAppend(samples ...Sample)

BulkAppend inserts the given samples into the time window, recomputing the average CPS. In addition to being more efficient than calling UpdateRelative or UpdateAbsolute repeatedly, it also allows the caller to specify the collection time of each sample.

func (*CostHistory) Data

func (hist *CostHistory) Data() Data

Data obtains a Data snapshot.

func (*CostHistory) Debug

func (hist *CostHistory) Debug(w io.Writer)

Debug writes a debugging dump to the provided io.Writer. No stability guarantees are provided for the output format.

func (*CostHistory) Init

func (hist *CostHistory) Init()

Init initializes a CostHistory.

func (*CostHistory) Now

func (hist *CostHistory) Now() time.Time

Now returns the current time in UTC.

func (*CostHistory) Reset

func (hist *CostHistory) Reset()

Reset resets the CostHistory to cost counter 0 and no sample data.

func (*CostHistory) Snapshot

func (hist *CostHistory) Snapshot() []Sample

Snapshot obtains a copy of the detailed contents of the CostHistory's sample time window. The first Sample represents the time and absolute value of the cost counter at the start of the window, and each subsequent Sample records (a) when the counter changed and (b) what value it changed to. The current cost counter is the value of Sample.Counter for the last Sample.

func (*CostHistory) Update

func (hist *CostHistory) Update()

Update updates the sample window and recomputes the average without adding a new sample.

func (*CostHistory) UpdateAbsolute

func (hist *CostHistory) UpdateAbsolute(counter uint64)

UpdateAbsolute creates a new sample with the given (larger) counter value, in addition to updating the sample window and recomputing the average.

If the new cost counter is less than the previous cost counter, it's assumed that a reset-to-zero has occurred, and the previous cost counter will be added to this and all future absolute cost counter values.

func (*CostHistory) UpdateRelative

func (hist *CostHistory) UpdateRelative(counterDelta uint64)

UpdateRelative creates a new sample with the given (non-negative) cost delta added to the current cost counter, in addition to updating the sample window and recomputing the average.

func (*CostHistory) WindowInterval

func (hist *CostHistory) WindowInterval() time.Duration

WindowInterval returns the calculated width of the sample window.

type Data

type Data struct {
	Now       time.Time
	Counter   uint64
	PerSecond float64
}

Data represents a snapshot of both the absolute cost counter and the average rate of growth in CPS.

type Packed

type Packed struct {
	Delta   time.Duration
	Counter uint64
}

Packed represents one data point measuring the cost counter's value over time. The cost counter is a monotonically increasing value which records the sum of all costs incurred up to a moment in time.

Unlike Sample, time is recorded as a delta relative to an epoch.

~~ RATIONALE ~~

Go 1.9 (2017) introduced monotonic time support, which allows t1.Sub(t2) to return a meaningful value even when the wallclock time has changed between t1 and t2, assuming both t1 and t2 were obtained with time.Now(). Other Time methods, such as Before and After, do *not* respect monotonic time; they look at the wallclock time *only*. As we are going to be doing a lot of Before/After-like comparisons between timestamps, it's best for us convert all Time values to the same time base to better deal with leap seconds, NTP time adjustments, and other such complications with using wallclock time.

We do this by defining an arbitrary Time as our epoch, obtaining it via time.Now() and converting all other Time values into Durations relative to it, and then doing all operations on Durations instead of on Times. We only convert back to Times when exporting data to another node, as monotonic times cannot be exported from the machine on which they live: monotonic times are measured as seconds elapsed since boot in most OSes, which means each node's monotonic counter has a different epoch, and wallclock time is the only way for nodes to interoperate with consistency.

This scheme has a minor bonus benefit: Durations are significantly smaller than Times, because Times maintain two 64-bit timestamps and a 32/64-bit timezone pointer while Durations are a single 64-bit value. This helps a bit when dealing with lots of samples, since it means that sizeof(Packed) is about half of sizeof(Sample).

References:

- https://golang.org/doc/go1.9#monotonic-time

- https://go.googlesource.com/proposal/+/master/design/12914-monotonic.md

func PackSamples

func PackSamples(epoch time.Time, in []Sample) []Packed

PackSamples converts []Sample to []Packed.

func (Packed) Unpack

func (packed Packed) Unpack(epoch time.Time) Sample

Unpack converts Packed to Sample.

type Sample

type Sample struct {
	Time    time.Time
	Counter uint64
}

Sample represents one data point measuring the cost counter's value over time. The cost counter is a monotonically increasing value which records the sum of all costs incurred up to a moment in time.

func SampleFromProto

func SampleFromProto(pb *roxy_v0.Sample) Sample

SampleFromProto converts roxy_v0.Sample to Sample.

func SamplesFromProto

func SamplesFromProto(in []*roxy_v0.Sample) []Sample

SamplesFromProto converts []*roxy_v0.Sample to []Sample.

func UnpackSamples

func UnpackSamples(epoch time.Time, in []Packed) []Sample

UnpackSamples converts []Packed to []Sample.

func (Sample) Pack

func (sample Sample) Pack(epoch time.Time) Packed

Pack converts Sample to Packed.

func (Sample) ToProto

func (sample Sample) ToProto() *roxy_v0.Sample

ToProto converts Sample to roxy_v0.Sample.