README
¶
circonusllhist
A golang implementation of the OpenHistogram libcircllhist library.
Overview
Package circllhist provides an implementation of OpenHistogram's fixed log-linear histogram data structure. This allows tracking of histograms in a composable way such that accurate error can be reasoned about.
License
Documentation
More complete docs can be found at godoc or pkg.go.dev
Usage Example
package main
import (
"fmt"
"github.com/openhistogram/circonusllhist"
)
func main() {
//Create a new histogram
h := circonusllhist.New()
//Insert value 123, three times
if err := h.RecordValues(123, 3); err != nil {
panic(err)
}
//Insert 1x10^1
if err := h.RecordIntScale(1, 1); err != nil {
panic(err)
}
//Print the count of samples stored in the histogram
fmt.Printf("%d\n", h.Count())
//Print the sum of all samples
fmt.Printf("%f\n", h.ApproxSum())
}
Usage Without Lookup Tables
By default, bi-level sparse lookup tables are used in this OpenHistogram implementation to improve insertion time by about 20%. However, the size of these tables ranges from a minimum of ~0.5KiB to a maximum of ~130KiB. While usage nearing the theoretical maximum is unlikely, as the lookup tables are kept as sparse tables, normal usage will be above the minimum. For applications where insertion time is not the most important factor and memory efficiency is, especially when datasets contain large numbers of individual histograms, opting out of the lookup tables is an appropriate choice. Generate new histograms without lookup tables like:
package main
import "github.com/openhistogram/circonusllhist"
func main() {
//Create a new histogram without lookup tables
h := circonusllhist.New(circonusllhist.NoLookup())
// ...
}
Notes on Serialization
When intentionally working without lookup tables, care must be taken to correctly serialize and deserialize the histogram data. The following example creates a histogram without lookup tables, serializes and deserializes it manually while never allocating any excess memory:
package main
import (
"bytes"
"fmt"
"github.com/openhistogram/circonusllhist"
)
func main() {
// create a new histogram without lookup tables
h := circonusllhist.New(circonusllhist.NoLookup())
if err := h.RecordValue(1.2); err != nil {
panic(err)
}
// serialize the histogram
var buf bytes.Buffer
if err := h.Serialize(&buf); err != nil {
panic(err)
}
// deserialize into a new histogram
h2, err := circonusllhist.DeserializeWithOptions(&buf, circonusllhist.NoLookup())
if err != nil {
panic(err)
}
// the two histograms are equal
fmt.Println(h.Equals(h2))
}
While the example above works cleanly when manual (de)serialization is required, a different approach is needed when implicitly (de)serializing histograms into a JSON format. The following example creates a histogram without lookup tables, serializes and deserializes it implicitly using Go's JSON library, ensuring no excess memory allocations occur:
package main
import (
"encoding/json"
"fmt"
"github.com/openhistogram/circonusllhist"
)
func main() {
// create a new histogram without lookup tables
h := circonusllhist.New(circonusllhist.NoLookup())
if err := h.RecordValue(1.2); err != nil {
panic(err)
}
// serialize the histogram
data, err := json.Marshal(h)
if err != nil {
panic(err)
}
// deserialize into a new histogram
var wrapper2 circonusllhist.HistogramWithoutLookups
if err := json.Unmarshal(data, &wrapper2); err != nil {
panic(err)
}
h2 := wrapper2.Histogram()
// the two histograms are equal
fmt.Println(h.Equals(h2))
}
Once the circonusllhist.HistogramWithoutLookups wrapper has been used as a deserialization target, the underlying histogram may be extracted with the Histogram() method. It is also possible to extract the histogram while allocating memory for lookup tables if necessary with the HistogramWithLookups() method.
Documentation
¶
Overview ¶
Package circllhist provides an implementation of Circonus' fixed log-linear histogram data structure. This allows tracking of histograms in a composable way such that accurate error can be reasoned about.
Index ¶
- Constants
- func MarshalJSON(h *Histogram) ([]byte, error)
- func UnmarshalJSONWithOptions(parent *Histogram, b []byte, options ...Option) error
- type Histogram
- func (h *Histogram) ApproxMean() float64
- func (h *Histogram) ApproxQuantile(qIn []float64) ([]float64, error)
- func (h *Histogram) ApproxSum() float64
- func (h *Histogram) BinCount() uint64
- func (h *Histogram) Copy() *Histogram
- func (h *Histogram) CopyAndReset() *Histogram
- func (h *Histogram) Count() uint64
- func (h *Histogram) DecStrings() []string
- func (h *Histogram) Equals(other *Histogram) bool
- func (h *Histogram) FullReset()
- func (h *Histogram) MarshalJSON() ([]byte, error)
- func (h *Histogram) Max() float64
- func (h *Histogram) Mean() float64
- func (h *Histogram) Merge(o *Histogram)
- func (h *Histogram) Min() float64
- func (h *Histogram) RecordCorrectedValue(v, expectedInterval int64) error
- func (h *Histogram) RecordDuration(v time.Duration) error
- func (h *Histogram) RecordIntScale(val int64, scale int) error
- func (h *Histogram) RecordIntScales(val int64, scale int, n int64) error
- func (h *Histogram) RecordValue(v float64) error
- func (h *Histogram) RecordValues(v float64, n int64) error
- func (h *Histogram) Reset()
- func (h *Histogram) Serialize(w io.Writer) error
- func (h *Histogram) SerializeB64(w io.Writer) error
- func (h *Histogram) SignificantFigures() int64
- func (h *Histogram) UnmarshalJSON(b []byte) error
- func (h *Histogram) ValueAtQuantile(q float64) float64
- type HistogramWithoutLookups
- type Option
- type Options
Constants ¶
const (
BVL1, BVL1MASK uint64 = iota, 0xff << (8 * iota)
BVL2, BVL2MASK
BVL3, BVL3MASK
BVL4, BVL4MASK
BVL5, BVL5MASK
BVL6, BVL6MASK
BVL7, BVL7MASK
BVL8, BVL8MASK
)
Variables ¶
This section is empty.
Functions ¶
func MarshalJSON ¶ added in v0.4.0
Types ¶
type Histogram ¶
type Histogram struct {
// contains filtered or unexported fields
}
Histogram tracks values are two decimal digits of precision with a bounded error that remains bounded upon composition.
func DeserializeWithOptions ¶ added in v0.4.0
func NewFromStrings ¶
NewFromStrings returns a Histogram created from DecStrings strings.
func NewNoLocks ¶
func NewNoLocks() *Histogram
NewNoLocks returns a new histogram not using locks. Deprecated: use New(NoLocks()) instead.
func (*Histogram) ApproxMean ¶
ApproxMean returns an approximate mean.
func (*Histogram) ApproxQuantile ¶
func (*Histogram) CopyAndReset ¶
CopyAndReset creates and returns an exact copy of a histogram, and resets it to default empty values.
func (*Histogram) DecStrings ¶
func (*Histogram) FullReset ¶ added in v0.1.3
func (h *Histogram) FullReset()
FullReset resets a histogram to default empty values.
func (*Histogram) MarshalJSON ¶
func (*Histogram) RecordCorrectedValue ¶
RecordCorrectedValue records the given value, correcting for stalls in the recording process. This only works for processes which are recording values at an expected interval (e.g., doing jitter analysis). Processes which are recording ad-hoc values (e.g., latency for incoming requests) can't take advantage of this. CH Compat.
func (*Histogram) RecordDuration ¶ added in v0.1.2
RecordDuration records the given time.Duration in seconds, returning an error if the value is out of range.
func (*Histogram) RecordIntScale ¶
RecordIntScale records an integer scaler value, returning an error if the value is out of range.
func (*Histogram) RecordIntScales ¶
RecordIntScales records n occurrences of the given value, returning an error if the value is out of range.
func (*Histogram) RecordValue ¶
RecordValue records the given value, returning an error if the value is out of range.
func (*Histogram) RecordValues ¶
RecordValues records n occurrences of the given value, returning an error if the value is out of range.
func (*Histogram) Reset ¶
func (h *Histogram) Reset()
Reset forgets all bins in the histogram (they remain allocated).
func (*Histogram) SignificantFigures ¶
SignificantFigures returns the significant figures used to create the histogram CH Compat.
func (*Histogram) UnmarshalJSON ¶
UnmarshalJSON - histogram will come in a base64 encoded serialized form.
func (*Histogram) ValueAtQuantile ¶
ValueAtQuantile returns the recorded value at the given quantile (0..1).
type HistogramWithoutLookups ¶ added in v0.4.0
type HistogramWithoutLookups struct {
// contains filtered or unexported fields
}
HistogramWithoutLookups holds a Histogram that's not configured to use a lookup table. This type is useful to round-trip serialize the underlying data while never allocating memory for the lookup table. The main Histogram type must use lookups by default to be compatible with the circllhist implementation of other languages. Furthermore, it is not possible to encode the lookup table preference into the serialized form, as that's again defined across languages. Therefore, the most straightforward manner by which a user can deserialize histogram data while not allocating lookup tables is by using a dedicated type in their structures describing on-disk forms. This structure can divulge the underlying Histogram, optionally allocating the lookup tables first.
func NewHistogramWithoutLookups ¶ added in v0.4.0
func NewHistogramWithoutLookups(histogram *Histogram) *HistogramWithoutLookups
NewHistogramWithoutLookups creates a new container for a Histogram without lookup tables.
func (*HistogramWithoutLookups) Histogram ¶ added in v0.4.0
func (h *HistogramWithoutLookups) Histogram() *Histogram
Histogram divulges the underlying Histogram that was deserialized. This Histogram will not have lookup tables allocated.
func (*HistogramWithoutLookups) HistogramWithLookups ¶ added in v0.4.0
func (h *HistogramWithoutLookups) HistogramWithLookups() *Histogram
HistogramWithLookups allocates lookup tables in the underlying Histogram that was deserialized, then divulges it.
func (*HistogramWithoutLookups) MarshalJSON ¶ added in v0.4.0
func (h *HistogramWithoutLookups) MarshalJSON() ([]byte, error)
MarshalJSON marshals a histogram to a base64 encoded serialized form.
func (*HistogramWithoutLookups) UnmarshalJSON ¶ added in v0.4.0
func (h *HistogramWithoutLookups) UnmarshalJSON(b []byte) error
UnmarshalJSON unmarshals a histogram from a base64 encoded serialized form.
type Option ¶ added in v0.4.0
type Option func(*Options)
Option knows how to mutate the Options to change initialization.
func NoLocks ¶ added in v0.4.0
func NoLocks() Option
NoLocks configures a histogram to not use locks.
func NoLookup ¶ added in v0.4.0
func NoLookup() Option
NoLookup configures a histogram to not use a lookup table for bins. This is an appropriate option to use when the data set being operated over contains a large number of individual histograms and the insert speed into any histogram is not of the utmost importance. This option reduces the baseline memory consumption of one Histogram by at least 0.5kB and up to 130kB while increasing the insertion time by ~20%.
type Options ¶ added in v0.4.0
type Options struct {
// Size is the number of bins.
Size uint16
// UseLocks determines if the histogram should use locks
UseLocks bool
// UseLookup determines if the histogram should use a lookup table for bins
UseLookup bool
}
Options are exposed options for initializing a histogram.
Source Files