ddsketch

Documentation

Index

• type DDSketch
  • func DecodeDDSketch(b []byte, storeProvider store.Provider, indexMapping mapping.IndexMapping) (*DDSketch, error)
  • func LogCollapsingHighestDenseDDSketch(relativeAccuracy float64, maxNumBins int) (*DDSketch, error)
  • func LogCollapsingLowestDenseDDSketch(relativeAccuracy float64, maxNumBins int) (*DDSketch, error)
  • func LogUnboundedDenseDDSketch(relativeAccuracy float64) (*DDSketch, error)
  • func NewDDSketch(indexMapping mapping.IndexMapping, positiveValueStore store.Store, ...) *DDSketch
  • func NewDDSketchFromStoreProvider(indexMapping mapping.IndexMapping, storeProvider store.Provider) *DDSketch
  • func NewDefaultDDSketch(relativeAccuracy float64) (*DDSketch, error)
  • func (s *DDSketch) Add(value float64) error
  • func (s *DDSketch) AddWithCount(value, count float64) error
  • func (s *DDSketch) ChangeMapping(newMapping mapping.IndexMapping, positiveStore store.Store, ...) *DDSketch
  • func (s *DDSketch) Clear()
  • func (s *DDSketch) Copy() *DDSketch
  • func (s *DDSketch) DecodeAndMergeWith(bb []byte) error
  • func (s *DDSketch) Encode(b *[]byte, omitIndexMapping bool)
  • func (s *DDSketch) ForEach(f func(value, count float64) (stop bool))
  • func (s *DDSketch) GetCount() float64
  • func (s *DDSketch) GetMaxValue() (float64, error)
  • func (s *DDSketch) GetMinValue() (float64, error)
  • func (s *DDSketch) GetSum() (sum float64)
  • func (s *DDSketch) GetValueAtQuantile(quantile float64) (float64, error)
  • func (s *DDSketch) GetValuesAtQuantiles(quantiles []float64) ([]float64, error)
  • func (s *DDSketch) IsEmpty() bool
  • func (s *DDSketch) MergeWith(other *DDSketch) error
  • func (s *DDSketch) Reweight(w float64) error
• type DDSketchWithExactSummaryStatistics
  • func DecodeDDSketchWithExactSummaryStatistics(b []byte, storeProvider store.Provider, indexMapping mapping.IndexMapping) (*DDSketchWithExactSummaryStatistics, error)
  • func NewDDSketchWithExactSummaryStatistics(mapping mapping.IndexMapping, storeProvider store.Provider) *DDSketchWithExactSummaryStatistics
  • func NewDefaultDDSketchWithExactSummaryStatistics(relativeAccuracy float64) (*DDSketchWithExactSummaryStatistics, error)
  • func (s *DDSketchWithExactSummaryStatistics) Add(value float64) error
  • func (s *DDSketchWithExactSummaryStatistics) AddWithCount(value, count float64) error
  • func (s *DDSketchWithExactSummaryStatistics) ChangeMapping(newMapping mapping.IndexMapping, storeProvider store.Provider, ...) *DDSketchWithExactSummaryStatistics
  • func (s *DDSketchWithExactSummaryStatistics) Clear()
  • func (s *DDSketchWithExactSummaryStatistics) Copy() *DDSketchWithExactSummaryStatistics
  • func (s *DDSketchWithExactSummaryStatistics) DecodeAndMergeWith(bb []byte) error
  • func (s *DDSketchWithExactSummaryStatistics) Encode(b *[]byte, omitIndexMapping bool)
  • func (s *DDSketchWithExactSummaryStatistics) ForEach(f func(value, count float64) (stop bool))
  • func (s *DDSketchWithExactSummaryStatistics) GetCount() float64
  • func (s *DDSketchWithExactSummaryStatistics) GetMaxValue() (float64, error)
  • func (s *DDSketchWithExactSummaryStatistics) GetMinValue() (float64, error)
  • func (s *DDSketchWithExactSummaryStatistics) GetSum() float64
  • func (s *DDSketchWithExactSummaryStatistics) GetValueAtQuantile(quantile float64) (float64, error)
  • func (s *DDSketchWithExactSummaryStatistics) GetValuesAtQuantiles(quantiles []float64) ([]float64, error)
  • func (s *DDSketchWithExactSummaryStatistics) IsEmpty() bool
  • func (s *DDSketchWithExactSummaryStatistics) MergeWith(o *DDSketchWithExactSummaryStatistics) error
  • func (s *DDSketchWithExactSummaryStatistics) RelativeAccuracy() float64
  • func (s *DDSketchWithExactSummaryStatistics) Reweight(factor float64) error

Types

type DDSketch

type DDSketch struct {
	mapping.IndexMapping
	// contains filtered or unexported fields
}

func DecodeDDSketch

func DecodeDDSketch(b []byte, storeProvider store.Provider, indexMapping mapping.IndexMapping) (*DDSketch, error)

DecodeDDSketch deserializes a sketch. Stores are built using storeProvider. The store type needs not match the store that the serialized sketch initially used. However, using the same store type may make decoding faster. In the absence of high performance requirements, store.BufferedPaginatedStoreConstructor is a sound enough choice of store provider. To avoid memory allocations, it is possible to use a store provider that reuses stores, by calling Clear() on previously used stores before providing the store. If the serialized data does not contain the index mapping, you need to specify the index mapping that was used in the sketch that was encoded. Otherwise, you can use nil and the index mapping will be decoded from the serialized data. It is possible to decode with this function an encoded DDSketchWithExactSummaryStatistics, but the exact summary statistics will be lost.

func LogCollapsingHighestDenseDDSketch

func LogCollapsingHighestDenseDDSketch(relativeAccuracy float64, maxNumBins int) (*DDSketch, error)

Constructs an instance of DDSketch that offers constant-time insertion and whose size grows until the maximum number of bins is reached, at which point bins with highest indices are collapsed, which causes the relative accuracy guarantee to be lost on highest quantiles if values are all positive, or the lowest and highest quantiles if values include negative numbers.

func LogCollapsingLowestDenseDDSketch

func LogCollapsingLowestDenseDDSketch(relativeAccuracy float64, maxNumBins int) (*DDSketch, error)

Constructs an instance of DDSketch that offers constant-time insertion and whose size grows until the maximum number of bins is reached, at which point bins with lowest indices are collapsed, which causes the relative accuracy guarantee to be lost on lowest quantiles if values are all positive, or the mid-range quantiles for values closest to zero if values include negative numbers.

func LogUnboundedDenseDDSketch

func LogUnboundedDenseDDSketch(relativeAccuracy float64) (*DDSketch, error)

Constructs an instance of DDSketch that offers constant-time insertion and whose size grows indefinitely to accommodate for the range of input values.

func NewDDSketch

func NewDDSketch(indexMapping mapping.IndexMapping, positiveValueStore store.Store, negativeValueStore store.Store) *DDSketch

func NewDDSketchFromStoreProvider

func NewDDSketchFromStoreProvider(indexMapping mapping.IndexMapping, storeProvider store.Provider) *DDSketch

func NewDefaultDDSketch

func NewDefaultDDSketch(relativeAccuracy float64) (*DDSketch, error)

func (*DDSketch) Add

func (s *DDSketch) Add(value float64) error

Adds a value to the sketch.

func (*DDSketch) AddWithCount

func (s *DDSketch) AddWithCount(value, count float64) error

Adds a value to the sketch with a float64 count.

func (*DDSketch) ChangeMapping

func (s *DDSketch) ChangeMapping(newMapping mapping.IndexMapping, positiveStore store.Store, negativeStore store.Store, scaleFactor float64) *DDSketch

ChangeMapping changes the store to a new mapping. it doesn't change s but returns a newly created sketch. positiveStore and negativeStore must be different stores, and be empty when the function is called. It is not the conversion that minimizes the loss in relative accuracy, but it avoids artefacts like empty bins that make the histograms look bad. scaleFactor allows to scale out / in all values. (changing units for eg)

func (*DDSketch) Clear

func (s *DDSketch) Clear()

Clear empties the sketch while allowing reusing already allocated memory.

func (*DDSketch) Copy

func (s *DDSketch) Copy() *DDSketch

Return a (deep) copy of this sketch.

func (*DDSketch) DecodeAndMergeWith

func (s *DDSketch) DecodeAndMergeWith(bb []byte) error

DecodeAndMergeWith deserializes a sketch and merges its content in the receiver sketch. If the serialized content contains an index mapping that differs from the one of the receiver, DecodeAndMergeWith returns an error.

func (*DDSketch) Encode

func (s *DDSketch) Encode(b *[]byte, omitIndexMapping bool)

Encode serializes the sketch and appends the serialized content to the provided []byte. If the capacity of the provided []byte is large enough, Encode does not allocate memory space. When the index mapping is known at the time of deserialization, omitIndexMapping can be set to true to avoid encoding it and to make the serialized content smaller. The encoding format is described in the encoding/flag module.

func (*DDSketch) ForEach

func (s *DDSketch) ForEach(f func(value, count float64) (stop bool))

ForEach applies f on the bins of the sketches until f returns true. There is no guarantee on the bin iteration order.

func (*DDSketch) GetCount

func (s *DDSketch) GetCount() float64

Return the total number of values that have been added to this sketch.

func (*DDSketch) GetMaxValue

func (s *DDSketch) GetMaxValue() (float64, error)

Return the maximum value that has been added to this sketch. Return a non-nil error if the sketch is empty.

func (*DDSketch) GetMinValue

func (s *DDSketch) GetMinValue() (float64, error)

Return the minimum value that has been added to this sketch. Returns a non-nil error if the sketch is empty.

func (*DDSketch) GetSum

func (s *DDSketch) GetSum() (sum float64)

GetSum returns an approximation of the sum of the values that have been added to the sketch. If the values that have been added to the sketch all have the same sign, the approximation error has the relative accuracy guarantees of the mapping used for this sketch.

func (*DDSketch) GetValueAtQuantile

func (s *DDSketch) GetValueAtQuantile(quantile float64) (float64, error)

Return the value at the specified quantile. Return a non-nil error if the quantile is invalid or if the sketch is empty.

func (*DDSketch) GetValuesAtQuantiles

func (s *DDSketch) GetValuesAtQuantiles(quantiles []float64) ([]float64, error)

Return the values at the respective specified quantiles. Return a non-nil error if any of the quantiles is invalid or if the sketch is empty.

func (*DDSketch) IsEmpty

func (s *DDSketch) IsEmpty() bool

Return true iff no value has been added to this sketch.

func (*DDSketch) MergeWith

func (s *DDSketch) MergeWith(other *DDSketch) error

Merges the other sketch into this one. After this operation, this sketch encodes the values that were added to both this and the other sketches.

func (*DDSketch) Reweight

func (s *DDSketch) Reweight(w float64) error

Reweight multiplies all values from the sketch by w, but keeps the same global distribution. w has to be strictly greater than 0.

type DDSketchWithExactSummaryStatistics

type DDSketchWithExactSummaryStatistics struct {
	// contains filtered or unexported fields
}

DDSketchWithExactSummaryStatistics returns exact count, sum, min and max, as opposed to DDSketch, which may return approximate values for those statistics. Because of the need to track them exactly, adding and merging operations are slightly more exepensive than those of DDSketch.

func DecodeDDSketchWithExactSummaryStatistics

func DecodeDDSketchWithExactSummaryStatistics(b []byte, storeProvider store.Provider, indexMapping mapping.IndexMapping) (*DDSketchWithExactSummaryStatistics, error)

DecodeDDSketchWithExactSummaryStatistics deserializes a sketch. Stores are built using storeProvider. The store type needs not match the store that the serialized sketch initially used. However, using the same store type may make decoding faster. In the absence of high performance requirements, store.DefaultProvider is a sound enough choice of store provider. To avoid memory allocations, it is possible to use a store provider that reuses stores, by calling Clear() on previously used stores before providing the store. If the serialized data does not contain the index mapping, you need to specify the index mapping that was used in the sketch that was encoded. Otherwise, you can use nil and the index mapping will be decoded from the serialized data. It is not possible to decode with this function an encoded DDSketch (unless it is empty), because it does not track exact summary statistics

func NewDDSketchWithExactSummaryStatistics

func NewDDSketchWithExactSummaryStatistics(mapping mapping.IndexMapping, storeProvider store.Provider) *DDSketchWithExactSummaryStatistics

func NewDefaultDDSketchWithExactSummaryStatistics

func NewDefaultDDSketchWithExactSummaryStatistics(relativeAccuracy float64) (*DDSketchWithExactSummaryStatistics, error)

func (*DDSketchWithExactSummaryStatistics) Add

func (s *DDSketchWithExactSummaryStatistics) Add(value float64) error

func (*DDSketchWithExactSummaryStatistics) AddWithCount

func (s *DDSketchWithExactSummaryStatistics) AddWithCount(value, count float64) error

func (*DDSketchWithExactSummaryStatistics) ChangeMapping

func (s *DDSketchWithExactSummaryStatistics) ChangeMapping(newMapping mapping.IndexMapping, storeProvider store.Provider, scaleFactor float64) *DDSketchWithExactSummaryStatistics

func (*DDSketchWithExactSummaryStatistics) Clear

func (s *DDSketchWithExactSummaryStatistics) Clear()

func (*DDSketchWithExactSummaryStatistics) Copy

func (s *DDSketchWithExactSummaryStatistics) Copy() *DDSketchWithExactSummaryStatistics

func (*DDSketchWithExactSummaryStatistics) DecodeAndMergeWith

func (s *DDSketchWithExactSummaryStatistics) DecodeAndMergeWith(bb []byte) error

func (*DDSketchWithExactSummaryStatistics) Encode

func (s *DDSketchWithExactSummaryStatistics) Encode(b *[]byte, omitIndexMapping bool)

func (*DDSketchWithExactSummaryStatistics) ForEach

func (s *DDSketchWithExactSummaryStatistics) ForEach(f func(value, count float64) (stop bool))

func (*DDSketchWithExactSummaryStatistics) GetCount

func (s *DDSketchWithExactSummaryStatistics) GetCount() float64

func (*DDSketchWithExactSummaryStatistics) GetMaxValue

func (s *DDSketchWithExactSummaryStatistics) GetMaxValue() (float64, error)

func (*DDSketchWithExactSummaryStatistics) GetMinValue

func (s *DDSketchWithExactSummaryStatistics) GetMinValue() (float64, error)

func (*DDSketchWithExactSummaryStatistics) GetSum

func (s *DDSketchWithExactSummaryStatistics) GetSum() float64

func (*DDSketchWithExactSummaryStatistics) GetValueAtQuantile

func (s *DDSketchWithExactSummaryStatistics) GetValueAtQuantile(quantile float64) (float64, error)

func (*DDSketchWithExactSummaryStatistics) GetValuesAtQuantiles

func (s *DDSketchWithExactSummaryStatistics) GetValuesAtQuantiles(quantiles []float64) ([]float64, error)

func (*DDSketchWithExactSummaryStatistics) IsEmpty

func (s *DDSketchWithExactSummaryStatistics) IsEmpty() bool

func (*DDSketchWithExactSummaryStatistics) MergeWith

func (s *DDSketchWithExactSummaryStatistics) MergeWith(o *DDSketchWithExactSummaryStatistics) error

func (*DDSketchWithExactSummaryStatistics) RelativeAccuracy

func (s *DDSketchWithExactSummaryStatistics) RelativeAccuracy() float64

func (*DDSketchWithExactSummaryStatistics) Reweight

func (s *DDSketchWithExactSummaryStatistics) Reweight(factor float64) error