Documentation ¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrorMismatchedVersion is returned from a Merge when two sketch versions do not match. ErrorMismatchedVersion = errors.New("sketch version mismatch") // ErrorMalformedPrecision is returned from a Merge when a sketch is found to have differing precisions (or // is malformed/incorrectly deserialized). ErrorMalformedPrecision = errors.New("sketch precision mismatch") )
Functions ¶
Types ¶
type BiasEstimate ¶
BiasEstimate holds an average cardinality estimate and a average resulting bias. This is produced by running a number of simulations on a set containing exactly TrueCardinality. (I.e. If a Sketch produces this RawEstimatedCardinality (without BiasCorrection or LinearCounting, we need to weight by Bias to get close to TrueCardinality).
func GenerateBiases ¶
func GenerateBiases(fn func() []byte, options *GenerationOptions) ([]*BiasEstimate, error)
GenerateBiases can be used to create a list of BiasEstimate for arbitrary []byte generated by fn. It should be used if defaultGeneratedBiases do not produce accurate estimates for a specific use-case (or if you have the patience/compute to run more precise estimates). The options used to generate the default biases are returned from DefaultGenerationOptions() and these are used if options is nil.
Depending on fn and options this can take some time and use significant memory...
WARNING: If fn produces a set of unique values less than options.MaxCardinality, this will never return. NOTE: For periodic log.Printf output, set envvar HLL_BIAS_LOG=1.
type GenerationOptions ¶
type GenerationOptions struct { MaxCardinality uint64 Repeats int InitialStep int StepRate float64 }
GenerationOptions contains parameters used for GenerateBiases.
func DefaultGenerationOptions ¶
func DefaultGenerationOptions() *GenerationOptions
DefaultGenerationOptions returns a copy of the default GenerationOptions.
type Sketch ¶
type Sketch interface { // Insert inserts element into the Sketch. Insert(element []byte) // Estimate returns an estimate (+/-3%) of the number of uniques (cardinality) of everything that //has been inserted. Estimate() uint64 // Merge merges this Sketch with other, returning itself (now combined with other) and an non-nil // error if the Merge could not be completed. Merge(other Sketch) (Sketch, error) // ProtoSerialize returns []byte representing this Sketch. The serialization format can be found // at: https://github.com/kixa/hll-protobuf ProtoSerialize() ([]byte, error) // contains filtered or unexported methods }
Sketch is an interface that wraps a HyperLogLog implementation for counting unique elements.
func NewCustomSketch ¶
NewCustomSketch returns a new Sketch using the biases registered under biasKey. If these biases are not found (previously registered via RegisterBiases), an error will be returned.
func ProtoDeserialize ¶
ProtoDeserialize returns a Sketch from an encoded protobuf version. The proto schema used can be found in the companion repository: https://github.com/kixa/hll-protobuf