Documentation ¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrorMismatchedVersion is returned from a Merge when two sketch versions do not match. ErrorMismatchedVersion = errors.New("sketch version mismatch") // ErrorMalformedPrecision is returned from a Merge when a sketch is found to have differing precisions (or // is malformed/incorrectly deserialized). ErrorMalformedPrecision = errors.New("sketch precision mismatch") )
Functions ¶
Types ¶
type BiasEstimate ¶
BiasEstimate holds an average cardinality estimate and a average resulting bias. This is produced by running a number of simulations on a set containing exactly TrueCardinality. (I.e. If a Sketch produces this RawEstimatedCardinality (without BiasCorrection or LinearCounting, we need to weight by Bias to get close to TrueCardinality).
func GenerateBiases ¶
func GenerateBiases(fn func() []byte, options *GenerationOptions) ([]*BiasEstimate, error)
GenerateBiases can be used to create a list of BiasEstimate for arbitrary []byte generated by fn. It should be used if defaultGeneratedBiases do not produce accurate estimates for a specific use-case (or if you have the patience/compute to run more precise estimates). The options used to generate the default biases are returned from DefaultGenerationOptions() and these are used if options is nil.
Depending on fn and options this can take some time and use significant memory...
WARNING: If fn produces a set of unique values less than options.MaxCardinality, this will never return. NOTE: For periodic log.Printf output, set envvar HLL_BIAS_LOG=1.
type GenerationOptions ¶
type GenerationOptions struct { MaxCardinality uint64 Repeats int InitialStep int StepRate float64 }
GenerationOptions contains parameters used for GenerateBiases.
func DefaultGenerationOptions ¶
func DefaultGenerationOptions() *GenerationOptions
DefaultGenerationOptions returns a copy of the default GenerationOptions.
type Sketch ¶
type Sketch interface { // Insert inserts element into the Sketch. Insert(element []byte) // Estimate returns an estimate (+/-3%) of the number of uniques (cardinality) of everything that //has been inserted. Estimate() uint64 // Merge merges this Sketch with other, returning itself (now combined with other) and an non-nil // error if the Merge could not be completed. Merge(other Sketch) (Sketch, error) // ProtoSketch returns the protobuf serializable struct representing this Sketch and should only be used for // embedding Sketches into larger protobuf messages. For all other use-cases use ProtoSerialize. // The reference proto format can be found at: https://github.com/kixa/hll-protobuf ProtoSketch() *hllProto.Sketch // ProtoSerialize returns []byte representing this Sketch. // The reference proto format can be found at: https://github.com/kixa/hll-protobuf // This is equivalent to calling proto.Marshal on the result of ProtoSketch. ProtoSerialize() ([]byte, error) // contains filtered or unexported methods }
Sketch is an interface that wraps a HyperLogLog implementation for counting unique elements.
func FromProtoSketch ¶ added in v1.1.1
FromProtoSketch returns a Sketch from a generated protobuf type. The proto schema used can be found in the companion repository: https://github.com/kixa/hll-protobuf
func NewCustomSketch ¶
NewCustomSketch returns a new Sketch using the biases registered under biasKey. If these biases are not found (previously registered via RegisterBiases), an error will be returned.
func ProtoDeserialize ¶
ProtoDeserialize returns a Sketch from an encoded protobuf version. The proto schema used can be found in the companion repository: https://github.com/kixa/hll-protobuf