hashcounter

README

hashcounter

Provides an efficient way to count the number of occurences of unique integers provided you can tolerate some collisions. Although not as efficient as HyperLogLog++, it can also be used to just count the number of distinct values.

The exposed functions are not thread-safe.

Do not use this package if you cannot tolerate any collisions. By default the key will be determined by calling xxhash.Sum64. You can provide your own hash function as long as it returns a uint64 (like crc64, murmur3, etc). Since there is some hashing involved, there might be collisions. You've been warned. If your byte slices can safely fit in a uint64 without losing any bits then feel free to send a custom hash function using something like binary.Uvarint.

Usage

c := new(hashcounter.C)
c.Add([]byte(`hello`), 1) // record 1 count of "hello"

cnt, ok := c.Get([]byte(`hello`)) // returns 1 and true

Benchmarks

Tradeoffs were made to prefer lower memory over faster execution speed.

BenchmarkAdd-12          2000000              1011 ns/op
BenchmarkGet-12          2000000               955 ns/op
BenchmarkGetKey-12       3000000              1197 ns/op
BenchmarkKey-12         20000000                83.8 ns/op

Documentation

Overview

Package hashcounter provides an efficient way to count the number of occurences of unique integers. The exposed functions are not thread-safe.

Do not use this package if you cannot tolerate any collisions. By default the key will be determined by calling xxhash.Sum64. You can provide your own hash function as long as it returns a uint64 (like crc64, murmur3, etc). Since there is some hashing involved, there might be collisions. You've been warned. If your byte slices can safely fit in a uint64 without losing any bits then feel free to send a custom hash function using something like binary.Uvarint.

You can initialize a new C using New or just using new(hashcounter.C).

Index

• type C
  • func New() *C
  • func NewWithHash(fn func([]byte) uint64) *C
  • func (m *C) Add(b []byte, v uint16)
  • func (m *C) Get(b []byte) (uint16, bool)
  • func (m *C) GetKey(k uint64) (uint16, bool)
  • func (m *C) Key(k []byte) uint64
  • func (m *C) Len() int
  • func (m *C) MarshalBinary() ([]byte, error)
  • func (m *C) Merge(n *C)
  • func (m *C) Range(f func(key uint64, value uint16) bool)
  • func (m *C) Reset()
  • func (m *C) UnmarshalBinary(b []byte) error

Types

type C

type C struct {
	// contains filtered or unexported fields
}

C holds an array of unique values and an associative count

func New

func New() *C

New returns a new instance of C

func NewWithHash

func NewWithHash(fn func([]byte) uint64) *C

NewWithHash returns a new instance of C with the provided hash function

func (*C) Add

func (m *C) Add(b []byte, v uint16)

Add adds the value to the given bytes

func (*C) Get

func (m *C) Get(b []byte) (uint16, bool)

Get returns the value of the given bytes and a boolean if it was found

func (*C) GetKey

func (m *C) GetKey(k uint64) (uint16, bool)

GetKey takes a key rather than bytes but otherwise behaves like Get

func (*C) Key

func (m *C) Key(k []byte) uint64

Key returns the uint64 key for the given bytes

func (*C) Len

func (m *C) Len() int

Len returns a count of all of the keys

func (*C) MarshalBinary

func (m *C) MarshalBinary() ([]byte, error)

MarshalBinary implements the encoding.BinaryMarshaler interface.

func (*C) Merge

func (m *C) Merge(n *C)

Merge adds every key from the sent C to the called on C. This assumes the hash functions are the same.

func (*C) Range

func (m *C) Range(f func(key uint64, value uint16) bool)

Range calls the given function for every value in the map and continues looping until the given bool. The returned key is going to be the result of Key(bytes). If you want the key to be reversable, you must pass a hash function to NewWithHash that allows you to reverse the operation.

func (*C) Reset

func (m *C) Reset()

Reset removes all of the keys and returns C to it's empty state

func (*C) UnmarshalBinary

func (m *C) UnmarshalBinary(b []byte) error

UnmarshalBinary implements the encoding.BinaryUnmarshaler interface.