cuckoo

Documentation

Overview

Package cuckoo provides a Cuckoo Filter, a Bloom filter replacement for approximated set-membership queries.

While Bloom filters are well-known space-efficient data structures to serve queries like "if item x is in a set?", they do not support deletion. Their variances to enable deletion (like counting Bloom filters) usually require much more space.

Cuckoo filters provide the flexibility to add and remove items dynamically. A cuckoo filter is based on cuckoo hashing (and therefore named as cuckoo filter). It is essentially a cuckoo hash table storing each key's fingerprint. Cuckoo hash tables can be highly compact, thus a cuckoo filter could use less space than conventional Bloom filters, for applications that require low false positive rates (< 3%).

For details about the algorithm and citations please use this article:

"Cuckoo Filter: Better Than Bloom" by Bin Fan, Dave Andersen and Michael Kaminsky (https://www.cs.cmu.edu/~dga/papers/cuckoo-conext2014.pdf)

Note: This implementation uses a a static bucket size of 4 fingerprints and a fingerprint size of 1 byte based on my understanding of an optimal bucket/fingerprint/size ratio from the aforementioned paper.

Index

• type Filter
  • func Decode(bytes []byte) (*Filter, error)
  • func NewFilter(capacity uint) *Filter
  • func (cf *Filter) Add(data []byte) bool
  • func (cf *Filter) AddUnique(data []byte) bool
  • func (cf *Filter) Count() uint
  • func (cf *Filter) Delete(data []byte) bool
  • func (cf *Filter) Encode() []byte
  • func (cf *Filter) Lookup(data []byte) bool
  • func (cf *Filter) Reset()

Types

type Filter

type Filter struct {
	// contains filtered or unexported fields
}

Filter is a probabalistic counter

func Decode

func Decode(bytes []byte) (*Filter, error)

Decode returns a Cuckoofilter from a byte slice

func NewFilter

func NewFilter(capacity uint) *Filter

NewFilter returns a new cuckoofilter with a given capacity. A capacity of 1000000 is a normal default, which allocates about ~1MB on 64-bit machines.

func (*Filter) Add

func (cf *Filter) Add(data []byte) bool

Add inserts data into the counter and returns true upon success

func (*Filter) AddUnique

func (cf *Filter) AddUnique(data []byte) bool

InsertUnique inserts data into the counter if not exists and returns true upon success

func (*Filter) Count

func (cf *Filter) Count() uint

Count returns the number of items in the counter

func (*Filter) Delete

func (cf *Filter) Delete(data []byte) bool

Delete data from counter if exists and return if deleted or not

func (*Filter) Encode

func (cf *Filter) Encode() []byte

Encode returns a byte slice representing a Cuckoofilter

func (*Filter) Lookup

func (cf *Filter) Lookup(data []byte) bool

Lookup returns true if data is in the counter

func (*Filter) Reset

func (cf *Filter) Reset()