bigmap

Documentation

Overview

Package bigmap implements a custom hashmap data structure where keys and values are binary and keys are meant to be well-distributed hashes, such as content IDs, object IDs, etc.

Unlike regular maps this map is limited to adding and getting elements but does not support any iteration or deletion, but is much more efficient in terms of memory usage.

Data for the hash table is stored in large contiguous memory blocks (for small sets) and automatically spills over to memory-mapped files for larger sets using only 8 bytes per key in RAM.

Index

• type Map
  • func NewMap(ctx context.Context) (*Map, error)
  • func NewMapWithOptions(ctx context.Context, opt *Options) (*Map, error)
  • func (s *Map) Close(ctx context.Context)
  • func (s *Map) Contains(key []byte) bool
  • func (s *Map) Get(ctx context.Context, output, key []byte) (result []byte, ok bool, err error)
  • func (s *Map) PutIfAbsent(ctx context.Context, key, value []byte) bool
• type Options
• type Set
  • func NewSet(ctx context.Context) (*Set, error)
  • func NewSetWithOptions(ctx context.Context, opt *Options) (*Set, error)
  • func (s *Set) Close(ctx context.Context)
  • func (s *Set) Contains(key []byte) bool
  • func (s *Set) Put(ctx context.Context, key []byte) bool

Types

type Map

type Map struct {
	// contains filtered or unexported fields
}

Map is a wrapper around internalMap that adds encryption.

func NewMap

func NewMap(ctx context.Context) (*Map, error)

NewMap creates new Map.

func NewMapWithOptions

func NewMapWithOptions(ctx context.Context, opt *Options) (*Map, error)

NewMapWithOptions creates new Map with options.

func (*Map) Close

func (s *Map) Close(ctx context.Context)

Close releases resources associated with the set.

func (*Map) Contains

func (s *Map) Contains(key []byte) bool

Contains returns true if a given key exists in the set.

func (*Map) Get

func (s *Map) Get(ctx context.Context, output, key []byte) (result []byte, ok bool, err error)

Get gets the element from the map and appends the value to the provided buffer.

func (*Map) PutIfAbsent

func (s *Map) PutIfAbsent(ctx context.Context, key, value []byte) bool

PutIfAbsent adds the element to the map, returns true if the element was added (as opposed to having existed before).

type Options

type Options struct {
	LoadFactorPercentage int   // grow the size of the hash table when this percentage full
	NumMemorySegments    int   // number of segments to keep in RAM
	MemorySegmentSize    int64 // size of a single memory segment
	FileSegmentSize      int   // size of a single file segment, defaults to 1 GiB

	InitialSizeLogarithm int // logarithm of the initial size of the hash table, default - 20
}

Options provides options for the internalMap.

type Set

type Set struct {
	// contains filtered or unexported fields
}

Set is a wrapper around Map that only supports Put() and Contains().

func NewSet

func NewSet(ctx context.Context) (*Set, error)

NewSet creates new Set.

func NewSetWithOptions

func NewSetWithOptions(ctx context.Context, opt *Options) (*Set, error)

NewSetWithOptions creates new Set with options.

func (*Set) Close

func (s *Set) Close(ctx context.Context)

Close releases resources associated with the set.

func (*Set) Contains

func (s *Set) Contains(key []byte) bool

Contains returns true if a given key exists in the set.

func (*Set) Put

func (s *Set) Put(ctx context.Context, key []byte) bool

Put adds the element to a set, returns true if the element was added (as opposed to having existed before).