storage

Documentation

Overview

Package storage implements a GC friendly in-memory storage engine by using map and byte array. It also supports compaction.

Index

• Variables
• type Entry
  • func NewEntry() *Entry
  • func (e *Entry) Decode(buf []byte)
  • func (e *Entry) Encode() []byte
• type SlabInfo
• type Storage
  • func Import(data []byte) (*Storage, error)
  • func New(size int) *Storage
  • func (s *Storage) Check(hkey uint64) bool
  • func (s *Storage) CompactTables() bool
  • func (s *Storage) Delete(hkey uint64) error
  • func (s *Storage) Export() ([]byte, error)
  • func (s *Storage) Get(hkey uint64) (*Entry, error)
  • func (s *Storage) GetKey(hkey uint64) (string, error)
  • func (s *Storage) GetRaw(hkey uint64) ([]byte, error)
  • func (s *Storage) GetTTL(hkey uint64) (int64, error)
  • func (s *Storage) Inuse() int
  • func (s *Storage) Len() int
  • func (s *Storage) MatchOnKey(expr string, f func(hkey uint64, entry *Entry) bool) error
  • func (s *Storage) NumTables() int
  • func (s *Storage) Put(hkey uint64, value *Entry) error
  • func (s *Storage) PutRaw(hkey uint64, value []byte) error
  • func (s *Storage) Range(f func(hkey uint64, entry *Entry) bool)
  • func (s *Storage) SlabInfo() SlabInfo
  • func (s *Storage) UpdateTTL(hkey uint64, data *Entry) error

Variables

var (

	// ErrKeyTooLarge is an error that indicates the given key is large than the determined key size.
	// The current maximum key length is 256.
	ErrKeyTooLarge = errors.New("key too large")

	// ErrKeyNotFound is an error that indicates that the requested key could not be found in the DB.
	ErrKeyNotFound = errors.New("key not found")
)

var ErrFragmented = errors.New("storage fragmented")

ErrFragmented is an error that indicates this storage instance is currently fragmented and it cannot be serialized.

Types

type Entry

type Entry struct {
	Key       string
	TTL       int64
	Timestamp int64
	Value     []byte
}

Entry represents a value with its metadata.

func NewEntry

func NewEntry() *Entry

func (*Entry) Decode

func (e *Entry) Decode(buf []byte)

func (*Entry) Encode

func (e *Entry) Encode() []byte

type SlabInfo

type SlabInfo struct {
	Allocated int
	Inuse     int
	Garbage   int
}

SlabInfo is used to expose internal data usage of a storage instance.

type Storage

type Storage struct {
	// contains filtered or unexported fields
}

Storage implements a new off-heap data store which uses built-in map to keep metadata and mmap syscall for allocating memory to store values. The allocated memory is not a subject of Golang's GC.

func Import

func Import(data []byte) (*Storage, error)

Import gets the serialized data by Export and creates a new storage instance.

func New

func New(size int) *Storage

New creates a new storage instance.

func (*Storage) Check

func (s *Storage) Check(hkey uint64) bool

Check checks the key existence.

func (*Storage) CompactTables

func (s *Storage) CompactTables() bool

func (*Storage) Delete

func (s *Storage) Delete(hkey uint64) error

Delete deletes the value for the given key. Delete will not returns error if key doesn't exist.

func (*Storage) Export

func (s *Storage) Export() ([]byte, error)

Export serializes underlying data structes into a byte slice. It may return ErrFragmented if the tables are fragmented. If you get this error, you should try to call Export again some time later.

func (*Storage) Get

func (s *Storage) Get(hkey uint64) (*Entry, error)

Get gets the value for the given key. It returns ErrKeyNotFound if the DB does not contains the key. The returned Entry is its own copy, it is safe to modify the contents of the returned slice.

func (*Storage) GetKey

func (s *Storage) GetKey(hkey uint64) (string, error)

GetKey gets the key for the given hkey. It returns ErrKeyNotFound if the DB does not contains the key.

func (*Storage) GetRaw

func (s *Storage) GetRaw(hkey uint64) ([]byte, error)

GetRaw extracts encoded value for the given hkey. This is useful for merging tables.

func (*Storage) GetTTL

func (s *Storage) GetTTL(hkey uint64) (int64, error)

GetTTL gets the timeout for the given key. It returns ErrKeyNotFound if the DB does not contains the key.

func (*Storage) Inuse

func (s *Storage) Inuse() int

Inuse returns total in-use space by the tables.

func (*Storage) Len

func (s *Storage) Len() int

Len returns the key cound in this storage.

func (*Storage) MatchOnKey

func (s *Storage) MatchOnKey(expr string, f func(hkey uint64, entry *Entry) bool) error

MatchOnKey calls a regular expression on keys and provides an iterator.

func (*Storage) NumTables

func (s *Storage) NumTables() int

NumTables returns the number of tables in a storage instance.

func (*Storage) Put

func (s *Storage) Put(hkey uint64, value *Entry) error

Put sets the value for the given key. It overwrites any previous value for that key

func (*Storage) PutRaw

func (s *Storage) PutRaw(hkey uint64, value []byte) error

PutRaw sets the raw value for the given key.

func (*Storage) Range

func (s *Storage) Range(f func(hkey uint64, entry *Entry) bool)

Range calls f sequentially for each key and value present in the map. If f returns false, range stops the iteration. Range may be O(N) with the number of elements in the map even if f returns false after a constant number of calls.

func (*Storage) SlabInfo

func (s *Storage) SlabInfo() SlabInfo

SlabInfo is a function which provides memory allocation and garbage ratio of a storage instance.

func (*Storage) UpdateTTL

func (s *Storage) UpdateTTL(hkey uint64, data *Entry) error

UpdateTTL updates the expiry for the given key.