db

Documentation

Index

• Variables
• type CompactionStats
  • func (stats *CompactionStats) AddTimeSince(start time.Time)
• type Database
  • func Init(filesys fs.Filesys) *Database
  • func Open(fs fs.Filesys) *Database
  • func (db *Database) Close()
  • func (db *Database) Compact()
  • func (db *Database) Delete(k Key)
  • func (db *Database) DeleteObsoleteFiles()
  • func (db *Database) Get(k Key) MaybeValue
  • func (db *Database) Put(k Key, v Value)
• type Decoder
  • func (r Decoder) FixedHandle() SliceHandle
  • func (r Decoder) Handle() SliceHandle
  • func (r *Decoder) IndexEntry() indexEntry
  • func (r Decoder) Key() Key
  • func (r *Decoder) KeyRange() KeyRange
  • func (r Decoder) KeyUpdate() KeyUpdate
• type Encoder
  • func (w *Encoder) FixedHandle(h SliceHandle)
  • func (w *Encoder) Handle(h SliceHandle)
  • func (w *Encoder) IndexEntry(e indexEntry)
  • func (w *Encoder) Key(k Key)
  • func (w *Encoder) KeyRange(keys KeyRange)
  • func (w *Encoder) KeyUpdate(e KeyUpdate)
• type Entry
• type Key
• type KeyRange
  • func (r KeyRange) Contains(k Key) bool
• type KeyUpdate
  • func (u KeyUpdate) IsPut() bool
• type Manifest
  • func (m *Manifest) CreateTable() tableCreator
  • func (m Manifest) Get(k Key) MaybeValue
  • func (m *Manifest) InstallTable(newTable Table, youngTables []uint32, level1tables []uint32, level int)
• type MaybeMaybeValue
• type MaybeValue
  • func SomeValue(v Value) MaybeValue
• type SliceHandle
  • func (h SliceHandle) IsValid() bool
• type Store
• type Table
  • func NewTable(ident uint32, f fs.ReadFile, entries []indexEntry) Table
  • func OpenTable(ident uint32, fs fs.Filesys) Table
  • func (t Table) Get(k Key) MaybeMaybeValue
  • func (t Table) Keys() KeyRange
  • func (t Table) Name() string
  • func (t Table) Updates() UpdateIterator
• type UpdateIterator
  • func MergeUpdates(iterators []UpdateIterator) UpdateIterator
• type Value

Variables

var NoValue = MaybeValue{Present: false, Value: nil}

Types

type CompactionStats

type CompactionStats struct {
	TotalTime time.Duration
}

func (*CompactionStats) AddTimeSince

func (stats *CompactionStats) AddTimeSince(start time.Time)

type Database

type Database struct {
	Stats *CompactionStats
	// contains filtered or unexported fields
}

A Database is a persistent key-value store.

func Init

func Init(filesys fs.Filesys) *Database

Init creates a new database in a filesystem, replacing anything in the directory.

func Open

func Open(fs fs.Filesys) *Database

Open recovers a Database from an existing on-disk database (of course this also works following a clean shutdown).

func (*Database) Close

func (db *Database) Close()

Close cleanly shuts down the database, and moreover pushes all data to tables for simple recovery.

func (*Database) Compact

func (db *Database) Compact()

Compact manually triggers a full compaction of the log and tables.

func (*Database) Delete

func (db *Database) Delete(k Key)

func (*Database) DeleteObsoleteFiles

func (db *Database) DeleteObsoleteFiles()

DeleteObsoleteFiles deletes files the database doesn't know about.

This isn't currently correctly called by recovery, but at some point it is required to clean up partially constructed tables that weren't successfully added to the database.

func (*Database) Get

func (db *Database) Get(k Key) MaybeValue

func (*Database) Put

func (db *Database) Put(k Key, v Value)

type Decoder

type Decoder struct {
	*bin.Decoder
}

func (Decoder) FixedHandle

func (r Decoder) FixedHandle() SliceHandle

func (Decoder) Handle

func (r Decoder) Handle() SliceHandle

func (*Decoder) IndexEntry

func (r *Decoder) IndexEntry() indexEntry

func (Decoder) Key

func (r Decoder) Key() Key

func (*Decoder) KeyRange

func (r *Decoder) KeyRange() KeyRange

func (Decoder) KeyUpdate

func (r Decoder) KeyUpdate() KeyUpdate

type Encoder

type Encoder struct {
	*bin.Encoder
}

func (*Encoder) FixedHandle

func (w *Encoder) FixedHandle(h SliceHandle)

func (*Encoder) Handle

func (w *Encoder) Handle(h SliceHandle)

func (*Encoder) IndexEntry

func (w *Encoder) IndexEntry(e indexEntry)

func (*Encoder) Key

func (w *Encoder) Key(k Key)

func (*Encoder) KeyRange

func (w *Encoder) KeyRange(keys KeyRange)

func (*Encoder) KeyUpdate

func (w *Encoder) KeyUpdate(e KeyUpdate)

type Entry

type Entry struct {
	Key
	Value
}

type Key

type Key uint64

type KeyRange

type KeyRange struct {
	Min Key
	Max Key
}

func (KeyRange) Contains

func (r KeyRange) Contains(k Key) bool

type KeyUpdate

type KeyUpdate struct {
	Key
	MaybeValue
}

func (KeyUpdate) IsPut

func (u KeyUpdate) IsPut() bool

type Manifest

type Manifest struct {
	// contains filtered or unexported fields
}

A Manifest is a handle to a set of Tables, along with a separate on-disk data structure to track which tables are part of the database. It manages the underlying tables, recovering and writing to them as necessary, and exports an interface to create tables and install them into the manifest in a crash-safe manner.

func (*Manifest) CreateTable

func (m *Manifest) CreateTable() tableCreator

CreateTable initializes a new table writer

This operation requires write permissions (for silly reasons - it only protects the identifier counter)

func (Manifest) Get

func (m Manifest) Get(k Key) MaybeValue

Get reads a key from the tables managed by the manifest file.

func (*Manifest) InstallTable

func (m *Manifest) InstallTable(newTable Table, youngTables []uint32, level1tables []uint32, level int)

InstallTable adds a previously created table to the tracked tables in the manifest.

Requires that the table already be stored in the right place (using m.CreateTable() and its associated operations).

This operation requires write permissions to the manifest.

type MaybeMaybeValue

type MaybeMaybeValue struct {
	Valid bool
	MaybeValue
}

MaybeMaybeValue is a poor man's option (option Value).

type MaybeValue

type MaybeValue struct {
	Present bool
	Value
}

func SomeValue

func SomeValue(v Value) MaybeValue

type SliceHandle

type SliceHandle struct {
	Offset uint64
	Length uint32
}

A SliceHandle represents a slice into a file.

This is what LevelDB calls a BlockHandle, but we don't have the same general notion of a variable-sized block.

func (SliceHandle) IsValid

func (h SliceHandle) IsValid() bool

IsValid reports whether a SliceHandle is valid, which requires that it addresses a non-empty range.

type Store

type Store interface {
	Get(k Key) MaybeValue
	Put(k Key, v Value)
	Delete(k Key)
	// TODO: iterator API
	Close()
}

type Table

type Table struct {
	// contains filtered or unexported fields
}

A Table is a handle to and index over a table, the basic immutable storage unit of the database (the equivalent of an SSTable in LevelDB, which is the SSTable of Bigtable).

func NewTable

func NewTable(ident uint32, f fs.ReadFile, entries []indexEntry) Table

NewTable creates the in-memory structure representing a table

func OpenTable

func OpenTable(ident uint32, fs fs.Filesys) Table

OpenTable reads a table on-disk, initializing the in-memory cache.

func (Table) Get

func (t Table) Get(k Key) MaybeMaybeValue

Get reads a key from the table.

Since tables represent only part of the database, this Get returns a MaybeMaybeValue to represent a key that is not part of the table, as opposed to a key the table has a deletion marker for.

func (Table) Keys

func (t Table) Keys() KeyRange

Keys gives the range of keys covered by this table.

func (Table) Name

func (t Table) Name() string

Name returns the filename used to store this table.

func (Table) Updates

func (t Table) Updates() UpdateIterator

Updates returns all the updates (puts an deletes) the table holds.

type UpdateIterator

type UpdateIterator interface {
	HasNext() bool
	Next() KeyUpdate
}

func MergeUpdates

func MergeUpdates(iterators []UpdateIterator) UpdateIterator

MergeUpdates takes several iterators and produces a merged iterator.

iterators should be sorted by key, and MergeUpdates will produce an iterator that is also sorted.

type Value

type Value []byte