wisckey

README

WisckeyDB

WisckeyDB is an embeddable, persistent, mutable and fast key-value (KV) database written in pure Go.

Documentation

Index

• Constants
• Variables
• type DB
  • func Open(opt Options) (*DB, error)
• type Logger
• type Manifest
  • func ReplayManifestFile(fp *os.File) (ret Manifest, truncOffset int64, err error)
• type Options
• type TableManifest

Constants

const (
	// ManifestFilename is the filename for the manifest file.
	ManifestFilename = "MANIFEST"
)

Variables

var DefaultOptions = Options{
	Sync:                    true,
	NumVersions:             1,
	ReadOnly:                false,
	Logger:                  nil,
	MaxTableSize:            64 << 20,
	MaxLevels:               7,
	ValueThreshold:          512,
	NumMemtables:            5,
	NumLevelZeroTables:      5,
	NumLevelZeroTablesStall: 10,
	LevelOneSize:            256 << 20,
	NumCompactors:           2,
}

DefaultOptions sets a list of recommended options for good performance. Feel free to modify these to suit your needs.

var (
	// ErrValueThreshold is returned when ValueThreshold is set to a value close to or greater than
	// uint16.
	ErrValueThreshold = errors.New("Invalid ValueThreshold, must be lower than uint16")
)

Types

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB provides the thread-safe various functions required to interact with wisckey.

func Open

func Open(opt Options) (*DB, error)

Open returns a new DB object.

type Logger

type Logger interface {
	Debug(v ...interface{})
	Debugf(format string, v ...interface{})

	Error(v ...interface{})
	Errorf(format string, v ...interface{})

	Info(v ...interface{})
	Infof(format string, v ...interface{})

	Warning(v ...interface{})
	Warningf(format string, v ...interface{})

	Fatal(v ...interface{})
	Fatalf(format string, v ...interface{})
}

Logger is implemented by any logging system that is used for standard logs.

type Manifest

type Manifest struct {
	Levels []levelManifest
	Tables map[uint64]TableManifest

	// Contains total number of creation and deletion changes in the manifest -- used to compute
	// whether it'd be useful to rewrite the manifest.
	Creations int
	Deletions int
}

Manifest represents the contents of the MANIFEST file in a Wisckey store.

The MANIFEST file describes the startup state of the db -- all LSM files and what level they're at.

It consists of a sequence of ManifestChangeSet objects. Each of these is treated atomically, and contains a sequence of ManifestChange's (file creations/deletions) which we use to reconstruct the manifest at startup.

func ReplayManifestFile

func ReplayManifestFile(fp *os.File) (ret Manifest, truncOffset int64, err error)

ReplayManifestFile reads the manifest file and constructs two manifest objects. (We need one immutable copy and one mutable copy of the manifest. Easiest way is to construct two of them.) Also, returns the last offset after a completely read manifest entry -- the file must be truncated at that point before further appends are made (if there is a partial entry after that). In normal conditions, truncOffset is the file size.

type Options

type Options struct {
	// Directory to store the lsm data in. If it doesn't exist, Wisckey will
	// try to create it for you.
	LSMDir string

	// File or device to store the value log in.
	VLogPath string

	// Sync all writes to disk. Setting this to false would achieve better
	// performance, but may cause data to be lost.
	Sync bool

	// How many versions to keep per key.
	NumVersions int

	// Open the DB as read-only. With this set, multiple processes can
	// open the same Wisckey DB. Note: if the DB being opened had crashed
	// before and has vlog data to be replayed, ReadOnly will cause Open
	// to fail with an appropriate message.
	ReadOnly bool

	// DB-specific logger which will override the global logger.
	Logger Logger

	// Each table (or file) is at most this size.
	MaxTableSize int64

	// Maximum number of levels of compaction.
	MaxLevels int

	// If value size >= this threshold, only store value offsets in lsm tree.
	ValueThreshold int

	// Maximum number of tables to keep in memory, before stalling.
	NumMemtables int

	// Maximum number of Level 0 tables before we start compacting.
	NumLevelZeroTables int

	// If we hit this number of Level 0 tables, we will stall until L0 is
	// compacted away.
	NumLevelZeroTablesStall int

	// Maximum total size for L1.
	LevelOneSize int64

	// Number of compaction workers to run concurrently. Setting this to zero would stop compactions
	// to happen within LSM tree. If set to zero, writes could block forever.
	NumCompactors int
	// contains filtered or unexported fields
}

Options are params for creating DB object.

This package provides DefaultOptions which contains options that should work for most applications. Consider using that as a starting point before customizing it for your own needs.

type TableManifest

type TableManifest struct {
	Level    uint8
	Checksum []byte
}

TableManifest contains information about a specific level in the LSM tree.