rosedb

Documentation

Index

• Constants
• Variables
• type Batch
  • func (b *Batch) Commit() error
  • func (b *Batch) Delete(key []byte) error
  • func (b *Batch) Exist(key []byte) (bool, error)
  • func (b *Batch) Get(key []byte) ([]byte, error)
  • func (b *Batch) Put(key []byte, value []byte) error
  • func (b *Batch) Rollback() error
• type BatchOptions
• type DB
  • func Open(options Options) (*DB, error)
  • func (db *DB) Close() error
  • func (db *DB) Delete(key []byte) error
  • func (db *DB) Exist(key []byte) (bool, error)
  • func (db *DB) Get(key []byte) ([]byte, error)
  • func (db *DB) Merge() error
  • func (db *DB) NewBatch(options BatchOptions) *Batch
  • func (db *DB) Put(key []byte, value []byte) error
  • func (db *DB) Stat() *Stat
  • func (db *DB) Sync() error
  • func (db *DB) Watch() (chan *Event, error)
• type Event
• type IndexRecord
• type IteratorOptions
• type LogRecord
• type LogRecordType
• type Options
• type Stat
• type WatchActionType
• type Watcher
  • func NewWatcher(capacity uint64) *Watcher

Constants

const (
	B  = 1
	KB = 1024 * B
	MB = 1024 * KB
	GB = 1024 * MB
)

Variables

var (
	ErrKeyIsEmpty      = errors.New("the key is empty")
	ErrKeyNotFound     = errors.New("key not found in database")
	ErrDatabaseIsUsing = errors.New("the database directory is used by another process")
	ErrReadOnlyBatch   = errors.New("the batch is read only")
	ErrBatchCommitted  = errors.New("the batch is committed")
	ErrBatchRollbacked = errors.New("the batch is rollbacked")
	ErrDBClosed        = errors.New("the database is closed")
	ErrMergeRunning    = errors.New("the merge operation is running")
	ErrWatchDisabled   = errors.New("the watch is disabled")
)

var DefaultBatchOptions = BatchOptions{
	Sync:     true,
	ReadOnly: false,
}

var DefaultIteratorOptions = IteratorOptions{
	Prefix:  nil,
	Reverse: false,
}

var DefaultOptions = Options{
	DirPath:        tempDBDir(),
	SegmentSize:    1 * GB,
	BlockCache:     0,
	Sync:           false,
	BytesPerSync:   0,
	WatchQueueSize: 0,
}

Types

type Batch

type Batch struct {
	// contains filtered or unexported fields
}

Batch is a batch operations of the database. If readonly is true, you can only get data from the batch by Get method. An error will be returned if you try to use Put or Delete method.

If readonly is false, you can use Put and Delete method to write data to the batch. The data will be written to the database when you call Commit method.

Batch is not a transaction, it does not guarantee isolation. But it can guarantee atomicity, consistency and durability(if the Sync options is true).

You must call Commit method to commit the batch, otherwise the DB will be locked.

func (*Batch) Commit

func (b *Batch) Commit() error

Commit commits the batch, if the batch is readonly or empty, it will return directly.

It will iterate the pendingWrites and write the data to the database, then write a record to indicate the end of the batch to guarantee atomicity. Finally, it will write the index.

func (*Batch) Delete

func (b *Batch) Delete(key []byte) error

Delete marks a key for deletion in the batch.

func (*Batch) Exist

func (b *Batch) Exist(key []byte) (bool, error)

Exist checks if the key exists in the database.

func (*Batch) Get

func (b *Batch) Get(key []byte) ([]byte, error)

Get retrieves the value associated with a given key from the batch.

func (*Batch) Put

func (b *Batch) Put(key []byte, value []byte) error

Put adds a key-value pair to the batch for writing.

func (*Batch) Rollback

func (b *Batch) Rollback() error

Rollback discards a uncommitted batch instance. the discard operation will clear the buffered data and release the lock.

type BatchOptions

type BatchOptions struct {
	// Sync has the same semantics as Options.Sync.
	Sync bool
	// ReadOnly specifies whether the batch is read only.
	ReadOnly bool
}

BatchOptions specifies the options for creating a batch.

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB represents a ROSEDB database instance. It is built on the bitcask model, which is a log-structured storage. It uses WAL to write data, and uses an in-memory index to store the key and the position of the data in the WAL, the index will be rebuilt when the database is opened.

The main advantage of ROSEDB is that it is very fast to write, read, and delete data. Because it only needs one disk IO to complete a single operation.

But since we should store all keys and their positions(index) in memory, our total data size is limited by the memory size.

So if your memory can almost hold all the keys, ROSEDB is the perfect stroage engine for you.

func Open

func Open(options Options) (*DB, error)

Open a database with the specified options. If the database directory does not exist, it will be created automatically.

Multiple processes can not use the same database directory at the same time, otherwise it will retrun ErrDatabaseIsUsing.

It will open the wal files in the database directory and load the index from them. Return the DB instance, or an error if any.

func (*DB) Close

func (db *DB) Close() error

Close the database, close all data files and release file lock. Set the closed flag to true. The DB instance cannot be used after closing.

func (*DB) Delete

func (db *DB) Delete(key []byte) error

Delete the specified key from the database. Actually, it will open a new batch and commit it. You can think the batch has only one Delete operation.

func (*DB) Exist

func (db *DB) Exist(key []byte) (bool, error)

Exist checks if the specified key exists in the database. Actually, it will open a new batch and commit it. You can think the batch has only one Exist operation.

func (*DB) Get

func (db *DB) Get(key []byte) ([]byte, error)

Get the value of the specified key from the database. Actually, it will open a new batch and commit it. You can think the batch has only one Get operation.

func (*DB) Merge

func (db *DB) Merge() error

Merge merges all the data files in the database. It will iterate all the data files, find the valid data, and rewrite the data to the new data file.

Merge operation maybe a very time-consuming operation when the database is large. So it is recommended to perform this operation when the database is idle.

func (*DB) NewBatch

func (db *DB) NewBatch(options BatchOptions) *Batch

NewBatch creates a new Batch instance.

func (*DB) Put

func (db *DB) Put(key []byte, value []byte) error

Put a key-value pair into the database. Actually, it will open a new batch and commit it. You can think the batch has only one Put operation.

func (*DB) Stat

func (db *DB) Stat() *Stat

Stat returns the statistics of the database.

func (*DB) Sync

func (db *DB) Sync() error

Sync all data files to the underlying storage.

func (*DB) Watch

func (db *DB) Watch() (chan *Event, error)

type Event

type Event struct {
	Action  WatchActionType
	Key     []byte
	Value   []byte
	BatchId uint64
}

Event is the event that occurs when the database is modified. It is used to synchronize the watch of the database.

type IndexRecord

type IndexRecord struct {
	// contains filtered or unexported fields
}

IndexRecord is the index record of the key. It contains the key, the record type and the position of the record in the wal. Only used in start up to rebuild the index.

type IteratorOptions

type IteratorOptions struct {
	// Prefix filters the keys by prefix.
	Prefix []byte

	// Reverse indicates whether the iterator is reversed.
	// false is forward, true is backward.
	Reverse bool
}

IteratorOptions is the options for the iterator.

type LogRecord

type LogRecord struct {
	Key     []byte
	Value   []byte
	Type    LogRecordType
	BatchId uint64
}

LogRecord is the log record of the key/value pair. It contains the key, the value, the record type and the batch id It will be encoded to byte slice and written to the wal.

type LogRecordType

type LogRecordType = byte

LogRecordType is the type of the log record.

const (
	// LogRecordNormal is the normal log record type.
	LogRecordNormal LogRecordType = iota
	// LogRecordDeleted is the deleted log record type.
	LogRecordDeleted
	// LogRecordBatchFinished is the batch finished log record type.
	LogRecordBatchFinished
)

type Options

type Options struct {
	// DirPath specifies the directory path where the WAL segment files will be stored.
	DirPath string

	// SegmentSize specifies the maximum size of each segment file in bytes.
	SegmentSize int64

	// BlockCache specifies the size of the block cache in number of bytes.
	// A block cache is used to store recently accessed data blocks, improving read performance.
	// If BlockCache is set to 0, no block cache will be used.
	BlockCache uint32

	// Sync is whether to synchronize writes through os buffer cache and down onto the actual disk.
	// Setting sync is required for durability of a single write operation, but also results in slower writes.
	//
	// If false, and the machine crashes, then some recent writes may be lost.
	// Note that if it is just the process that crashes (machine does not) then no writes will be lost.
	//
	// In other words, Sync being false has the same semantics as a write
	// system call. Sync being true means write followed by fsync.
	Sync bool

	// BytesPerSync specifies the number of bytes to write before calling fsync.
	BytesPerSync uint32

	// WatchQueueSize the cache length of the watch queue.
	// if the size greater than 0, which means enable the watch.
	WatchQueueSize uint64
}

Options specifies the options for opening a database.

type Stat

type Stat struct {
	// Total number of keys
	KeysNum int
	// Total disk size of database directory
	DiskSize int64
}

Stat represents the statistics of the database.

type WatchActionType

type WatchActionType = byte

const (
	WatchActionPut WatchActionType = iota
	WatchActionDelete
)

type Watcher

type Watcher struct {
	// contains filtered or unexported fields
}

Watcher temporarily stores event information, as it is generated until it is synchronized to DB's watch.

If the event is overflow, It will remove the oldest data, even if event hasn't been read yet.

func NewWatcher

func NewWatcher(capacity uint64) *Watcher