logdb

package
v2.0.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 3, 2019 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package logdb implements the persistent log storage used by Dragonboat.

This package is internally used by Dragonboat, applications are not expected to import this package.

Index

Constants

View Source 
const (
	// RocksDBLogDBName is the type name of the rocksdb LogDB.
	RocksDBLogDBName = "rocksdb-logdb"
	// LogDBType is the logdb type name.
	LogDBType = "fully-batched rocksdb"
)
View Source 
const (
	// RDBPatched defines Whether rocksdb is dragonboat patched.
	RDBPatched = false
)

Variables

View Source 
var (

	// RDBContextValueSize defines the size of byte array managed in RDB context.
	RDBContextValueSize uint64 = 1024 * 1024 * 64
)

Functions

func OpenLogDB 

func OpenLogDB(dirs []string, lowLatencyDirs []string) (raftio.ILogDB, error)

OpenLogDB opens a LogDB instance using the default implementation.

Types

type IKvStore 

type IKvStore interface {
	// Close closes the underlying Key-Value store.
	Close() error
	// IterateValue iterates the key range specified by the first key fk and
	// last key lk. The inc boolean flag indicates whether it is inclusive for
	// the last key lk. For each iterated entry, the specified op will be invoked
	// on that key-value pair, the specified op func returns a boolean flag to
	// indicate whether IterateValue should continue to iterate entries.
	IterateValue(fk []byte,
		lk []byte, inc bool, op func(key []byte, data []byte) (bool, error))
	// GetValue queries the value specified the input key, the returned value
	// byte slice is passed to the specified op func.
	GetValue(key []byte, op func([]byte) error) error
	// Save value saves the specified key value pair to the underlying key-value
	// pair.
	SaveValue(key []byte, value []byte) error
	// DeleteValue deletes the key-value pair specified by the input key.
	DeleteValue(key []byte) error
	// GetWriteBatch returns an IWriteBatch object to be used by RDB.
	GetWriteBatch(ctx raftio.IContext) IWriteBatch
	// CommitWriteBatch atomically writes everything included in the write batch
	// to the underlying key-value store.
	CommitWriteBatch(wb IWriteBatch) error
	// RemoveEntries removes entries specified by the range [firstKey, lastKey).
	// RemoveEntries is called in the main execution thread of raft, it is
	// suppose to immediately return without significant delay.
	RemoveEntries(firstKey []byte, lastKey []byte) error
	// Compaction is called by the compaction goroutine to compact the key-value
	// store for the specified range [firstKey, lastKey). This method is expected
	// to complete in the order of seconds.
	Compaction(firstKey []byte, lastKey []byte) error
}

IKvStore is the interface used by the RDB struct to access the underlying Key-Value store.

type IWriteBatch 

type IWriteBatch interface {
	Destroy()
	Put([]byte, []byte)
	Clear()
	Count() int
}

IWriteBatch is the interface representing a write batch capable of atomically writing many key-value pairs to the key-value store.

type LogReader 

type LogReader struct {
	sync.Mutex
	// contains filtered or unexported fields
}

LogReader is the struct used to manage logs that have already been persisted into LogDB.

func NewLogReader 

func NewLogReader(clusterID uint64,
	nodeID uint64, logdb raftio.ILogDB) *LogReader

NewLogReader creates and returns a new LogReader instance.

func (*LogReader) Append 

func (lr *LogReader) Append(entries []pb.Entry) error

Append marks the specified entries as persisted and make them available from logreader.

func (*LogReader) ApplySnapshot 

func (lr *LogReader) ApplySnapshot(snapshot pb.Snapshot) error

ApplySnapshot applies the specified snapshot.

func (*LogReader) Compact 

func (lr *LogReader) Compact(index uint64) error

Compact compacts raft log entries up to index.

func (*LogReader) CreateSnapshot 

func (lr *LogReader) CreateSnapshot(snapshot pb.Snapshot) error

CreateSnapshot keeps the metadata of the specified snapshot.

func (*LogReader) Entries 

func (lr *LogReader) Entries(low uint64,
	high uint64, maxSize uint64) ([]pb.Entry, error)

Entries returns persisted entries between [low, high) with a total limit of up to maxSize bytes.

func (*LogReader) GetRange 

func (lr *LogReader) GetRange() (uint64, uint64)

GetRange returns the index range of all logs managed by the LogReader instance.

func (*LogReader) NodeState 

func (lr *LogReader) NodeState() (pb.State, pb.Membership)

NodeState returns the initial state.

func (*LogReader) SetRange 

func (lr *LogReader) SetRange(firstIndex uint64, length uint64)

SetRange updates the LogReader to reflect what is available in it.

func (*LogReader) SetState 

func (lr *LogReader) SetState(s pb.State)

SetState sets the persistent state.

func (*LogReader) Snapshot 

func (lr *LogReader) Snapshot() pb.Snapshot

Snapshot returns the metadata of the lastest snapshot.

func (*LogReader) Term 

func (lr *LogReader) Term(index uint64) (uint64, error)

Term returns the term of the entry specified by the entry index.

type PooledKey 

type PooledKey struct {
	// contains filtered or unexported fields
}

PooledKey represents keys that are managed by a sync.Pool to be reused.

func NewKey 

func NewKey(sz uint64, pool *sync.Pool) *PooledKey

NewKey creates and returns a new PooledKey instance.

func (*PooledKey) Key 

func (k *PooledKey) Key() []byte

Key returns the []byte of the key.

func (*PooledKey) Release 

func (k *PooledKey) Release()

Release puts the key back to the pool.

func (*PooledKey) SetEntryBatchKey 

func (k *PooledKey) SetEntryBatchKey(clusterID uint64,
	nodeID uint64, batchID uint64)

SetEntryBatchKey sets the key value opf the entry batch.

func (*PooledKey) SetEntryKey 

func (k *PooledKey) SetEntryKey(clusterID uint64, nodeID uint64, index uint64)

SetEntryKey sets the key value to the specified entry key.

func (*PooledKey) SetMaxIndexKey 

func (k *PooledKey) SetMaxIndexKey(clusterID uint64, nodeID uint64)

SetMaxIndexKey sets the key value to the max index record key.

func (*PooledKey) SetStateKey 

func (k *PooledKey) SetStateKey(clusterID uint64, nodeID uint64)

SetStateKey sets the key value to the specified State.

type RDB 

type RDB struct {
	// contains filtered or unexported fields
}

RDB is the struct used to manage rocksdb backed persistent Log stores.

type ShardedRDB 

type ShardedRDB struct {
	// contains filtered or unexported fields
}

ShardedRDB is a LogDB implementation using sharded rocksdb instances. This is the community version of the ShardedRDB known to be less optimal in terms of both IO latency and throughput.

func OpenShardedRDB 

func OpenShardedRDB(dirs []string, lldirs []string) (*ShardedRDB, error)

OpenShardedRDB creates a ShardedRDB instance.

func (*ShardedRDB) Close 

func (mw *ShardedRDB) Close()

Close closes the ShardedRDB instance.

func (*ShardedRDB) DeleteSnapshot 

func (mw *ShardedRDB) DeleteSnapshot(clusterID uint64,
	nodeID uint64, snapshotIndex uint64) error

DeleteSnapshot removes the specified snapshot metadata from the log db.

func (*ShardedRDB) GetBootstrapInfo 

func (mw *ShardedRDB) GetBootstrapInfo(clusterID uint64,
	nodeID uint64) (*pb.Bootstrap, error)

GetBootstrapInfo returns the saved bootstrap info for the given node.

func (*ShardedRDB) GetLogDBThreadContext 

func (mw *ShardedRDB) GetLogDBThreadContext() raftio.IContext

GetLogDBThreadContext return a IContext instance.

func (*ShardedRDB) IterateEntries 

func (mw *ShardedRDB) IterateEntries(ents []pb.Entry,
	size uint64, clusterID uint64, nodeID uint64, low uint64, high uint64,
	maxSize uint64) ([]pb.Entry, uint64, error)

IterateEntries returns a list of saved entries starting with index low up to index high with a max size of maxSize.

func (*ShardedRDB) ListNodeInfo 

func (mw *ShardedRDB) ListNodeInfo() ([]raftio.NodeInfo, error)

ListNodeInfo lists all available NodeInfo found in the log db.

func (*ShardedRDB) ListSnapshots 

func (mw *ShardedRDB) ListSnapshots(clusterID uint64,
	nodeID uint64) ([]pb.Snapshot, error)

ListSnapshots lists all available snapshots associated with the specified raft node.

func (*ShardedRDB) Name 

func (mw *ShardedRDB) Name() string

Name returns the type name of the instance.

func (*ShardedRDB) ReadRaftState 

func (mw *ShardedRDB) ReadRaftState(clusterID uint64,
	nodeID uint64, lastIndex uint64) (*raftio.RaftState, error)

ReadRaftState returns the persistent state of the specified raft node.

func (*ShardedRDB) RemoveEntriesTo 

func (mw *ShardedRDB) RemoveEntriesTo(clusterID uint64,
	nodeID uint64, index uint64) error

RemoveEntriesTo removes entries associated with the specified raft node up to the specified index.

func (*ShardedRDB) SaveBootstrapInfo 

func (mw *ShardedRDB) SaveBootstrapInfo(clusterID uint64,
	nodeID uint64, bootstrap pb.Bootstrap) error

SaveBootstrapInfo saves the specified bootstrap info for the given node.

func (*ShardedRDB) SaveRaftState 

func (mw *ShardedRDB) SaveRaftState(updates []pb.Update,
	ctx raftio.IContext) error

SaveRaftState saves the raft state and logs found in the raft.Update list to the log db.

func (*ShardedRDB) SaveSnapshots 

func (mw *ShardedRDB) SaveSnapshots(updates []pb.Update) error

SaveSnapshots saves all snapshot metadata found in the raft.Update list.

Source Files

View all Source files

Directories

Path Synopsis
Package gorocksdb provides the ability to create and access RocksDB databases.
 Package gorocksdb provides the ability to create and access RocksDB databases.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.