raftbadgerdb

README

raft-badger

This repository provides a storage backend for the excellent raft package from Hashicorp. Raft is a distributed consensus protocol. Distributed consensus has many practical applications, ranging from fault-tolerant databases to clock synchronization to things like Google's PageRank.

This package exports the BadgerStore, which is an implementation of both a LogStore and StableStore (interfaces used by the raft package for reading/writing logs as part of its consensus protocol).

• raft-badger
  • installation
  • usage
  • developing
  • motivation
  • misc.
  • todo

installation

go get -u github.com/markthethomas/raft-badger

usage

Create a new BadgerStore and pass it to Raft when setting up.

//...
var logStore raft.LogStore
var stableStore raft.StableStore
myPath  := filepath.Join(s.RaftDir) // replace this with what you're actually using
badgerDB, err := raftbadgerdb.NewBadgerStore(myPath)
if err != nil {
  return fmt.Errorf("error creating new badger store: %s", err)
}
logStore = badgerDB
stableStore = badgerDB

r, err := raft.NewRaft(config, (*fsm)(s), logStore, stableStore, snapshots, transport)
//...

developing

To run tests, run:

go test -cover -coverprofile=coverage.out .

To view coverage, run:

go tool cover -html=coverage.out

To run the benchmark, run:

go test -race -bench .

motivation

This package is meant to be used with the raft package from Hashicorb. This package borrows heavily from the excellent raft-boltdb package, also from Hashicorp. I wanted to learn about Badger and similar tools and needed to use Raft + a durable backend.

misc.

• raft-badger uses prefix keys to "bucket" logs and config, avoiding the need for multiple badger database files for each type of k/v raft sets
• encodes/decodes the raft Log types using Go's gob for efficient encoding/decoding of keys See more at https://blog.golang.org/gobs-of-data.
• images used are from the raft website and the badger repository, respectively
• thanks to the authors of the excellent raft-boltdb package for providing patterns to follow in satisfying the requisite raft interfaces 🙌
• curious to learn more about the raft protocol? check out the raft website. There's also a beginner's guide at Free Code Camp

todo

• support custom badger options
• explore other encodings besides gob
• add more examples of use with raft

Documentation

Index

• Variables
• type BadgerStore
  • func New(options Options) (*BadgerStore, error)
  • func NewBadgerStore(path string) (*BadgerStore, error)
  • func (b *BadgerStore) Close() error
  • func (b *BadgerStore) DeleteRange(min, max uint64) error
  • func (b *BadgerStore) FirstIndex() (uint64, error)
  • func (b *BadgerStore) Get(k []byte) ([]byte, error)
  • func (b *BadgerStore) GetLog(idx uint64, log *raft.Log) error
  • func (b *BadgerStore) GetUint64(key []byte) (uint64, error)
  • func (b *BadgerStore) LastIndex() (uint64, error)
  • func (b *BadgerStore) Set(k, v []byte) error
  • func (b *BadgerStore) SetUint64(key []byte, val uint64) error
  • func (b *BadgerStore) StoreLog(log *raft.Log) error
  • func (b *BadgerStore) StoreLogs(logs []*raft.Log) error
• type Options

Variables

var (

	// ErrKeyNotFound is an error indicating a given key does not exist
	ErrKeyNotFound = errors.New("not found")
)

Types

type BadgerStore

type BadgerStore struct {
	// contains filtered or unexported fields
}

BadgerStore provides access to Badger for Raft to store and retrieve log entries. It also provides key/value storage, and can be used as a LogStore and StableStore. See https://godoc.org/github.com/hashicorp/raft#StableStore and https://godoc.org/github.com/hashicorp/raft#LogStore

func New

func New(options Options) (*BadgerStore, error)

New uses the supplied options to open a badger db and prepare it for use as a raft backend.

func NewBadgerStore

func NewBadgerStore(path string) (*BadgerStore, error)

NewBadgerStore takes a file path and returns a connected Raft backend.

func (*BadgerStore) Close

func (b *BadgerStore) Close() error

Close is used to gracefully close the DB connection.

func (*BadgerStore) DeleteRange

func (b *BadgerStore) DeleteRange(min, max uint64) error

DeleteRange is used to delete logs within a given range inclusively.

func (*BadgerStore) FirstIndex

func (b *BadgerStore) FirstIndex() (uint64, error)

FirstIndex returns the first known index from the Raft log.

func (*BadgerStore) Get

func (b *BadgerStore) Get(k []byte) ([]byte, error)

Get is used to retrieve a value from the k/v store by key

func (*BadgerStore) GetLog

func (b *BadgerStore) GetLog(idx uint64, log *raft.Log) error

GetLog is used to retrieve a log from Badger at a given index.

func (*BadgerStore) GetUint64

func (b *BadgerStore) GetUint64(key []byte) (uint64, error)

GetUint64 is like Get, but handles uint64 values

func (*BadgerStore) LastIndex

func (b *BadgerStore) LastIndex() (uint64, error)

LastIndex returns the last known index from the Raft log.

func (*BadgerStore) Set

func (b *BadgerStore) Set(k, v []byte) error

Set is used to set a key/value set outside of the raft log

func (*BadgerStore) SetUint64

func (b *BadgerStore) SetUint64(key []byte, val uint64) error

SetUint64 is like Set, but handles uint64 values

func (*BadgerStore) StoreLog

func (b *BadgerStore) StoreLog(log *raft.Log) error

StoreLog is used to store a single raft log

func (*BadgerStore) StoreLogs

func (b *BadgerStore) StoreLogs(logs []*raft.Log) error

StoreLogs is used to store a set of raft logs

type Options

type Options struct {
	// BadgerOptions contains any Badger-specific options
	BadgerOptions *badger.Options
	// Path is the directory
	Path string
}

Options contains all the configuration used to open BadgerDB