raft

package module
v0.0.0-...-ef52421 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 23, 2018 License: MIT Imports: 14 Imported by: 5

README 

* Raft Consensus with Gorums

This repository contains an implementation of the [[https://raft.github.io/raft.pdf][Raft Consensus Algorithm]] for use with the [[https://github.com/relab/gorums][Gorums framework]].

- github.com/relab/raft :: Shared code and common interfaces.
- github.com/relab/raft/raftgorums :: Raft with Gorums code.

Documentation

Index

Constants

View Source 
const (
	KeyTerm uint64 = iota
	KeyVotedFor
	KeyFirstIndex
	KeyNextIndex
	KeySnapshot
)

Keys for indexing term and who was voted for.

Variables

View Source 
var ErrKeyNotFound = errors.New("key not found")

ErrKeyNotFound means that the given key could not be found in the storage.

View Source 
var NOOP = []byte("noop")

NOOP is used in the data field for a no-op entry.

Functions

func NewPromiseEntry 

func NewPromiseEntry(entry *commonpb.Entry) (PromiseEntry, Future)

NewPromiseEntry returns a PromiseEntry and a Future which can be used to get the response from the promise at a later time.

Types

type CacheStorage 

type CacheStorage struct {
	// contains filtered or unexported fields
}

CacheStorage wraps a Storage adding a layer of caching. It uses the underlying storage as a fallback if the data is not cached.

func NewCacheStorage 

func NewCacheStorage(s Storage, cacheSize int) *CacheStorage

NewCacheStorage returns a new initialized CacheStorage.

func (*CacheStorage) FirstIndex 

func (cs *CacheStorage) FirstIndex() (uint64, error)

FirstIndex implements the Storage interface.

func (*CacheStorage) Get 

func (cs *CacheStorage) Get(key uint64) (uint64, error)

Get implements the Storage interface.

func (*CacheStorage) GetEntries 

func (cs *CacheStorage) GetEntries(first, last uint64) ([]*commonpb.Entry, error)

GetEntries implements the Storage interface.

func (*CacheStorage) GetEntry 

func (cs *CacheStorage) GetEntry(index uint64) (*commonpb.Entry, error)

GetEntry implements the Storage interface.

func (*CacheStorage) GetSnapshot 

func (cs *CacheStorage) GetSnapshot() (*commonpb.Snapshot, error)

GetSnapshot implements the Storage interface.

func (*CacheStorage) NextIndex 

func (cs *CacheStorage) NextIndex() (uint64, error)

NextIndex implements the Storage interface.

func (*CacheStorage) RemoveEntries 

func (cs *CacheStorage) RemoveEntries(first, last uint64) error

RemoveEntries implements the Storage interface.

func (*CacheStorage) Set 

func (cs *CacheStorage) Set(key uint64, value uint64) error

Set implements the Storage interface.

func (*CacheStorage) SetSnapshot 

func (cs *CacheStorage) SetSnapshot(snapshot *commonpb.Snapshot) error

SetSnapshot implements the Storage interface.

func (*CacheStorage) StoreEntries 

func (cs *CacheStorage) StoreEntries(entries []*commonpb.Entry) error

StoreEntries implements the Storage interface.

type ErrNotLeader 

type ErrNotLeader struct {
	Leader uint64
}

ErrNotLeader is returned by Raft when a method is invoked requiring the server to be the leader, and it's not. A hint about the actual leader is provided.

func (ErrNotLeader) Error 

func (e ErrNotLeader) Error() string

type Event 

type Event [][]string

Event is a slice of CSV records.

func NewEvent 

func NewEvent() *Event

NewEvent returns a Event struct initialized with a header record.

func (*Event) Record 

func (e *Event) Record(event EventType)

Record records a new CSV record with time set to time.Now().

func (*Event) Write 

func (e *Event) Write(path string)

Write writes all records to a file.

type EventType 

type EventType int

EventType is the types of event that Event can record. 

const (
	// EventCatchup a follower invoked a catchup.
	EventCatchup EventType = 0
	// EventFailure a server suspected another server of failing.
	EventFailure EventType = 1
	// EventElection an election was initiated.
	EventElection EventType = 2
	// EventPreElection a pre-election was initiated.
	EventPreElection EventType = 3
	// EventBecomeLeader a candidate won an election.
	EventBecomeLeader EventType = 4
	// EventProposeAddServer a leader received a add server request.
	EventProposeAddServer EventType = 5
	// EventProposeRemoveServer a leader received a remove server request.
	EventProposeRemoveServer EventType = 6
	// EventCaughtUp indicates that the a server has caught up to the point
	// where it has applied the configuration change which added it to the
	// cluster.
	EventCaughtUp EventType = 7
	// EventRemoved the remove server request was committed.
	EventRemoved EventType = 8
	// EventAdded the add server request was committed.
	EventAdded EventType = 9
	// EventApplyConfiguration a new configuration is now being used.
	EventApplyConfiguration EventType = 10
	// EventTerminated a server received a termination signal.
	EventTerminated EventType = 11
	// EventStartReplicate the leader started replicating entries to a
	// server.
	EventStartReplicate = 12
	// EventInjectEntries the leader responds to a catchup request by
	// injecting the missing entries in the next request.
	EventInjectEntries = 13
)

type FileStorage 

type FileStorage struct {
	*bolt.DB
}

FileStorage is an implementation of the Storage interface for file based storage.

func NewFileStorage 

func NewFileStorage(path string, overwrite bool) (*FileStorage, error)

NewFileStorage returns a new FileStorage using the file given with the path argument. Overwrite decides whether to use the database if it already exists or overwrite it.

func (*FileStorage) FirstIndex 

func (fs *FileStorage) FirstIndex() (uint64, error)

FirstIndex implements the Storage interface.

func (*FileStorage) Get 

func (fs *FileStorage) Get(key uint64) (uint64, error)

Get implements the Storage interface.

func (*FileStorage) GetEntries 

func (fs *FileStorage) GetEntries(first, last uint64) ([]*commonpb.Entry, error)

GetEntries implements the Storage interface. TODO We can reduce allocation by passing the slice to fill.

func (*FileStorage) GetEntry 

func (fs *FileStorage) GetEntry(index uint64) (*commonpb.Entry, error)

GetEntry implements the Storage interface.

func (*FileStorage) GetSnapshot 

func (fs *FileStorage) GetSnapshot() (*commonpb.Snapshot, error)

GetSnapshot implements the Storage interface.

func (*FileStorage) NextIndex 

func (fs *FileStorage) NextIndex() (uint64, error)

NextIndex implements the Storage interface.

func (*FileStorage) RemoveEntries 

func (fs *FileStorage) RemoveEntries(first, last uint64) error

RemoveEntries implements the Storage interface.

func (*FileStorage) Set 

func (fs *FileStorage) Set(key uint64, value uint64) error

Set implements the Storage interface.

func (*FileStorage) SetSnapshot 

func (fs *FileStorage) SetSnapshot(snapshot *commonpb.Snapshot) error

SetSnapshot implements the Storage interface.

func (*FileStorage) StoreEntries 

func (fs *FileStorage) StoreEntries(entries []*commonpb.Entry) error

StoreEntries implements the Storage interface.

type Future 

type Future interface {
	ResultCh() <-chan Result
}

Future allows a result to be read after the operation who created it has completed.

type Latency 

type Latency [][]string

Latency is a slice of CSV records.

func NewLatency 

func NewLatency() *Latency

NewLatency returns a Latency struct initialized with a header record.

func (*Latency) Record 

func (l *Latency) Record(start time.Time)

Record records a new CSV record with start as the start time and time.Now() as the end time.

func (*Latency) Write 

func (l *Latency) Write(path string)

Write writes all records to a file.

type Memory 

type Memory struct {
	// contains filtered or unexported fields
}

Memory implements the Storage interface as an in-memory storage.

func NewMemory 

func NewMemory(kvstore map[uint64]uint64, log map[uint64]*commonpb.Entry) *Memory

NewMemory returns a memory backed storage.

func (*Memory) FirstIndex 

func (m *Memory) FirstIndex() (uint64, error)

FirstIndex implements the Storage interface.

func (*Memory) Get 

func (m *Memory) Get(key uint64) (uint64, error)

Get implements the Storage interface.

func (*Memory) GetEntries 

func (m *Memory) GetEntries(first, last uint64) ([]*commonpb.Entry, error)

GetEntries implements the Storage interface.

func (*Memory) GetEntry 

func (m *Memory) GetEntry(index uint64) (*commonpb.Entry, error)

GetEntry implements the Storage interface.

func (*Memory) GetSnapshot 

func (m *Memory) GetSnapshot() (*commonpb.Snapshot, error)

GetSnapshot implements the Storage interface.

func (*Memory) NextIndex 

func (m *Memory) NextIndex() (uint64, error)

NextIndex implements the Storage interface.

func (*Memory) RemoveEntries 

func (m *Memory) RemoveEntries(first, last uint64) error

RemoveEntries implements the Storage interface.

func (*Memory) Set 

func (m *Memory) Set(key, value uint64) error

Set implements the Storage interface.

func (*Memory) SetSnapshot 

func (m *Memory) SetSnapshot(*commonpb.Snapshot) error

SetSnapshot implements the Storage interface.

func (*Memory) StoreEntries 

func (m *Memory) StoreEntries(entries []*commonpb.Entry) error

StoreEntries implements the Storage interface.

type PanicStorage 

type PanicStorage struct {
	// contains filtered or unexported fields
}

PanicStorage wraps a Storage with methods that panic instead of returning a error.

func NewPanicStorage 

func NewPanicStorage(s Storage, logger logrus.FieldLogger) *PanicStorage

NewPanicStorage returns a new initialized PanicStorage.

func (*PanicStorage) FirstIndex 

func (ps *PanicStorage) FirstIndex() uint64

FirstIndex calls underlying FirstIndex method and panics if there is any error.

func (*PanicStorage) Get 

func (ps *PanicStorage) Get(key uint64) uint64

Get calls underlying Get method and panics if there is any error.

func (*PanicStorage) GetEntries 

func (ps *PanicStorage) GetEntries(first, last uint64) []*commonpb.Entry

GetEntries calls underlying GetEntries method and panics if there is any error.

func (*PanicStorage) GetEntry 

func (ps *PanicStorage) GetEntry(index uint64) *commonpb.Entry

GetEntry calls underlying GetEntry method and panics if there is any error.

func (*PanicStorage) GetSnapshot 

func (ps *PanicStorage) GetSnapshot() *commonpb.Snapshot

GetSnapshot calls underlying GetSnapshot method and panics if there is any error.

func (*PanicStorage) NextIndex 

func (ps *PanicStorage) NextIndex() uint64

NextIndex calls underlying NextIndex method and panics if there is any error.

func (*PanicStorage) RemoveEntries 

func (ps *PanicStorage) RemoveEntries(first, last uint64)

RemoveEntries calls underlying RemoveEntries method and panics if there is any error.

func (*PanicStorage) Set 

func (ps *PanicStorage) Set(key uint64, value uint64)

Set calls underlying Set method and panics if there is any error.

func (*PanicStorage) SetSnapshot 

func (ps *PanicStorage) SetSnapshot(snapshot *commonpb.Snapshot)

SetSnapshot calls underlying SetSnapshot method and panics if there is any error.

func (*PanicStorage) StoreEntries 

func (ps *PanicStorage) StoreEntries(entries []*commonpb.Entry)

StoreEntries calls underlying StoreEntries method and panics if there is any error.

type PromiseEntry 

type PromiseEntry interface {
	Write(uint64) PromiseLogEntry
	Read() PromiseLogEntry
	Respond(interface{})
}

PromiseEntry is a promise to either write some entry to the log, or read some result from the state machine. Invoking Write turns it into a promise to write to the log, invoking read turns it into the other. Respond can be used to respond early, if we cannot proceed with the request. Respond is non-blocking but must only be invoked once.

type PromiseLogEntry 

type PromiseLogEntry interface {
	Entry() *commonpb.Entry
	Duration() time.Duration
	Respond(interface{})
}

PromiseLogEntry is a promise to execute some request, usually committing an entry to the log. Respond is used to inform a listener (Future) about the result of the promise. Respond is non-blocking but must only be invoked once.

func NewPromiseNoFuture 

func NewPromiseNoFuture(entry *commonpb.Entry) PromiseLogEntry

NewPromiseNoFuture returns a struct implementing the PromiseLogEntry but does nothing when Respond is called.

type Raft 

type Raft interface {
	// ProposeCmd proposes a command. Blocks until Raft handles the message
	// or the context is canceled, i.e., server is busy. Immediately returns
	// an ErrNotLeader error if server isn't the leader. If everything works
	// out the command will be applied to the state machine and the result
	// available through the future returned.
	ProposeCmd(context.Context, []byte) (Future, error)

	// ReadCmd works the same way as ProposeCmd but does not write any
	// entries to the log.
	ReadCmd(context.Context, []byte) (Future, error)

	// ProposeConf proposes a new configuration. Behaves as ProposeCmd.
	ProposeConf(context.Context, *commonpb.ReconfRequest) (Future, error)
}

Raft represents the interface a Raft node needs expose to the application layer.

type Result 

type Result struct {
	Index uint64
	Value interface{}
}

Result contains the index of the committed entry and the accompanied response.

type StateMachine 

type StateMachine interface {
	Apply(*commonpb.Entry) interface{}

	Snapshot() <-chan *commonpb.Snapshot
	Restore(*commonpb.Snapshot)
}

StateMachine provides an interface for state machines using the Raft log. Raft will not call any of these methods concurrently, i.e., your StateMachine implementation does not need to be thread-safe.

type Storage 

type Storage interface {
	Set(key uint64, value uint64) error
	Get(key uint64) (uint64, error)

	// Entries must be stored such that Entry.Index can be used to retrieve
	// that entry in the future.
	StoreEntries([]*commonpb.Entry) error
	// Retrieves entry with Entry.Index == index.
	GetEntry(index uint64) (*commonpb.Entry, error)
	// Get the inclusive range of entries from first to last.
	GetEntries(first, last uint64) ([]*commonpb.Entry, error)
	// Remove the inclusive range of entries from first to last.
	RemoveEntries(first, last uint64) error

	// Should return 1 if not set.
	FirstIndex() (uint64, error)
	// Should return 1 if not set.
	NextIndex() (uint64, error)

	SetSnapshot(*commonpb.Snapshot) error
	GetSnapshot() (*commonpb.Snapshot, error)
}

Storage provides an interface for storing and retrieving Raft state.

Source Files

View all Source files

Directories

Path Synopsis
Package commonpb is a generated protocol buffer package.
 Package commonpb is a generated protocol buffer package.
gorumspb
Package gorums is a generated protocol buffer package.
 Package gorums is a generated protocol buffer package.
raftpb
Package raftpb is a generated protocol buffer package.
 Package raftpb is a generated protocol buffer package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.