README
¶
Documentation
¶
Index ¶
- func Duration(duration time.Duration) time.Duration
- func FSM() raft.FSM
- func FSMs(n int) []raft.FSM
- func Server(t *testing.T, fsm raft.FSM, options ...Option) (*raft.Raft, func())
- func WaitLeader(t testing.TB, raft *raft.Raft, timeout time.Duration)
- type Action
- type Control
- type Dispatch
- type Event
- type Option
- type Term
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Duration ¶
Duration is a convenience to scale the given duration according to the GO_RAFT_TEST_LATENCY environment variable.
Types ¶
type Action ¶
type Action struct {
// contains filtered or unexported fields
}
Action defines what should happen when the event defined in the term occurs.
type Control ¶
type Control struct {
// contains filtered or unexported fields
}
Control the events happening in a cluster of raft servers, such has leadership changes, failures and shutdowns.
func Cluster ¶
func Cluster(t testing.TB, fsms []raft.FSM, options ...Option) (map[raft.ServerID]*raft.Raft, *Control)
Cluster creates n raft servers, one for each of the given FSMs, and returns a Control object that can be used to create deterministic test scenarios, deciding which server is elected as leader and if and when a failure should happen during its term.
Each raft.Raft instance is created with sane test-oriented default dependencies, which include:
- very low configuration timeouts - in-memory transports - in-memory log and stable stores - in-memory snapshot stores
You can tweak the default dependencies using the Config, Transport and LogStore options.
All created raft servers will be part of the cluster and act as voting servers, unless the Servers option is used.
If a GO_RAFT_TEST_LATENCY environment is found, the default configuration timeouts will be scaled up accordingly (useful when running tests on slow hardware). A latency of 1.0 is a no-op, since it just keeps the default values unchanged. A value greater than 1.0 increases the default timeouts by that factor. See also the Duration helper.
func (*Control) Barrier ¶
func (c *Control) Barrier()
Barrier is used to wait for the cluster to settle to a stable state, where all in progress Apply() commands are committed across all FSM associated with servers that are not disconnected and all in progress snapshots and restores have been performed.
Usually you don't wan't to concurrently keep invoking Apply() on the cluster raft instances while Barrier() is running.
func (*Control) Close ¶
func (c *Control) Close()
Close the control for this raft cluster, shutting down all servers and stopping all monitoring goroutines.
It must be called by every test creating a test cluster with Cluster().
func (*Control) Commands ¶
Commands returns the total number of command logs applied by the FSM of the server with the given ID.
Example ¶
A three-server raft cluster is created, the first server gets elected as leader, but when it tries to append the second FSM command log the two followers disconnect just before acknowledging the leader that they have appended the command.
t := &testing.T{}
// Create 3 dummy raft FSMs. This are just for the example, you should
// use you own FSMs implementation.
fsms := rafttest.FSMs(3)
// Create a cluster of 3 raft servers, using the dummy FSMs.
rafts, control := rafttest.Cluster(t, fsms)
defer control.Close()
// Elect the first server as leader, and set it up to lose leadership
// when the second FSM command log is appended to the two follower
// servers. Both servers will append the log, but they will disconnect
// from the leader before they can report the successful append.
control.Elect("0").When().Command(2).Appended().Depose()
// The raft server with server ID "0" is now the leader.
r := rafts["0"]
// Apply the first command log, which succeeds.
if err := r.Apply([]byte{}, time.Second).Error(); err != nil {
log.Fatal("failed to apply first FSM command log", err)
}
// Apply the second command log, which fails.
err := r.Apply([]byte{}, time.Second).Error()
if err == nil {
log.Fatal("applyig the second FSM command log did not fail")
}
if err != raft.ErrLeadershipLost {
log.Fatal("wrong error when applying the second FSM command log", err)
}
// Elect the second server as leader and let it catch up with
// logs. Since the second FSM command log has reached a quorum it will
// be committed everywhere.
control.Elect("1")
control.Barrier()
Output: number of commands applied by the first FSM: 2 number of commands applied by the second FSM: 2 number of commands applied by the third FSM: 2
func (*Control) Depose ¶
func (c *Control) Depose()
Depose the current leader.
When calling this method a leader must have been previously elected with Elect().
It must not be called if the current term has scheduled a depose action with Action.Depose().
func (*Control) Elect ¶
Elect a server as leader.
When calling this method there must be no leader in the cluster and server transports must all be disconnected from eacher.
func (*Control) Restores ¶
Restores returns the total number of restores performed by the FSM of the server with the given ID.
func (*Control) Snapshots ¶
Snapshots returns the total number of snapshots performed by the FSM of the server with the given ID.
Example ¶
A three-server raft cluster is created, the first server gets elected as leader, after it commits the first FSM log command, one of the followers disconnects. The leader applies another four FSM log commands, taking a snapshot after the third is committed, and reconnecting the follower after the fourth is committed. After reconnection the follower will restore its FSM state from the leader's snapshot, since the TrailingLogs config is set to 1 by default.
t := &testing.T{}
// Create 3 dummy raft FSMs. This are just for the example, you should
// use you own FSMs implementation.
fsms := rafttest.FSMs(3)
// Create a cluster of 3 raft servers, using the dummy FSMs.
rafts, control := rafttest.Cluster(t, fsms)
defer control.Close()
// Elect the first server as leader
term := control.Elect("0")
// Set up the leader to take a snapshot after committing the fifth FSM
// command log.
term.When().Command(4).Committed().Snapshot()
// The raft server with server ID "0" is now the leader.
r := rafts["0"]
// Apply four command logs, which all succeed. The fourth log is what
// triggers the snapshot.
for i := 0; i < 4; i++ {
if err := r.Apply([]byte{}, time.Second).Error(); err != nil {
log.Fatal("failed to apply first FSM command log", err)
}
if i == 0 {
term.Disconnect("1")
}
}
// Wait for the cluster to settle, in particular for the snapshot to
// complete.
control.Barrier()
// Apply the fifth log, which will reconnect the disconnected follower.
if err := r.Apply([]byte{}, time.Second).Error(); err != nil {
log.Fatal("failed to apply first FSM command log", err)
}
term.Reconnect("1")
// Wait for the cluster to settle, in particular for all FSMs to catch
// up (the disconnected follower will restore from the snapshot).
control.Barrier()
Output: number of snapshots performed by the first server: 1 number of restores performed by the second server: 1 number of commands applied by the first FSM: 5 number of commands applied by the second FSM: 5 number of commands applied by the third FSM: 5
type Dispatch ¶
type Dispatch struct {
// contains filtered or unexported fields
}
Dispatch defines at which phase of the dispatch process a command log event should fire.
func (*Dispatch) Appended ¶
Appended configures the command log event to occurr when the command log is appended by all followers, but not yet committed by the leader.
type Event ¶
type Event struct {
// contains filtered or unexported fields
}
Event that is expected to happen during a Term.
type Option ¶
type Option func([]*dependencies)
Option can be used to tweak the dependencies of test Raft servers created with Cluster() or Server().
func DiscardLogger ¶
func DiscardLogger() Option
DiscardLogger is a convenience around Config that sets the output stream of raft's logger to ioutil.Discard.
func Latency ¶
Latency is a convenience around Config that scales the values of the various raft timeouts that would be set by default by Cluster.
This option is orthogonal to the GO_RAFT_TEST_LATENCY environment variable. If this option is used and GO_RAFT_TEST_LATENCY is set, they will compound. E.g. passing a factor of 2.0 to this option and setting GO_RAFT_TEST_LATENCY to 3.0 will have the net effect that default timeouts are scaled by a factor of 6.0.
func LogStore ¶
LogStore can be used to create custom log stores.
The given function takes a node index as argument and returns the LogStore that the node should use.
func Servers ¶
Servers can be used to indicate which nodes should be initially part of the created cluster.
If this option is not used, the default is to have all nodes be part of the cluster.
type Term ¶
type Term struct {
// contains filtered or unexported fields
}
A Term holds information about an event that should happen while a certain node is the leader.
func (*Term) Disconnect ¶
Disconnect a follower, which will stop receiving RPCs.
Source Files
Directories