README
¶
raftexample
raftexample is an example usage of etcd's raft library. It provides a simple REST API for a key-value store cluster backed by the Raft consensus algorithm.
Getting Started
Building raftexample
Clone etcd to <directory>/src/github.com/CloudcadeSF/etcd
export GOPATH=<directory>
cd <directory>/src/github.com/CloudcadeSF/etcd/contrib/raftexample
go build -o raftexample
Running single node raftexample
First start a single-member cluster of raftexample:
raftexample --id 1 --cluster http://127.0.0.1:12379 --port 12380
Each raftexample process maintains a single raft instance and a key-value server. The process's list of comma separated peers (--cluster), its raft ID index into the peer list (--id), and http key-value server port (--port) are passed through the command line.
Next, store a value ("hello") to a key ("my-key"):
curl -L http://127.0.0.1:12380/my-key -XPUT -d hello
Finally, retrieve the stored key:
curl -L http://127.0.0.1:12380/my-key
Running a local cluster
First install goreman, which manages Procfile-based applications.
The Procfile script will set up a local example cluster. Start it with:
goreman start
This will bring up three raftexample instances.
Now it's possible to write a key-value pair to any member of the cluster and likewise retrieve it from any member.
Fault Tolerance
To test cluster recovery, first start a cluster and write a value "foo":
goreman start
curl -L http://127.0.0.1:12380/my-key -XPUT -d foo
Next, remove a node and replace the value with "bar" to check cluster availability:
goreman run stop raftexample2
curl -L http://127.0.0.1:12380/my-key -XPUT -d bar
curl -L http://127.0.0.1:32380/my-key
Finally, bring the node back up and verify it recovers with the updated value "bar":
goreman run start raftexample2
curl -L http://127.0.0.1:22380/my-key
Dynamic cluster reconfiguration
Nodes can be added to or removed from a running cluster using requests to the REST API.
For example, suppose we have a 3-node cluster that was started with the commands:
raftexample --id 1 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 12380
raftexample --id 2 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 22380
raftexample --id 3 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379 --port 32380
A fourth node with ID 4 can be added by issuing a POST:
curl -L http://127.0.0.1:12380/4 -XPOST -d http://127.0.0.1:42379
Then the new node can be started as the others were, using the --join option:
raftexample --id 4 --cluster http://127.0.0.1:12379,http://127.0.0.1:22379,http://127.0.0.1:32379,http://127.0.0.1:42379 --port 42380 --join
The new node should join the cluster and be able to service key/value requests.
We can remove a node using a DELETE request:
curl -L http://127.0.0.1:12380/3 -XDELETE
Node 3 should shut itself down once the cluster has processed this request.
Design
The raftexample consists of three components: a raft-backed key-value store, a REST API server, and a raft consensus server based on etcd's raft implementation.
The raft-backed key-value store is a key-value map that holds all committed key-values. The store bridges communication between the raft server and the REST server. Key-value updates are issued through the store to the raft server. The store updates its map once raft reports the updates are committed.
The REST server exposes the current raft consensus by accessing the raft-backed key-value store. A GET command looks up a key in the store and returns the value, if any. A key-value PUT command issues an update proposal to the store.
The raft server participates in consensus with its cluster peers. When the REST server submits a proposal, the raft server transmits the proposal to its peers. When raft reaches a consensus, the server publishes all committed updates over a commit channel. For raftexample, this commit channel is consumed by the key-value store.
Documentation
Source Files