raftify

package module
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 27, 2020 License: Apache-2.0 Imports: 17 Imported by: 0

README

Raftify

CircleCI codecov Go Report Card License

⚠ This project has not yet had a security audit or stress test and is therefore not ready for use in production! Use at your own risk!

Raftify is a Go implementation of the Raft leader election algorithm without the Raft log and enables the creation of a self-managing cluster of nodes by transforming an application into a Raft node. It is meant to be a more cost-efficient small-scale alternative to running a validator cluster with a separate full-fledged Raft consensus layer.

It is designed to be directly embedded into an application and provide a direct way of communicating between individual nodes, omitting the overhead caused by replicating a log. Raftify was built with one particular use case in mind: running a self-managing cluster of Cosmos validators.

Requirements

  • Golang 1.14+

Configuration Reference

The configuration is to be provided in a raftify.json file and must be located in the working directory specified in the second parameter of the InitNode method.

ℹ For Gaia, the working directory is ~/.gaiad/config/ by default.

Key Value Description
id string (Mandatory) The node's identifier.
Must be unique.
max_nodes int (Mandatory) The self-imposed limit of nodes to be run in the cluster.
Must be greater than 0 and must never be exceeded.
expect int (Mandatory) The number of nodes expected to be online in order to bootstrap the cluster and start the leader election. Once the expected number of nodes is online, all cluster members will be started simultaneously.
Must be 1 or higher and must never exceed the self-imposed max_nodes limit.
⚠ Please use expect = 1 for single-node setups only. If you plan on running more than one node, set the expect value to the final cluster size on ALL nodes.
encrypt string (Optional) The hex representation of the secret key used to encrypt messages.
The value must be either 16, 24, or 32 bytes to select AES-128, AES-192, or AES-256.
Use this tool to generate a key.
performance int (Optional) The modifier used to multiply the maximum and minimum timeout and ticker settings. Higher values increase leader stability and reduce bandwidth and CPU but also increase the time needed to recover from a leader failure.
Must be 1 or higher. Defaults to 1 which is also the maximum performance setting.
log_level string (Optional) The minimum log level for console log messages.
Can be DEBUG, INFO, WARN, ERR. Defaults to WARN.
bind_addr string (Optional) The address to bind the node application to.
Defaults to 0.0.0.0.
bind_port string (Optional) The port to bind the node application to.
Defaults to 7946.
peer_list []string (Optional) The list of IP addresses of all cluster members (optionally including the address of the local node). It is used to determine the quorum in a non-bootstrapped cluster.
For example, if your peerlist has n = 3 nodes then math.Floor((n/2)+1) = 2 nodes will need to be up and running to bootstrap the cluster.
Addresses must be provided in the host:port format.
Must not be empty if more than one node is expected.
Example Configuration
{
    "id": "My-Unique-Name",
    "max_nodes": 3,
    "expect": 3,
    "encrypt": "8ba4770b00f703fcc9e7d94f857db0e76fd53178d3d55c3e600a9f0fda9a75ad",
    "performance": 1,
    "log_level": "WARN",
    "bind_addr": "192.168.0.25",
    "bind_port": 3000,
    "peer_list": [
        "192.168.0.25:3000",
        "192.168.0.26:3000",
        "192.168.0.27:3000"
    ]
}

Getting Started

For a step-by-step guide on how to get started with your raftified Cosmos validator, check out this tutorial.

Testing

Use

make unit-tests

to run unit tests, and

make integration-tests

to run integration tests.

Documentation

Index

Examples

Constants

View Source 
const (
	// Time interval measured in milliseconds in which candidates send out
	// vote requests and leaders send out heartbeats.
	TickerInterval = 200

	// Minimum time measured in milliseconds that a non-leader waits for a
	// current heartbeat to arrive before switching to a precandidate.
	MinTimeout = 800

	// Maximum time measured in milliseconds that a non-leader waits for a
	// current heartbeat to arrive before switching to a precandidate.
	MaxTimeout = 1200

	// Maximum number of cycles a leader can go without getting heartbeat
	// responses from a majority of cluster members. If there are not enough
	// heartbeat responses for more than cycles than this value, the leader
	// is forced to step down.
	MaxSubQuorumCycles = int(MinTimeout/TickerInterval) - 1

	// Maximum number of cycles a precandidate couldn't reach the quorum
	// in order to become a candidate. If this threshold is met or exceeded
	// it is safe to assume the cluster suffers a network partition and the
	// node in question is partitioned out into a smaller cluster that can
	// never reach the quorum. Upon reaching this threshold, a rejoin event
	// is triggered to make the node in question aware of the network partition.
	MaxMissedPrevoteCycles = 5
)

Timeout and ticker settings for maximum performance.

Variables

This section is empty.

Functions

This section is empty.

Types

type ChannelEventDelegate 

type ChannelEventDelegate struct {
	// contains filtered or unexported fields
}

ChannelEventDelegate is a simpler delegate that is used only to receive notifications about members joining and leaving.

func (*ChannelEventDelegate) NotifyJoin 

func (d *ChannelEventDelegate) NotifyJoin(newNode *memberlist.Node)

NotifyJoin implements the EventDelegate interface.

func (*ChannelEventDelegate) NotifyLeave 

func (d *ChannelEventDelegate) NotifyLeave(oldNode *memberlist.Node)

NotifyLeave implements the EventDelegate interface.

func (*ChannelEventDelegate) NotifyUpdate 

func (d *ChannelEventDelegate) NotifyUpdate(updatedNode *memberlist.Node)

NotifyUpdate implements the EventDelegate interface.

type Config 

type Config struct {
	// Mandatory. The unique identifier of a node.
	ID string `json:"id"`

	// Mandatory. Self-imposed limit of nodes that can be run in one cluster.
	// This is needed to allocate enough memory for the buffered channel used
	// for event messages.
	MaxNodes int `json:"max_nodes"`

	// Mandatory. The 16-, 24- or 32-byte AES encryption key used to encrypt
	// the message exchange between cluster members.
	Encrypt string `json:"encrypt"`

	// The performance multiplier that determines how the timeouts and
	// intervals scale. This can be used to adjust the timeout settings
	// for higher latency environments.
	Performance int `json:"performance"`

	// The number of expected nodes to go online before starting the
	// Raft leader election and bootstrapping the cluster.
	Expect int `json:"expect"`

	// The log levels for raftify; can be DEBUG, INFO, WARN or ERR.
	LogLevel string `json:"log_level"`

	// The address to bind the node to.
	BindAddr string `json:"bind_addr"`

	// The port to bind the node to.
	BindPort int `json:"bind_port"`

	// The list of peers to contact in order to join an existing cluster
	// or form a new one.
	PeerList []string `json:"peer_list"`
}

Config contains the contents of the raftify.json file.

type Heartbeat 

type Heartbeat struct {
	Term        uint64 `json:"term"`
	Quorum      int    `json:"quorum"`
	HeartbeatID uint64 `json:"heartbeat_id"`
	LeaderID    string `json:"leader_id"`
}

Heartbeat defines the message sent out by the leader to all cluster members.

type HeartbeatIDList 

type HeartbeatIDList struct {
	// contains filtered or unexported fields
}

HeartbeatIDList is a custom type for a list of heartbeat IDs. This is needed in order to be able to differentiate between heartbeat responses within their respective ticker cycles and those which came from an outdated cycle and therefore do not count anymore.

type HeartbeatResponse 

type HeartbeatResponse struct {
	Term        uint64 `json:"term"`
	HeartbeatID uint64 `json:"heartbeat_id"`
	FollowerID  string `json:"follower_id"`
}

HeartbeatResponse defines the response of a follower to a leader's heartbeat message.

type Message 

type Message struct {
	Type    MessageType     `json:"type"`
	Content json.RawMessage `json:"content"`
}

Message is a wrapper struct for all messages used to determine the message type.

type MessageDelegate 

type MessageDelegate struct {
	// contains filtered or unexported fields
}

MessageDelegate is the interface that clients must implement if they want to hook into the gossip layer of Memberlist.

func (*MessageDelegate) GetBroadcasts 

func (d *MessageDelegate) GetBroadcasts(overhead, limit int) [][]byte

GetBroadcasts implements the Delegate interface.

func (*MessageDelegate) LocalState 

func (d *MessageDelegate) LocalState(join bool) []byte

LocalState implements the Delegate interface.

func (*MessageDelegate) MergeRemoteState 

func (d *MessageDelegate) MergeRemoteState(buf []byte, join bool)

MergeRemoteState implements the Delegate interface.

func (*MessageDelegate) NodeMeta 

func (d *MessageDelegate) NodeMeta(limit int) []byte

NodeMeta implements the Delegate interface.

func (*MessageDelegate) NotifyMsg 

func (d *MessageDelegate) NotifyMsg(msg []byte)

NotifyMsg implements the Delegate interface.

type MessageType 

type MessageType uint8

MessageType is a custom type for all valid raftify messages. 

const (
	// A heartbeat message is sent out by a leader to signal availability.
	HeartbeatMsg MessageType = iota

	// A heartbeat response message is sent by the node who received the
	// heartbeat to the leader it originated from.
	HeartbeatResponseMsg

	// A prevote request message is sent out by aprecandidate in order to
	// make sure there truly is no more leader and a new candidacy needs to
	// be initiated.
	PreVoteRequestMsg

	// A prevote response message is sent by the node who received the
	// prevote request to the precandidate it originated from.
	PreVoteResponseMsg

	// A vote request message is sent out by a candidate in order to become
	// the new cluster leader.
	VoteRequestMsg

	// A vote response message is sent by the node who received the vote
	// request to the candidate it originated from.
	VoteResponseMsg

	// A new quorum message is sent out by a voluntarily leaving node. It triggers
	// an immediate quorum change instead of having to wait for the cluster to
	// detect and kick the dead node eventually.
	NewQuorumMsg
)

Constants for valid messages.

type NewQuorum added in v0.3.0 

type NewQuorum struct {
	NewQuorum int    `json:"new_quorum"`
	LeavingID string `json:"leaving_id"`
}

NewQuorum defines the message sent out by a node that is voluntarily leaving the cluster, triggering an immediate quorum change. This does not include crash-related leave events.

type Node 

type Node struct {
	// contains filtered or unexported fields
}

Node contains core attributes that every node has regardless of node state.

Example (HandlePreVoteRequest) 
// Initialize and start dummy node
node := initDummyNode("TestNode", 1, 1, 5000)
node.createMemberlist()
defer node.memberlist.Shutdown()

// Make prevote request message
pvr := PreVoteRequest{
	PreCandidateID: "TestNode",
}

// Invalid state, not granted
node.toFollower(0)
node.handlePreVoteRequest(pvr)

// Valid state, not granted
node.toPreCandidate()
node.handlePreVoteRequest(pvr)

// Valid state, granted
pvr.NextTerm = node.currentTerm + 1
node.handlePreVoteRequest(pvr)

Output:

[INFO] raftify: ->[] TestNode [127.0.0.1:5000] joined the cluster.
[INFO] raftify: Entering follower state for term 0
[DEBUG] raftify: Received prevote request from TestNode
[WARN] raftify: received prevote request as Follower
[DEBUG] raftify: Sent prevote response to TestNode (not granted)
[DEBUG] raftify: Entering precandidate state for term 1
[DEBUG] raftify: Received prevote request from TestNode
[DEBUG] raftify: Received outdated prevote request from TestNode, skipping...
[DEBUG] raftify: Sent prevote response to TestNode (not granted)
[DEBUG] raftify: Received prevote request from TestNode
[DEBUG] raftify: Sent prevote response to TestNode (granted)
Example (PrintMemberlist) 
node := initDummyNode("TestNode", 1, 1, 4000)
node.createMemberlist()
node.printMemberlist()

Output:

[INFO] raftify: ->[] TestNode [127.0.0.1:4000] joined the cluster.
[INFO] raftify: The cluster has currently 1 members:
[INFO] raftify: - TestNode [127.0.0.1]

func InitNode 

func InitNode(logger *log.Logger, workingDir string) (*Node, error)

InitNode initializes a new raftified node. Blocks until cluster is successfully bootstrapped.

func (*Node) GetHealthScore 

func (n *Node) GetHealthScore() int

GetHealthScore returns the health score according to memberlist. Lower numbers are better, and 0 means "totally healthy".

func (*Node) GetID 

func (n *Node) GetID() string

GetID returns the node's unique ID.

func (*Node) GetMembers 

func (n *Node) GetMembers() map[string]string

GetMembers returns a map of the current memberlist with a key "id" and a value "address" in the host:port format.

func (*Node) GetState 

func (n *Node) GetState() State

GetState returns the state the node's current state which is either Follower, PreCandidate, Candidate or Leader.

func (*Node) Shutdown 

func (n *Node) Shutdown() error

Shutdown stops all timers/tickers and listeners, closes channels, leaves the memberlist and shuts down the node.

type PreVoteRequest 

type PreVoteRequest struct {
	NextTerm       uint64 `json:"next_term"`
	PreCandidateID string `json:"pre_candidate_id"`
}

PreVoteRequest defines the message sent out by a follower who is about to become a candidate in order to check whether there truly isn't a leader anymore.

type PreVoteResponse 

type PreVoteResponse struct {
	Term           uint64 `json:"term"`
	FollowerID     string `json:"follower_id"`
	PreVoteGranted bool   `json:"pre_vote_granted"`
}

PreVoteResponse defines the response of a follower to a candidate-to-be's pre vote request.

type State 

type State uint8

State is a custom type for all valid raftify node states. 

const (
	// Bootstrap is the state a node is in if it's waiting for the expected number of
	// nodes to go online before starting the Raft leader election.
	Bootstrap State = iota

	// Rejoin is the state a node is in if it times out or crashes and restarts.
	// In this state, it attempts to rejoin the existing cluster it dropped out of.
	Rejoin

	// Followers reset their timeout if they receive a heartbeat message from a leader.
	// If the timeout elapses, they become a precandidate.
	Follower

	// PreCandidates start collecting prevotes in order to determine if any other cluster
	// member has seen a leader and therefore make sure that there truly isn't one anymore
	// and a new one needs to be elected. Once the majority of prevotes have been granted,
	// it becomes a candidate.
	PreCandidate

	// Candidates enter a new election term and start collecting votes in order to be
	// promoted to the new cluster leader. A candidate votes for itself and waits for other
	// nodes to respond to its vote request. Sometimes a split vote can happen which means
	// that there are multiple candidates trying to become leader simultaneously such that
	// there are not enough votes left to reach quorum. In that case, the nodes wait for
	// the next timeout to start a new term. Once the majority of votes have been granted,
	// it becomes a leader.
	Candidate

	// The leader periodically sends out heartbeats to its followers to signal its availability.
	// If it suffers any sort of failure it automatically restarts as a follower. If it's
	// partitioned out and doesn't receive the majority of heartbeat responses it steps down.
	Leader

	// PreShutdown is the state in which a node prepares its own voluntary leave by notifying the
	// rest of the cluster of an immediate quorum change.
	PreShutdown

	// Shutdown is the state in which a node initiates a shutdown and gracefully allows the
	// runLoop goroutine to be exited and killed.
	Shutdown
)

Constants for valid node states.

type VersionInfo added in v0.2.0 

type VersionInfo struct {
	Name      string
	Version   string
	GoVersion string
}

VersionInfo defines version information about Raftify.

func (VersionInfo) GetVersionInfo added in v0.2.0 

func (v VersionInfo) GetVersionInfo() VersionInfo

GetVersionInfo returns version information.

type VoteList 

type VoteList struct {
	// contains filtered or unexported fields
}

VoteList is a custom type for a list of nodes who haven't (pre)voted yet.

type VoteRequest 

type VoteRequest struct {
	Term        uint64 `json:"term"`
	CandidateID string `json:"candidate_id"`
}

VoteRequest defines the message sent out by a candidate to all cluster members to ask for votes in order to become leader.

type VoteResponse 

type VoteResponse struct {
	Term        uint64 `json:"term"`
	FollowerID  string `json:"follower_id"`
	VoteGranted bool   `json:"vote_granted"`
}

VoteResponse defines the response of a follower to a candidate's vote request message.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.