canoe

README

#canoe

canoe was born to get the simplicity of Hashicorp's raft library with the full set of features you get with the raft library in etcd.

On the backend, canoe is a wrapper around etcd-raft.

canoe is currently not considered production ready. There are still several caveats which need to be worked out.

Configuration API

Note: Do not mess with this unless you know what you're doing. Canoe should handle all these requests internally, but some advanced applications - such as arbiters - may wish to do manual adjustment to cluster membership

Endpoint :<APIPort>/peers

POST

Request JSON Data:

• id - The ID of the new canoe node to add to the cluster
• raft_port - The RaftPort of the new canoe node to add to the cluster
• config_port - The APIPort of the new canoe node to add to the cluster

Response JSON Data:

• id - The ID of the canoe node which the request was sent to
• raft_port - The RaftPort of the canoe node which the request was sent to
• config_port - The APIPort of the canoe node which the request was send to
• remote_peers - List of other peers in the canoe cluster with id, raft_port, and config_port as fields

GET

Response JSON Data:

• id - The ID of the canoe node which the request was sent to
• raft_port - The RaftPort of the canoe node which the request was sent to
• config_port - The APIPort of the canoe node which the request was send to
• remote_peers - List of other peers in the canoe cluster with id, raft_port, and config_port as fields

DELETE

Request JSON Data:

• id - The ID of the new canoe node to delete from the cluster

Success - 200

Documentation

Index

• Variables
• func Uint64UUID() uint64
• type FSM
• type FilterFn
• type InitializationBackoffArgs
• type LogData
• type Logger
• type Node
  • func NewNode(args *NodeConfig) (*Node, error)
  • func (rn *Node) Destroy() error
  • func (rn *Node) IsIDRemoved(id uint64) bool
  • func (rn *Node) IsRunning() bool
  • func (rn *Node) Process(ctx context.Context, m raftpb.Message) error
  • func (rn *Node) Propose(data []byte) error
  • func (rn *Node) RegisterObserver(o *Observer)
  • func (rn *Node) ReportSnapshot(id uint64, status raft.SnapshotStatus)
  • func (rn *Node) ReportUnreachable(id uint64)
  • func (rn *Node) Start() error
  • func (rn *Node) Stop() error
  • func (rn *Node) UniqueID() uint64
  • func (rn *Node) UnregisterObserver(o *Observer)
• type NodeConfig
• type Observation
• type Observer
  • func NewObserver(channel chan Observation, filter FilterFn) *Observer
• type SnapshotConfig
• type SnapshotData

Variables

var DefaultInitializationBackoffArgs = &InitializationBackoffArgs{
	InitialInterval:     500 * time.Millisecond,
	RandomizationFactor: .5,
	Multiplier:          2,
	MaxInterval:         5 * time.Second,
	MaxElapsedTime:      2 * time.Minute,
}

DefaultInitializationBackoffArgs are the default backoff args

var DefaultLogger = &logrus.Logger{
	Out: os.Stderr,
	Formatter: &textFormatter{
		Prefix:        "canoe",
		FullTimestamp: true,
	},
	Level: logrus.InfoLevel,
}

DefaultLogger gives a default logger for canoe

var DefaultSnapshotConfig = &SnapshotConfig{
	Interval:             -1 * time.Minute,
	MinCommittedLogs:     0,
	MaxRetainedSnapshots: 0,
}

DefaultSnapshotConfig is what is used for snapshotting when SnapshotConfig isn't specified Note: by default we do not snapshot

var ErrorRemovedFromCluster = errors.New("I have been removed from cluster")

ErrorRemovedFromCluster is returned when an operation failed because this Node has been removed from the cluster

Functions

func Uint64UUID

func Uint64UUID() uint64

Uint64UUID returns a UUID encoded to uint64

Types

type FSM

type FSM interface {

	// Apply is called whenever a new log is committed to raft.
	// The FSM is responsible for applying it in an atomic fashion
	Apply(entry LogData) error

	// Snapshot should return a snapshot in the form of restorable info for the entire FSM
	Snapshot() (SnapshotData, error)

	// Restore should take a snapshot, and use it to restore the state of the FSM
	Restore(snap SnapshotData) error
}

FSM is an interface for what your state machine needs to define in order to be compatible with canoe

type FilterFn

type FilterFn func(o Observation) bool

FilterFn is a function used to filter what events an Observer gets piped

type InitializationBackoffArgs

type InitializationBackoffArgs struct {
	InitialInterval     time.Duration
	Multiplier          float64
	MaxInterval         time.Duration
	MaxElapsedTime      time.Duration
	RandomizationFactor float64
}

InitializationBackoffArgs defines the backoff arguments for initializing a Node into a cluster as attempts to join or bootstrap a cluster are dependent on other nodes

type LogData

type LogData []byte

LogData is the format of data you should expect in Apply operations on the FSM. It is also what you should pass to Propose calls to a Node

type Logger

type Logger interface {
	Debug(v ...interface{})
	Debugf(format string, v ...interface{})

	Error(v ...interface{})
	Errorf(format string, v ...interface{})

	Info(v ...interface{})
	Infof(format string, v ...interface{})

	Warning(v ...interface{})
	Warningf(format string, v ...interface{})

	Fatal(v ...interface{})
	Fatalf(format string, v ...interface{})

	Panic(v ...interface{})
	Panicf(format string, v ...interface{})
}

Logger is a clone of etcd.Logger interface. We have it cloned in case we want to add more functionality

type Node

type Node struct {
	// contains filtered or unexported fields
}

Node is a raft node. It is responsible for communicating with all other nodes on the cluster, and in general doing all the rafty things

func NewNode

func NewNode(args *NodeConfig) (*Node, error)

NewNode creates a new node from the config options

func (*Node) Destroy

func (rn *Node) Destroy() error

Destroy is a HARD stop. It first reconfigures the raft cluster to remove itself(ONLY do this if you are intending to permenantly leave the cluster and know consequences around consensus) - read the raft paper's reconfiguration section before using this. It then halts all running goroutines

WARNING! - Destroy will recursively remove everything under <DataDir>/snap and <DataDir>/wal

func (*Node) IsIDRemoved

func (rn *Node) IsIDRemoved(id uint64) bool

IsIDRemoved fulfills the requirement for rafthttp.Raft interface

func (*Node) IsRunning

func (rn *Node) IsRunning() bool

IsRunning reports if the raft node is running

func (*Node) Process

func (rn *Node) Process(ctx context.Context, m raftpb.Message) error

Process fulfills the requirement for rafthttp.Raft interface

func (*Node) Propose

func (rn *Node) Propose(data []byte) error

Propose asks raft to apply the data to the state machine

func (*Node) RegisterObserver

func (rn *Node) RegisterObserver(o *Observer)

RegisterObserver registers and begins to send observations down an Observer

func (*Node) ReportSnapshot

func (rn *Node) ReportSnapshot(id uint64, status raft.SnapshotStatus)

ReportSnapshot fulfills the requirement for rafthttp.Raft

func (*Node) ReportUnreachable

func (rn *Node) ReportUnreachable(id uint64)

ReportUnreachable fulfills the interface for rafthttp.Raft

func (*Node) Start

func (rn *Node) Start() error

Start starts the raft node

func (*Node) Stop

func (rn *Node) Stop() error

Stop will stop the raft node.

Note: stopping will not remove this node from the cluster. This means that it will affect consensus and quorum

func (*Node) UniqueID

func (rn *Node) UniqueID() uint64

UniqueID returns the unique id for the raft node. This can be useful to get when defining your state machine so you don't have to define a new ID for identification and ownership purposes if your application needs that

func (*Node) UnregisterObserver

func (rn *Node) UnregisterObserver(o *Observer)

UnregisterObserver is called when one no longer needs to look for a particular raft event occuring

type NodeConfig

type NodeConfig struct {
	// If not specified or 0, will autogenerate a new UUID
	// It is typically safe to let canoe autogenerate a UUID
	ID uint64

	// If not specified 0x100 will be used
	ClusterID uint64

	FSM               FSM
	RaftPort          int
	ConfigurationPort int

	// BootstrapPeers is a list of peers which we believe to be part of a cluster we wish to join.
	// For now, this list is ignored if the node is marked as a BootstrapNode
	BootstrapPeers []string

	// BootstrapNode is currently needed when bootstrapping a new cluster, a single node must mark itself
	// as the bootstrap node.
	BootstrapNode bool

	// DataDir is where your data will be persisted to disk
	// for use when either you need to restart a node, or
	// it goes offline and needs to be restarted
	DataDir string

	InitBackoff *InitializationBackoffArgs
	// if nil, then default to no snapshotting
	SnapshotConfig *SnapshotConfig

	Logger Logger
}

NodeConfig exposes all the configuration options of a Node

type Observation

type Observation interface{}

Observation is sent out to each observer. An obeservation can have many different types. It is currently used to detect the successful addition of a node to a cluster during the cluster join or bootstrap phase

type Observer

type Observer struct {
	// contains filtered or unexported fields
}

Observer is a struct responsible for monitoring raft's internal operations if one wants to perform unpredicted operations

func NewObserver

func NewObserver(channel chan Observation, filter FilterFn) *Observer

NewObserver gets an observer. Note, if you aren't actively consuming the observer, the observations will get lost

type SnapshotConfig

type SnapshotConfig struct {

	// How often do you want to Snapshot and compact logs?
	Interval time.Duration

	// If the interval ticks but not enough logs have been commited then ignore
	// the snapshot this interval
	// This can be useful if you expect your snapshot procedure to have an expensive base cost
	MinCommittedLogs uint64

	// MaxRetainedSnapshots specifies how many snapshots you want to save from
	// purging at a given time
	MaxRetainedSnapshots uint
}

SnapshotConfig defines when you want raft to take a snapshot and compact the WAL

type SnapshotData

type SnapshotData []byte

SnapshotData defines what a snapshot should look like