raft

Documentation

Overview

Package raft implements a Consensus component for IPFS Cluster which uses Raft (go-libp2p-raft).

Index

• Constants
• Variables
• func CleanupRaft(dataFolder string, keep int) error
• func LastStateRaw(cfg *Config) (io.Reader, bool, error)
• func SnapshotSave(cfg *Config, newState state.State, pids []peer.ID) error
• type Config
  • func (cfg *Config) ConfigKey() string
  • func (cfg *Config) Default() error
  • func (cfg *Config) GetDataFolder() string
  • func (cfg *Config) LoadJSON(raw []byte) error
  • func (cfg *Config) ToJSON() ([]byte, error)
  • func (cfg *Config) Validate() error
• type Consensus
  • func NewConsensus(host host.Host, cfg *Config, state state.State, staging bool) (*Consensus, error)
  • func (cc *Consensus) AddPeer(pid peer.ID) error
  • func (cc *Consensus) Clean() error
  • func (cc *Consensus) Leader() (peer.ID, error)
  • func (cc *Consensus) LogPin(pin api.Pin) error
  • func (cc *Consensus) LogUnpin(pin api.Pin) error
  • func (cc *Consensus) Peers() ([]peer.ID, error)
  • func (cc *Consensus) Ready() <-chan struct{}
  • func (cc *Consensus) RmPeer(pid peer.ID) error
  • func (cc *Consensus) Rollback(state state.State) error
  • func (cc *Consensus) SetClient(c *rpc.Client)
  • func (cc *Consensus) Shutdown() error
  • func (cc *Consensus) State() (state.State, error)
  • func (cc *Consensus) WaitForSync() error
• type LogOp
  • func (op *LogOp) ApplyTo(cstate consensus.State) (consensus.State, error)
• type LogOpType

Constants

const (
	LogOpPin = iota + 1
	LogOpUnpin
)

Type of consensus operation

Variables

var (
	DefaultDataSubFolder        = "raft"
	DefaultWaitForLeaderTimeout = 15 * time.Second
	DefaultCommitRetries        = 1
	DefaultNetworkTimeout       = 10 * time.Second
	DefaultCommitRetryDelay     = 200 * time.Millisecond
	DefaultBackupsRotate        = 6
)

Configuration defaults

var RaftLogCacheSize = 512

RaftLogCacheSize is the maximum number of logs to cache in-memory. This is used to reduce disk I/O for the recently committed entries.

var RaftMaxSnapshots = 5

RaftMaxSnapshots indicates how many snapshots to keep in the consensus data folder. TODO: Maybe include this in Config. Not sure how useful it is to touch this anyways.

Functions

func CleanupRaft

func CleanupRaft(dataFolder string, keep int) error

CleanupRaft moves the current data folder to a backup location

func LastStateRaw

func LastStateRaw(cfg *Config) (io.Reader, bool, error)

LastStateRaw returns the bytes of the last snapshot stored, its metadata, and a flag indicating whether any snapshot was found.

func SnapshotSave

func SnapshotSave(cfg *Config, newState state.State, pids []peer.ID) error

SnapshotSave saves the provided state to a snapshot in the raft data path. Old raft data is backed up and replaced by the new snapshot. pids contains the config-specified peer ids to include in the snapshot metadata if no snapshot exists from which to copy the raft metadata

Types

type Config

type Config struct {
	config.Saver

	// A folder to store Raft's data.
	DataFolder string

	// InitPeerset provides the list of initial cluster peers for new Raft
	// peers (with no prior state). It is ignored when Raft was already
	// initialized or when starting in staging mode.
	InitPeerset []peer.ID
	// LeaderTimeout specifies how long to wait for a leader before
	// failing an operation.
	WaitForLeaderTimeout time.Duration
	// NetworkTimeout specifies how long before a Raft network
	// operation is timed out
	NetworkTimeout time.Duration
	// CommitRetries specifies how many times we retry a failed commit until
	// we give up.
	CommitRetries int
	// How long to wait between retries
	CommitRetryDelay time.Duration
	// BackupsRotate specifies the maximum number of Raft's DataFolder
	// copies that we keep as backups (renaming) after cleanup.
	BackupsRotate int

	// A Hashicorp Raft's configuration object.
	RaftConfig *hraft.Config
	// contains filtered or unexported fields
}

Config allows to configure the Raft Consensus component for ipfs-cluster. The component's configuration section is represented by ConfigJSON. Config implements the ComponentConfig interface.

func (*Config) ConfigKey

func (cfg *Config) ConfigKey() string

ConfigKey returns a human-friendly indentifier for this Config.

func (*Config) Default

func (cfg *Config) Default() error

Default initializes this configuration with working defaults.

func (*Config) GetDataFolder

func (cfg *Config) GetDataFolder() string

GetDataFolder returns the Raft data folder that we are using.

func (*Config) LoadJSON

func (cfg *Config) LoadJSON(raw []byte) error

LoadJSON parses a json-encoded configuration (see jsonConfig). The Config will have default values for all fields not explicited in the given json object.

func (*Config) ToJSON

func (cfg *Config) ToJSON() ([]byte, error)

ToJSON returns the pretty JSON representation of a Config.

func (*Config) Validate

func (cfg *Config) Validate() error

Validate checks that this configuration has working values, at least in appearance.

type Consensus

type Consensus struct {
	// contains filtered or unexported fields
}

Consensus handles the work of keeping a shared-state between the peers of an IPFS Cluster, as well as modifying that state and applying any updates in a thread-safe manner.

func NewConsensus

func NewConsensus(
	host host.Host,
	cfg *Config,
	state state.State,
	staging bool,
) (*Consensus, error)

NewConsensus builds a new ClusterConsensus component using Raft. The state is used to initialize the Consensus system, so any information in it is discarded once the raft state is loaded. The singlePeer parameter controls whether this Raft peer is be expected to join a cluster or it should run on its own.

func (*Consensus) AddPeer

func (cc *Consensus) AddPeer(pid peer.ID) error

AddPeer adds a new peer to participate in this consensus. It will forward the operation to the leader if this is not it.

func (*Consensus) Clean

func (cc *Consensus) Clean() error

Clean removes all raft data from disk. Next time a full new peer will be bootstrapped.

func (*Consensus) Leader

func (cc *Consensus) Leader() (peer.ID, error)

Leader returns the peerID of the Leader of the cluster. It returns an error when there is no leader.

func (*Consensus) LogPin

func (cc *Consensus) LogPin(pin api.Pin) error

LogPin submits a Cid to the shared state of the cluster. It will forward the operation to the leader if this is not it.

func (*Consensus) LogUnpin

func (cc *Consensus) LogUnpin(pin api.Pin) error

LogUnpin removes a Cid from the shared state of the cluster.

func (*Consensus) Peers

func (cc *Consensus) Peers() ([]peer.ID, error)

Peers return the current list of peers in the consensus. The list will be sorted alphabetically.

func (*Consensus) Ready

func (cc *Consensus) Ready() <-chan struct{}

Ready returns a channel which is signaled when the Consensus algorithm has finished bootstrapping and is ready to use

func (*Consensus) RmPeer

func (cc *Consensus) RmPeer(pid peer.ID) error

RmPeer removes a peer from this consensus. It will forward the operation to the leader if this is not it.

func (*Consensus) Rollback

func (cc *Consensus) Rollback(state state.State) error

Rollback replaces the current agreed-upon state with the state provided. Only the consensus leader can perform this operation.

func (*Consensus) SetClient

func (cc *Consensus) SetClient(c *rpc.Client)

SetClient makes the component ready to perform RPC requets

func (*Consensus) Shutdown

func (cc *Consensus) Shutdown() error

Shutdown stops the component so it will not process any more updates. The underlying consensus is permanently shutdown, along with the libp2p transport.

func (*Consensus) State

func (cc *Consensus) State() (state.State, error)

State retrieves the current consensus State. It may error if no State has been agreed upon or the state is not consistent. The returned State is the last agreed-upon State known by this node.

func (*Consensus) WaitForSync

func (cc *Consensus) WaitForSync() error

WaitForSync waits for a leader and for the state to be up to date, then returns.

type LogOp

type LogOp struct {
	Cid  api.PinSerial
	Type LogOpType
	// contains filtered or unexported fields
}

LogOp represents an operation for the OpLogConsensus system. It implements the consensus.Op interface and it is used by the Consensus component.

func (*LogOp) ApplyTo

func (op *LogOp) ApplyTo(cstate consensus.State) (consensus.State, error)

ApplyTo applies the operation to the State

type LogOpType

type LogOpType int

LogOpType expresses the type of a consensus Operation