raft

package
v0.0.0-...-a52a6fd Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 1, 2020 License: Apache-2.0, MIT, Apache-2.0, + 1 more Imports: 30 Imported by: 0

Documentation

Overview

Package raft implements a Consensus component for IPFS Cluster which uses Raft (go-libp2p-raft).

Index

Constants

View Source 
const (
	LogOpPin = iota + 1
	LogOpUnpin
)

Type of consensus operation

Variables

View Source 
var (
	DefaultDataSubFolder        = "raft"
	DefaultWaitForLeaderTimeout = 15 * time.Second
	DefaultCommitRetries        = 1
	DefaultNetworkTimeout       = 10 * time.Second
	DefaultCommitRetryDelay     = 200 * time.Millisecond
	DefaultBackupsRotate        = 6
	DefaultDatastoreNamespace   = "/r" // from "/raft"
)

Configuration defaults

View Source 
var RaftLogCacheSize = 512

RaftLogCacheSize is the maximum number of logs to cache in-memory. This is used to reduce disk I/O for the recently committed entries.

View Source 
var RaftMaxSnapshots = 5

RaftMaxSnapshots indicates how many snapshots to keep in the consensus data folder. TODO: Maybe include this in Config. Not sure how useful it is to touch this anyways.

Functions

func CleanupRaft 

func CleanupRaft(cfg *Config) error

CleanupRaft moves the current data folder to a backup location

func LastStateRaw 

func LastStateRaw(cfg *Config) (io.Reader, bool, error)

LastStateRaw returns the bytes of the last snapshot stored, its metadata, and a flag indicating whether any snapshot was found.

func OfflineState 

func OfflineState(cfg *Config, store ds.Datastore) (state.State, error)

OfflineState state returns a cluster state by reading the Raft data and writing it to the given datastore which is then wrapped as a state.State. Usually an in-memory datastore suffices. The given datastore should be thread-safe.

func SnapshotSave 

func SnapshotSave(cfg *Config, newState state.State, pids []peer.ID) error

SnapshotSave saves the provided state to a snapshot in the raft data path. Old raft data is backed up and replaced by the new snapshot. pids contains the config-specified peer ids to include in the snapshot metadata if no snapshot exists from which to copy the raft metadata

Types

type Config 

type Config struct {
	config.Saver

	// A folder to store Raft's data.
	DataFolder string

	// InitPeerset provides the list of initial cluster peers for new Raft
	// peers (with no prior state). It is ignored when Raft was already
	// initialized or when starting in staging mode.
	InitPeerset []peer.ID
	// LeaderTimeout specifies how long to wait for a leader before
	// failing an operation.
	WaitForLeaderTimeout time.Duration
	// NetworkTimeout specifies how long before a Raft network
	// operation is timed out
	NetworkTimeout time.Duration
	// CommitRetries specifies how many times we retry a failed commit until
	// we give up.
	CommitRetries int
	// How long to wait between retries
	CommitRetryDelay time.Duration
	// BackupsRotate specifies the maximum number of Raft's DataFolder
	// copies that we keep as backups (renaming) after cleanup.
	BackupsRotate int
	// Namespace to use when writing keys to the datastore
	DatastoreNamespace string

	// A Hashicorp Raft's configuration object.
	RaftConfig *hraft.Config

	// Tracing enables propagation of contexts across binary boundaries.
	Tracing bool
	// contains filtered or unexported fields
}

Config allows to configure the Raft Consensus component for ipfs-cluster. The component's configuration section is represented by ConfigJSON. Config implements the ComponentConfig interface.

func (*Config) ApplyEnvVars 

func (cfg *Config) ApplyEnvVars() error

ApplyEnvVars fills in any Config fields found as environment variables.

func (*Config) ConfigKey 

func (cfg *Config) ConfigKey() string

ConfigKey returns a human-friendly indentifier for this Config.

func (*Config) Default 

func (cfg *Config) Default() error

Default initializes this configuration with working defaults.

func (*Config) GetDataFolder 

func (cfg *Config) GetDataFolder() string

GetDataFolder returns the Raft data folder that we are using.

func (*Config) LoadJSON 

func (cfg *Config) LoadJSON(raw []byte) error

LoadJSON parses a json-encoded configuration (see jsonConfig). The Config will have default values for all fields not explicited in the given json object.

func (*Config) ToDisplayJSON 

func (cfg *Config) ToDisplayJSON() ([]byte, error)

ToDisplayJSON returns JSON config as a string.

func (*Config) ToJSON 

func (cfg *Config) ToJSON() ([]byte, error)

ToJSON returns the pretty JSON representation of a Config.

func (*Config) Validate 

func (cfg *Config) Validate() error

Validate checks that this configuration has working values, at least in appearance.

type Consensus 

type Consensus struct {
	// contains filtered or unexported fields
}

Consensus handles the work of keeping a shared-state between the peers of an IPFS Cluster, as well as modifying that state and applying any updates in a thread-safe manner.

func NewConsensus 

func NewConsensus(
	host host.Host,
	cfg *Config,
	store ds.Datastore,
	staging bool,
) (*Consensus, error)

NewConsensus builds a new ClusterConsensus component using Raft.

Raft saves state snapshots regularly and persists log data in a bolt datastore. Therefore, unless memory usage is a concern, it is recommended to use an in-memory go-datastore as store parameter.

The staging parameter controls if the Raft peer should start in staging mode (used when joining a new Raft peerset with other peers).

The store parameter should be a thread-safe datastore.

func (*Consensus) AddPeer 

func (cc *Consensus) AddPeer(ctx context.Context, pid peer.ID) error

AddPeer adds a new peer to participate in this consensus. It will forward the operation to the leader if this is not it.

func (*Consensus) Clean 

func (cc *Consensus) Clean(ctx context.Context) error

Clean removes the Raft persisted state.

func (*Consensus) Distrust 

func (cc *Consensus) Distrust(ctx context.Context, pid peer.ID) error

Distrust is a no-op.

func (*Consensus) IsTrustedPeer 

func (cc *Consensus) IsTrustedPeer(ctx context.Context, p peer.ID) bool

IsTrustedPeer returns true. In Raft we trust all peers.

func (*Consensus) Leader 

func (cc *Consensus) Leader(ctx context.Context) (peer.ID, error)

Leader returns the peerID of the Leader of the cluster. It returns an error when there is no leader.

func (*Consensus) LogPin 

func (cc *Consensus) LogPin(ctx context.Context, pin *api.Pin) error

LogPin submits a Cid to the shared state of the cluster. It will forward the operation to the leader if this is not it.

func (*Consensus) LogUnpin 

func (cc *Consensus) LogUnpin(ctx context.Context, pin *api.Pin) error

LogUnpin removes a Cid from the shared state of the cluster.

func (*Consensus) Peers 

func (cc *Consensus) Peers(ctx context.Context) ([]peer.ID, error)

Peers return the current list of peers in the consensus. The list will be sorted alphabetically.

func (*Consensus) Ready 

func (cc *Consensus) Ready(ctx context.Context) <-chan struct{}

Ready returns a channel which is signaled when the Consensus algorithm has finished bootstrapping and is ready to use

func (*Consensus) RmPeer 

func (cc *Consensus) RmPeer(ctx context.Context, pid peer.ID) error

RmPeer removes a peer from this consensus. It will forward the operation to the leader if this is not it.

func (*Consensus) Rollback 

func (cc *Consensus) Rollback(state state.State) error

Rollback replaces the current agreed-upon state with the state provided. Only the consensus leader can perform this operation.

func (*Consensus) SetClient 

func (cc *Consensus) SetClient(c *rpc.Client)

SetClient makes the component ready to perform RPC requets

func (*Consensus) Shutdown 

func (cc *Consensus) Shutdown(ctx context.Context) error

Shutdown stops the component so it will not process any more updates. The underlying consensus is permanently shutdown, along with the libp2p transport.

func (*Consensus) State 

func (cc *Consensus) State(ctx context.Context) (state.ReadOnly, error)

State retrieves the current consensus State. It may error if no State has been agreed upon or the state is not consistent. The returned State is the last agreed-upon State known by this node. No writes are allowed, as all writes to the shared state should happen through the Consensus component methods.

func (*Consensus) Trust 

func (cc *Consensus) Trust(ctx context.Context, pid peer.ID) error

Trust is a no-op.

func (*Consensus) WaitForSync 

func (cc *Consensus) WaitForSync(ctx context.Context) error

WaitForSync waits for a leader and for the state to be up to date, then returns.

type LogOp 

type LogOp struct {
	SpanCtx trace.SpanContext `codec:"s,omitempty"`
	TagCtx  []byte            `codec:"t,omitempty"`
	Cid     *api.Pin          `codec:"c,omitempty"`
	Type    LogOpType         `codec:"p,omitempty"`
	// contains filtered or unexported fields
}

LogOp represents an operation for the OpLogConsensus system. It implements the consensus.Op interface and it is used by the Consensus component.

func (*LogOp) ApplyTo 

func (op *LogOp) ApplyTo(cstate consensus.State) (consensus.State, error)

ApplyTo applies the operation to the State

type LogOpType 

type LogOpType int

LogOpType expresses the type of a consensus Operation

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.