epaxos

package module
v0.0.0-...-16e5038 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 7, 2019 License: MIT Imports: 23 Imported by: 0

README

ePaxos

Build Status GoDoc

Actor based implementation of the ePaxos consensus algorithm.

Quick Start

To install this implementation of ePaxos on a system:

$ go get github.com/bbengfort/epaxos/...

This should install the epaxos command on your system. Create a configuration file that defines the peers for the network and other parameters as follows:

{
  "timeout": "100ms",
  "aggregate": true,
  "log_level": 5,
  "peers": [
    {
      "pid": 1,
      "name": "alpha",
      "ip_address": "127.0.0.1",
      "domain": "localhost",
      "port": 3264
    },
    {
      "pid": 2,
      "name": "bravo",
      "ip_address": "127.0.0.1",
      "domain": "localhost",
      "port": 3265
    },
    {
      "pid": 3,
      "name": "charlie",
      "ip_address": "127.0.0.1",
      "domain": "localhost",
      "port": 3266
    }
  ]
}

The configuration file can be stored as a .toml, .json, .yml, .yaml file in the following locations:

  • /etc/epaxos.json
  • ~/.epaxos.json
  • $(pwd)/epaxos.json

Or the path to the configuration file can be passed to the command at runtime. To run an ePaxos replica process:

$ epaxos serve -n alpha

The -n command specifies which peer configures the local replica, by default if no name is specified, then the hostname of the machine is used. To commit a command to the ePaxos log:

$ epaxos commit -k "key" -v "value"

This commits the command named "key" with the specified "value" to the log. Note that the client is automatically redirected to a leader in a round-robin fashion and requires the same configuration to connect.

Documentation

Overview

Package epaxos implements the ePaxos consensus algorithm.

Index

Constants

View Source 
const (
	LogTrace uint8 = iota
	LogDebug
	LogInfo
	LogCaution
	LogStatus
	LogWarn
	LogSilent
)

Levels for implementing the debug and trace message functionality.

View Source 
const CautionThreshold = 80

CautionThreshold for issuing caution logs after accumulating cautions.

View Source 
const DefaultRetries = 3

DefaultRetries specifies the number of times to attempt a commit.

View Source 
const MessageBufferSize = 1024

MessageBufferSize represents the number of messages that can be queued to send to the remote server before the process starts to block to prevent back pressure.

View Source 
const PackageVersion = "0.1"

PackageVersion of the current ePaxos implementation

Variables

View Source 
var (
	ErrEventTypeError   = errors.New("captured event with wrong value type")
	ErrEventSourceError = errors.New("captured event with wrong source type")
	ErrUnknownState     = errors.New("epaxos in an unknown state")
	ErrNotListening     = errors.New("replica is not listening for events")
	ErrRetries          = errors.New("could not connect after several attempts")
	ErrNoNetwork        = errors.New("no network specified in the configuration")
	ErrBenchmarkMode    = errors.New("specify either fixed duration or maximum operations benchmark mode")
	ErrBenchmarkRun     = errors.New("benchmark has already been run")
)

Standard errors for primary operations.

Functions

func LogLevel 

func LogLevel() string

LogLevel returns a string representation of the current level

func SetLogLevel 

func SetLogLevel(level uint8)

SetLogLevel modifies the log level for messages at runtime. Ensures that the highest level that can be set is the trace level.

func SetLogger 

func SetLogger(l *log.Logger)

SetLogger sets the logger for writing output to. Can set to a noplog to remove all log messages (or set the log level to silent).

Types

type Actor 

type Actor interface {
	Listen() error        // Run the actor model listen for events and handle them
	Close() error         // Stop the actor from receiving new events (handles remaining pending events)
	Dispatch(Event) error // Outside callers can dispatch events to the actor
	Handle(Event) error   // Handler method for each event in sequence
}

Actor objects listen for events (messages) and then can create more actors, send more messages or make local decisions that modify their own private state. Actors implement lockless concurrent operation (indeed, none of the structs in this package implement mutexes and are not thread safe independently). Concurrency here is based on the fact that only a single actor is initialized and reads event objects one at a time off of a buffered channel. All actor methods should be private as a result so they are not called from other threads.

func NewActor 

func NewActor(callback Callback) Actor

NewActor returns a new simple actor that passes events one at a time to the callback function specified by looping on an internal buffered channel so that event dispatchers are not blocked.

func NewLocker 

func NewLocker(callback Callback) Actor

NewLocker returns a new simple actor that passes events one at a time to the callback function specified by locking on every dispatch call so that event dispatchers must wait until the event is successfully dispatched.

type Callback 

type Callback func(Event) error

Callback is a function that can receive events.

type Client 

type Client struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Client maintains network information embedded in the configuration to connect to an ePaxos consensus quorum and make propose requests.

func NewClient 

func NewClient(remote string, options *Config) (client *Client, err error)

NewClient creates a new ePaxos client to connect to a quorum.

func (*Client) Del 

func (c *Client) Del(key string) error

Del a value for a key (execute a delete operation)

func (*Client) Get 

func (c *Client) Get(key string) ([]byte, error)

Get a value for a key (execute a read operation)

func (*Client) Propose 

func (c *Client) Propose(access pb.AccessType, key string, value []byte) (rep *pb.ProposeReply, err error)

Propose an operation to be applied to the state store.

func (*Client) Put 

func (c *Client) Put(key string, value []byte, execute bool) error

Put a value for a key (execute a write operation)

type ComplexValidator 

type ComplexValidator struct {
	TagName string
}

ComplexValidator validates complex types that multiconfig doesn't understand

func (*ComplexValidator) Validate 

func (v *ComplexValidator) Validate(s interface{}) error

Validate implements the multiconfig.Validator interface.

type Config 

type Config struct {
	Name      string       `required:"false" json:"name,omitempty"`             // unique name of the local replica, hostname by default
	Seed      int64        `required:"false" json:"seed,omitempty"`             // random seed to initialize random generator
	Timeout   string       `default:"500ms" validate:"duration" json:"timeout"` // timeout to wait for responses (parseable duration)
	Aggregate bool         `default:"false" json:"aggregate"`                   // aggregate operations from multiple concurrent clients
	Thrifty   bool         `default:"false" json:"thrifty"`                     // whether or not to send thrifty quorum messages
	LogLevel  int          `default:"3" validate:"uint" json:"log_level"`       // verbosity of logging, lower is more verbose
	Peers     []peers.Peer `json:"peers"`                                       // definition of all hosts on the network

	// Experimental configuration
	// TODO: remove after benchmarks
	Uptime  string `required:"false" validate:"duration" json:"uptime"` // run for a time limit and then shutdown
	Metrics string `requred:"false" json:"metrics"`                     // location to write benchmarks to disk
}

Config uses the multiconfig loader and validators to store configuration values required to run ePaxos. Configuration can be stored as a JSON, TOML, or YAML file in the current working directory as epaxos.json, in the user's home directory as .epaxos.json or in /etc/epaxos.json (with the extension of the file format of choice). Configuration can also be added from the environment using environment variables prefixed with $EPAXOS_ and the all caps version of the configuration name.

func (*Config) GetName 

func (c *Config) GetName() (name string, err error)

GetName returns the name of the local host defined by the configuration or using the hostname by default.

func (*Config) GetPath 

func (c *Config) GetPath() (string, error)

GetPath searches possible configuration paths returning the first path it finds; this path is used when loading the configuration from disk. An error is returned if no configuration file exists.

func (*Config) GetPeer 

func (c *Config) GetPeer() (peers.Peer, error)

GetPeer returns the local peer configuration or an error if no peer is found in the configuration. If the name is not set on the configuration, the hostname is used.

func (*Config) GetQuorum 

func (c *Config) GetQuorum() uint32

GetQuorum returns the number of replicas required for a quourm based on the peers defined in the configuration.

func (*Config) GetRemotes 

func (c *Config) GetRemotes() (remotes []peers.Peer, err error)

GetRemotes returns all peer configurations for remote hosts on the network, e.g. by excluding the local peer configuration.

func (*Config) GetThrifty 

func (c *Config) GetThrifty() []uint32

GetThrifty returns the peers to send broadcast messages to. If not thrifty, it returns nil, otherwise it returns the next n peers by PID where n is one less than the majority of replicas.

func (*Config) GetTimeout 

func (c *Config) GetTimeout() (time.Duration, error)

GetTimeout parses the timeout duration and returns it.

func (*Config) GetUptime 

func (c *Config) GetUptime() (time.Duration, error)

GetUptime parses the uptime duration and returns it.

func (*Config) Load 

func (c *Config) Load() error

Load the configuration from default values, then from a configuration file, and finally from the environment. Validate the configuration when loaded.

func (*Config) Update 

func (c *Config) Update(o *Config) error

Update the configuration from another configuration struct

func (*Config) Validate 

func (c *Config) Validate() error

Validate the loaded configuration using the multiconfig multi validator.

type Event 

type Event interface {
	Type() EventType
	Source() interface{}
	Value() interface{}
}

Event represents actions that occur during consensus. Listeners can register callbacks with event handlers for specific event types.

type EventType 

type EventType uint16

EventType is an enumeration of the kind of events that can occur. 

const (
	UnknownEvent EventType = iota
	ErrorEvent
	MessageEvent
	ProposeRequestEvent
	PreacceptRequestEvent
	PreacceptReplyEvent
	AcceptRequestEvent
	AcceptReplyEvent
	CommitRequestEvent
	CommitReplyEvent
	BeaconRequestEvent
	BeaconReplyEvent
)

Event types represented in ePaxos

func (EventType) String 

func (t EventType) String() string

String returns the name of event types

type Logs 

type Logs struct {
	// contains filtered or unexported fields
}

Logs is a 2D array that maintains the state of all replicas by keeping track of Instances of operations that can be applied to the cluster state. This type exposes a number of helpful methods to interact with the log and sequence numbers.

An instance in a log is identified by (pid, slot) where the PID is the id of the replica who is the quorum leader for the instance and the slot is the index in the log for that replica where the instance resides. Instances are further defined by sequence numbers and dependencies. Sequence numbers expose the order that the instance was applied to this particular log, and dependencies order all instnaces with respect to all internal logs.

func NewLog 

func NewLog(config *Config) *Logs

NewLog creates a new 2D log for epaxos.

func (*Logs) Create 

func (l *Logs) Create(replica uint32, ops []*pb.Operation) (inst *pb.Instance, err error)

Create an instance based on the current log state and append it. This is a helper method for quickly adding a set of operations to the log in a consistent manner.

func (*Logs) Get 

func (l *Logs) Get(replica uint32, slot uint64) (inst *pb.Instance, err error)

Get an instance in the specified replica's log at the specified index.

func (*Logs) Insert 

func (l *Logs) Insert(inst *pb.Instance) (err error)

Insert an instance into a log into the replica/slot specified by the instnace. Will return an error if there is already an instance in that slot or if inserting does not act like appending (e.g. the inst slot is not the next slot).

func (*Logs) LastApplied 

func (l *Logs) LastApplied(replica uint32) (idx uint64, err error)

LastApplied returns the slot of the last applied instance for the specified replica.

type Remote 

type Remote struct {
	sync.Mutex
	peers.Peer
	// contains filtered or unexported fields
}

Remote maintains a connection to a peer on the network.

func NewRemote 

func NewRemote(p peers.Peer, r *Replica) (*Remote, error)

NewRemote creates a new remote associated with the replica.

func (*Remote) Close 

func (c *Remote) Close() (err error)

Close the messenger process gracefully by closing the messages channel, wait for the messenger to finish sending the last messages, then clean up the connection to the remote peer. Note that any sends after Close() will cause a panic.

func (*Remote) Connect 

func (c *Remote) Connect() (err error)

Connect to the remote peer and establish birdirectional streams. This is the external version of connect that is used to establish the go routine that will send and receive messages from the remote host and dispatch them to the replic'as event chan. It is the messenger routine's responsibility to keep the channel open.

func (*Remote) Send 

func (c *Remote) Send(req *pb.PeerRequest)

Send a message to the remote. This places the message on a buffered channel, which will be sent in the order they are received. The response is dispatched to the actor event listener to be handled in the order responses are received.

func (*Remote) SendBeacon 

func (c *Remote) SendBeacon()

SendBeacon sends a beacon message that establishes a conenction to the remote.

type Remotes 

type Remotes map[uint32]*Remote

Remotes is a collection of remote peers that must be ordered by PID

type Replica 

type Replica struct {
	peers.Peer
	// contains filtered or unexported fields
}

Replica represents the local consensus replica and is the primary object implemented in a running system. There should only be one replica per process.

func New 

func New(options *Config) (replica *Replica, err error)

New ePaxos replica with the specified config.

func (*Replica) Broadcast 

func (r *Replica) Broadcast(req *pb.PeerRequest, toall bool)

Broadcast a request to all members in the quorum using thrifty communications if so configured. The toall flag forces the request to be broadcast even if thrifty.

func (*Replica) Close 

func (r *Replica) Close() error

Close the event handler and stop listening for events. TODO: gracefully shutdown the grpc server as well.

func (*Replica) Commit 

func (r *Replica) Commit(inst *pb.Instance)

Commit an instance and broadcast the commit to all members in the quroum and reply to the client(s) that initiated the proposal.

func (*Replica) Connect 

func (r *Replica) Connect() error

Connect the replica to its remote peers. If in thrifty mode, only connects to its thrifty neighbors rather than establishing connections to all peers.

func (*Replica) Consensus 

func (r *Replica) Consensus(stream pb.Epaxos_ConsensusServer) (err error)

Consensus receives PeerRequest messages from remote peers and dispatches them to the primary replica process. This method waits for the handler to create a reply before receiving the next message.

func (*Replica) Dispatch 

func (r *Replica) Dispatch(e Event) error

Dispatch events by clients to the replica.

func (*Replica) Handle 

func (r *Replica) Handle(e Event) error

Handle the events in serial order.

func (*Replica) Listen 

func (r *Replica) Listen() error

Listen for messages from peers and clients and run the event loop.

func (*Replica) Propose 

func (r *Replica) Propose(ctx context.Context, in *pb.ProposeRequest) (*pb.ProposeReply, error)

Propose is the primary entry point for client requests. This method is the gRPC handler that essentially dispatches the propose event to the replica and listens for the replica to send back a response so the client can be replied to.

Source Files

View all Source files

Directories

Path Synopsis
cmd
epaxos

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.