store

Documentation

Overview

Package store contains logic for managing the file system and maintaining consensus via the raft implementation.

Index

• Constants
• Variables
• type ApplyWriter
  • func (aw *ApplyWriter) Write(buf []byte) (int, error)
• type Command
• type Config
• type FSM
  • func (fsm *FSM) Apply(l *raft.Log) interface{}
  • func (fsm *FSM) Restore(rc io.ReadCloser) error
  • func (fsm *FSM) Snapshot() (raft.FSMSnapshot, error)
• type FSMSnapshot
  • func (f *FSMSnapshot) Persist(sink raft.SnapshotSink) error
  • func (f *FSMSnapshot) Release()
• type Node
• type Operation
• type Raft
• type Store
  • func Open(c *Config, fs *filesystem.FileSystem, md *metadata.Store) (*Store, error)
  • func OpenWithRaft(c *Config, r Raft, fs *filesystem.FileSystem, md *metadata.Store) *Store
  • func (s *Store) Close() error
  • func (s *Store) Copy(src, dest string) error
  • func (s *Store) Delete(name string) error
  • func (s *Store) DescribeNode() *Node
  • func (s *Store) FSM() raft.FSM
  • func (s *Store) Join(nodeID, addr string) error
  • func (s *Store) Move(src, dest string) error
  • func (s *Store) Set(md metadata.Metadata) (metadata.Metadata, error)
  • func (s *Store) Stats() map[string]string
  • func (s *Store) Writer(name string) io.Writer
• type WriteMode

Constants

const (
	Write              = Operation("write")
	Move               = Operation("move")
	Copy               = Operation("copy")
	Delete             = Operation("delete")
	Set                = Operation("set")
	WriteModeAppend    = WriteMode("append")
	WriteModeOverwrite = WriteMode("overwrite")
)

Constants for operation types/write modes

Variables

var (
	// ErrUnknownOperation is the error returned when attempting to apply
	// an unknown operation.
	ErrUnknownOperation = errors.New("unknown operation")

	// ErrUnknownWriteMode is the error returned when making a write against
	// the store using an invalid WriteMode.
	ErrUnknownWriteMode = errors.New("unknown write mode")
)

var (
	// ErrNotLeader is the error returned when any requests for state modification
	// are made on a raft node that is not the leader.
	ErrNotLeader = errors.New("not leader")
)

Types

type ApplyWriter

type ApplyWriter struct {
	// contains filtered or unexported fields
}

The ApplyWriter type is used as an io.Writer implementation that applies write commands against the store.

func (*ApplyWriter) Write

func (aw *ApplyWriter) Write(buf []byte) (int, error)

Write the data in the buffer as a command to the store.

type Command

type Command struct {
	Op          Operation         `json:"op"`
	Name        string            `json:"name,omitempty"`
	Data        []byte            `json:"data,omitempty"`
	Source      string            `json:"source,omitempty"`
	Destination string            `json:"destination,omitempty"`
	Mode        WriteMode         `json:"mode,omitempty"`
	Metadata    metadata.Metadata `json:"metadata,omitempty"`
}

The Command type is used to apply a change to the file system

type Config

type Config struct {
	LocalID       string        // The raft server ID
	BindAddr      string        // The raft bind address
	Bootstrap     bool          // Whether or not to bootstrap the cluster (for single node setups)
	Directory     string        // The directory to store raft data in
	SnapshotCount int           // The maximum number of snapshots to keep
	MaxPool       int           // The maximum number of pooled connections
	Timeout       time.Duration // Timeout duration for raft commands.
	Root          string        // The filesystem root
	LogWriter     io.Writer     // Writer implementation for logging
}

The Config type contains configuration values for the store.

type FSM

type FSM struct {
	// contains filtered or unexported fields
}

The FSM type is responsible for interacting with the log

func (*FSM) Apply

func (fsm *FSM) Apply(l *raft.Log) interface{}

Apply log is invoked once a log entry is committed. It returns a value which will be made available in the ApplyFuture returned by Raft.Apply method if that method was called on the same Raft node as the FSM.

func (*FSM) Restore

func (fsm *FSM) Restore(rc io.ReadCloser) error

Restore is used to restore an FSM from a snapshot. It is not called concurrently with any other command. The FSM must discard all previous state.

func (*FSM) Snapshot

func (fsm *FSM) Snapshot() (raft.FSMSnapshot, error)

Snapshot is used to support log compaction. This call should return an FSMSnapshot which can be used to save a point-in-time snapshot of the FSM. Apply and Snapshot are not called in multiple threads, but Apply will be called concurrently with Persist. This means the FSM should be implemented in a fashion that allows for concurrent updates while a snapshot is happening.

type FSMSnapshot

type FSMSnapshot struct {
	// contains filtered or unexported fields
}

The FSMSnapshot type is used to capture the current state of the file system.

func (*FSMSnapshot) Persist

func (f *FSMSnapshot) Persist(sink raft.SnapshotSink) error

Persist dumps all necessary state to the WriteCloser 'sink',

func (*FSMSnapshot) Release

func (f *FSMSnapshot) Release()

Release is invoked when the snapshot is no longer required.

type Node

type Node struct {
	LocalID  string
	IsLeader bool
}

The Node type contains attributes of a raft node.

type Operation

type Operation string

The Operation type denotes a certain type of operation to make against the file system.

type Raft

type Raft interface {
	Shutdown() raft.Future
	GetConfiguration() raft.ConfigurationFuture
	State() raft.RaftState
	Apply([]byte, time.Duration) raft.ApplyFuture
	RemoveServer(raft.ServerID, uint64, time.Duration) raft.IndexFuture
	AddVoter(raft.ServerID, raft.ServerAddress, uint64, time.Duration) raft.IndexFuture
	Stats() map[string]string
}

The Raft interface describes raft implementations.

type Store

type Store struct {
	*filesystem.FileSystem
	*metadata.Store
	// contains filtered or unexported fields
}

The Store type is used to perform operations against the file system and maintain the current state of the file system across many raft nodes.

func Open

func Open(c *Config, fs *filesystem.FileSystem, md *metadata.Store) (*Store, error)

Open a connection to the store using the provided configuration.

func OpenWithRaft

func OpenWithRaft(c *Config, r Raft, fs *filesystem.FileSystem, md *metadata.Store) *Store

OpenWithRaft creates a new store that uses the given Raft implementation instead of opening one via hashicorp/raft. Used for testing.

func (*Store) Close

func (s *Store) Close() error

Close the connection with the store.

func (*Store) Copy

func (s *Store) Copy(src, dest string) error

Copy a file from one location to another.

func (*Store) Delete

func (s *Store) Delete(name string) error

Delete a file

func (*Store) DescribeNode

func (s *Store) DescribeNode() *Node

DescribeNode returns information on the raft node.

func (*Store) FSM

func (s *Store) FSM() raft.FSM

FSM returns an FSM instance to append entries to the log.

func (*Store) Join

func (s *Store) Join(nodeID, addr string) error

Join adds a new node to the raft cluster with the provided id. Raft requests will be made to the provided address.

func (*Store) Move

func (s *Store) Move(src, dest string) error

Move a file from one location to another.

func (*Store) Set

func (s *Store) Set(md metadata.Metadata) (metadata.Metadata, error)

Set metadata for a file.

func (*Store) Stats

func (s *Store) Stats() map[string]string

Stats returns the raft statistics for this node.

func (*Store) Writer

func (s *Store) Writer(name string) io.Writer

Writer returns an io.Writer implementation that will apply the contents of the buffer as a command.

type WriteMode

type WriteMode string

The WriteMode type denotes expected behaviour of a write operation.