replication

Documentation

Overview

Package replication implements the core part of dqlite, setting up raft-based replication of the SQLite WAL.

Index

• type FSM
  • func NewFSM(registry *registry.Registry) *FSM
  • func (f *FSM) Apply(log *raft.Log) interface{}
  • func (f *FSM) Restore(reader io.ReadCloser) error
  • func (f *FSM) Snapshot() (raft.FSMSnapshot, error)
• type FSMSnapshot
  • func (s *FSMSnapshot) Persist(sink raft.SnapshotSink) error
  • func (s *FSMSnapshot) Release()
• type Methods
  • func NewMethods(reg *registry.Registry, raft *raft.Raft) *Methods
  • func (m *Methods) Abort(conn *sqlite3.SQLiteConn) sqlite3.ErrNo
  • func (m *Methods) ApplyTimeout(timeout time.Duration)
  • func (m *Methods) Begin(conn *sqlite3.SQLiteConn) sqlite3.ErrNo
  • func (m *Methods) End(conn *sqlite3.SQLiteConn) sqlite3.ErrNo
  • func (m *Methods) Frames(conn *sqlite3.SQLiteConn, frames *sqlite3.ReplicationFramesParams) sqlite3.ErrNo
  • func (m *Methods) Undo(conn *sqlite3.SQLiteConn) sqlite3.ErrNo

Types

type FSM

type FSM struct {
	// contains filtered or unexported fields
}

FSM implements the raft finite-state machine used to replicate SQLite data.

func NewFSM

func NewFSM(registry *registry.Registry) *FSM

NewFSM creates a new Raft state machine for executing dqlite-specific command.

func (*FSM) Apply

func (f *FSM) Apply(log *raft.Log) interface{}

Apply log is invoked once a log entry is committed. It returns a value which will be made available in the ApplyFuture returned by Raft.Apply method if that method was called on the same Raft node as the FSM.

func (*FSM) Restore

func (f *FSM) Restore(reader io.ReadCloser) error

Restore is used to restore an FSM from a snapshot. It is not called concurrently with any other command. The FSM must discard all previous state.

func (*FSM) Snapshot

func (f *FSM) Snapshot() (raft.FSMSnapshot, error)

Snapshot is used to support log compaction.

From the raft's package documentation:

"This call should return an FSMSnapshot which can be used to save a
point-in-time snapshot of the FSM. Apply and Snapshot are not called in
multiple threads, but Apply will be called concurrently with Persist. This
means the FSM should be implemented in a fashion that allows for
concurrent updates while a snapshot is happening."

In dqlite's case we do the following:

• For each database that we track (i.e. that we have a follower connection for), create a backup using sqlite3_backup() and then read the content of the backup file and the current WAL file. Since nothing else is writing to the database (FSM.Apply won't be called until FSM.Snapshot completes), we could probably read the database bytes directly to increase efficiency, but for now we do concurrent-write-safe backup as good measure.

• For each database we track, look for ongoing transactions and include their ID in the FSM snapshot, so their state can be re-created upon snapshot Restore.

This is a bit heavy-weight but should be safe. Optimizations can be added as needed.

type FSMSnapshot

type FSMSnapshot struct {
	// contains filtered or unexported fields
}

FSMSnapshot is returned by an FSM in response to a Snapshot It must be safe to invoke FSMSnapshot methods with concurrent calls to Apply.

func (*FSMSnapshot) Persist

func (s *FSMSnapshot) Persist(sink raft.SnapshotSink) error

Persist should dump all necessary state to the WriteCloser 'sink', and call sink.Close() when finished or call sink.Cancel() on error.

func (*FSMSnapshot) Release

func (s *FSMSnapshot) Release()

Release is invoked when we are finished with the snapshot.

type Methods

type Methods struct {
	// contains filtered or unexported fields
}

Methods implements the SQLite replication C API using the sqlite3 bindings.

func NewMethods

func NewMethods(reg *registry.Registry, raft *raft.Raft) *Methods

NewMethods returns a new Methods instance that can be used as callbacks API for raft-based SQLite replication of a single connection.

func (*Methods) Abort

func (m *Methods) Abort(conn *sqlite3.SQLiteConn) sqlite3.ErrNo

Abort is the hook invoked by SQLite when a write transaction fails to begin.

func (*Methods) ApplyTimeout

func (m *Methods) ApplyTimeout(timeout time.Duration)

ApplyTimeout sets the maximum amount of time to wait before giving up applying a raft command. The default is 10 seconds.

func (*Methods) Begin

func (m *Methods) Begin(conn *sqlite3.SQLiteConn) sqlite3.ErrNo

Begin is the hook invoked by SQLite when a new write transaction is being started within a connection in leader replication mode on this node.

func (*Methods) End

func (m *Methods) End(conn *sqlite3.SQLiteConn) sqlite3.ErrNo

End is the hook invoked by sqlite when ending a write transaction.

func (*Methods) Frames

func (m *Methods) Frames(conn *sqlite3.SQLiteConn, frames *sqlite3.ReplicationFramesParams) sqlite3.ErrNo

Frames is the hook invoked by sqlite when new frames need to be flushed to the write-ahead log.

func (*Methods) Undo

func (m *Methods) Undo(conn *sqlite3.SQLiteConn) sqlite3.ErrNo

Undo is the hook invoked by sqlite when a write transaction needs to be rolled back.