dqlite

README

dqlite

This repository provides the dqlite Go package, which can be used to replicate a SQLite database across a cluster, using the Raft algorithm.

Design higlights

• No external processes needed: dqlite is just a Go library, you link it it to your application exactly like you would with SQLite.
• Replication needs a SQLite patch which is not yet included upstream.
• The Go Raft package from Hashicorp is used internally for replicating the write-ahead log frames of SQLite across all nodes.

How does it compare to rqlite?

The main differences from rqlite are:

• Full support for transactions
• No need for statements to be deterministic (e.g. you can use time())
• Frame-based replication instead of statement-based replication, this means in dqlite there's more data flowing between nodes, so expect lower performance. Should not really matter for most use cases.

Status

This is beta software for now, but we'll get to rc/release soon.

Demo

To see dqlite in action, make sure you have the following dependencies installed:

• Go (tested on 1.8)
• gcc
• any dependency/header that SQLite needs to build from source
• Python 3

Then run:

go get -d github.com/CanonicalLtd/dqlite
cd $GOPATH/src/github.com/CanonicalLtd/dqlite
make dependencies
./run-demo

This should spawn three dqlite-based nodes, each of one running the code in the demo Go source.

Each node inserts data in a test table and then dies abruptly after a random timeout. Leftover transactions and failover to other nodes should be handled gracefully.

While the demo is running, to get more details about what's going on behind the scenes you can also open another terminal and run a command like:

watch ls -l /tmp/dqlite-demo-*/ /tmp/dqlite-demo-*/snapshots/

and see how the data directories of the three nodes evolve in terms SQLite databases (test.db), write-ahead log files (test.db-wal), raft logs store (raft.db), and raft snapshots.

Documentation

The documentation for this package can be found on Godoc.

FAQ

Q: How does dqlite behave during conflict situations? Does Raft select a winning WAL write and any others in flight are aborted?

A: There can't be a conflict situation. Raft's model is that only the leader can append new log entries, which translated to dqlite means that only the leader can write new WAL frames. So this means that any attempt to perform a write transaction on a non-leader node will fail with a sqlite3x.ErrNotLeader error (and in this case clients are supposed to retry against whoever is the new leader).

Q: When not enough nodes are available, are writes hung until consensus?

A: Yes, however there's a (configurable) timeout. This is a consequence of Raft sitting in the CP spectrum of the CAP theorem: in case of a network partition it chooses consistency and sacrifices availability.

Documentation

Overview

Package dqlite implements a database/sql/driver with raft-based SQLite replication.

Index

• func NewFSM(r *Registry) raft.FSM
• type Conn
  • func (c *Conn) Begin() (driver.Tx, error)
  • func (c *Conn) Close() error
  • func (c *Conn) Exec(query string, args []driver.Value) (driver.Result, error)
  • func (c *Conn) Prepare(query string) (driver.Stmt, error)
• type Driver
  • func NewDriver(r *Registry, raft *raft.Raft, config DriverConfig) (*Driver, error)
  • func (d *Driver) Leader() string
  • func (d *Driver) Open(uri string) (driver.Conn, error)
  • func (d *Driver) Recover(token uint64) error
  • func (d *Driver) Servers() ([]string, error)
• type DriverConfig
• type Registry
  • func NewRegistry(dir string) *Registry
• type Stmt
  • func (s *Stmt) Close() error
  • func (s *Stmt) Exec(args []driver.Value) (driver.Result, error)
  • func (s *Stmt) NumInput() int
  • func (s *Stmt) Query(args []driver.Value) (driver.Rows, error)
• type Tx
  • func (tx *Tx) Commit() error
  • func (tx *Tx) Rollback() error
  • func (tx *Tx) Token() uint64

Functions

func NewFSM

func NewFSM(r *Registry) raft.FSM

NewFSM creates a new dqlite FSM, suitable to be passed to raft.NewRaft.

It will handle replication of the SQLite write-ahead log.

This is mostly an internal implementation detail of dqlite, but it needs to be exposed since the raft.Raft parameter that NewDriver accepts doesn't allow access to the FSM that it was passed when created with raft.NewRaft().

Types

type Conn

type Conn struct {
	// contains filtered or unexported fields
}

Conn implements the sql.Conn interface.

func (*Conn) Begin

func (c *Conn) Begin() (driver.Tx, error)

Begin starts and returns a new transaction.

func (*Conn) Close

func (c *Conn) Close() error

Close invalidates and potentially stops any current prepared statements and transactions, marking this connection as no longer in use.

Because the sql package maintains a free pool of connections and only calls Close when there's a surplus of idle connections, it shouldn't be necessary for drivers to do their own connection caching.

func (*Conn) Exec

func (c *Conn) Exec(query string, args []driver.Value) (driver.Result, error)

Exec may return ErrSkip.

Deprecated: Drivers should implement ExecerContext instead (or additionally).

func (*Conn) Prepare

func (c *Conn) Prepare(query string) (driver.Stmt, error)

Prepare returns a prepared statement, bound to this connection.

type Driver

type Driver struct {
	// contains filtered or unexported fields
}

Driver manages a node partecipating to a dqlite replicated cluster.

func NewDriver

func NewDriver(r *Registry, raft *raft.Raft, config DriverConfig) (*Driver, error)

NewDriver creates a new node of a dqlite cluster, which also implements the driver.Driver interface.

The Registry instance must be the same one that was passed to NewFSM to build the raft.FSM which in turn got passed to raft.NewRaft for creating the raft instance.

func (*Driver) Leader

func (d *Driver) Leader() string

Leader returns the address of the current raft leader, if any.

func (*Driver) Open

func (d *Driver) Open(uri string) (driver.Conn, error)

Open starts a new connection to a SQLite database.

The given name must be a pure file name without any directory segment, dqlite will connect to a database with that name in its data directory.

Query parameters are always valid except for "mode=memory".

If this node is not the leader, or the leader is unknown an ErrNotLeader error is returned.

func (*Driver) Recover

func (d *Driver) Recover(token uint64) error

Recover tries to recover a transaction that errored because leadership was lost at commit time.

If this driver is the newly elected leader and a quorum was reached for the lost commit command, recovering will succeed and the transaction can safely be considered committed.

func (*Driver) Servers

func (d *Driver) Servers() ([]string, error)

Servers returns the addresses of all current raft servers. It returns an error if this server is not the leader.

type DriverConfig

type DriverConfig struct {
	Logger              *log.Logger   // Logger to use to emit messages.
	BarrierTimeout      time.Duration // Maximum amount of time to wait for the FSM to catch up with logs.
	ApplyTimeout        time.Duration // Maximum amount of time to wait for a raft FSM command to be applied.
	CheckpointThreshold uint64        // Minimum number of WAL frames before performing a checkpoint.
}

DriverConfig holds configuration options for a dqlite SQL Driver.

type Registry

type Registry registry.Registry

Registry tracks internal data shared by the dqlite Driver and FSM.

func NewRegistry

func NewRegistry(dir string) *Registry

NewRegistry creates a new Registry, which is expected to be passed to both NewFSM and NewDriver.

The dir parameter is the directory where dqlite will store the underlying SQLite database files.

type Stmt

type Stmt struct {
	// contains filtered or unexported fields
}

Stmt is a prepared statement. It is bound to a Conn and not used by multiple goroutines concurrently.

func (*Stmt) Close

func (s *Stmt) Close() error

Close closes the statement.

func (*Stmt) Exec

func (s *Stmt) Exec(args []driver.Value) (driver.Result, error)

Exec executes a query that doesn't return rows, such

func (*Stmt) NumInput

func (s *Stmt) NumInput() int

NumInput returns the number of placeholder parameters.

func (*Stmt) Query

func (s *Stmt) Query(args []driver.Value) (driver.Rows, error)

Query executes a query that may return rows, such as a

type Tx

type Tx struct {
	// contains filtered or unexported fields
}

Tx is a transaction.

func (*Tx) Commit

func (tx *Tx) Commit() error

Commit the transaction.

func (*Tx) Rollback

func (tx *Tx) Rollback() error

Rollback the transaction.

func (*Tx) Token

func (tx *Tx) Token() uint64

Token returns the internal ID for this transaction, that can be passed to driver.Recover() in case the commit fails because of lost leadership.