database

README

General concept

Package database represents a database layer in Dusk Network. Current design should embody both backend-agnostic and storage-agnostic concepts.

Terminology

Storage: Underlying storage can be any kind of KV stores (BoltDB, LMDB, LevelDB, RocksDB, Redis), non-KV stores (SQLite) or even a custom ACID-complient flat-file store.

Backend/Driver: represents a DB implementation that uses one or multiple underlying storages to provide block chain database specifics with regard to particular purpose. It must implement Driver, DB and Tx intefaces.

Interfaces exposed to upper layers:

• database.Driver: DB connection initiator. Inspired by database/sql/driver
• database.DB: A thin layer on top of database.Tx providing a manageable Tx execution. Inspired by BoltDB
• database.Tx: A transaction layer tightly coupled with DUSK block chain specifics to fully benefit from underlying storage capabilities

Available Drivers

• /database/heavy driver is designed to provide efficient, robust and persistent DUSK block chain DB on top of syndtr/goleveldb/leveldb store (unofficial LevelDB porting). It must be Mainnet-complient.

Testing Drivers

• /database/testing implements a boilerplate method to verify if a registered driver does satisfy minimum database requirements. The package defines a set of unit tests that are executed only on registered drivers. It can serve also as a detailed and working database guideline.

Code example:

More code examples can be found in /database/heavy/database_test.go

// By changing Driver name, one can switch between different backends
// Each backend is bound to one or multiple underlying stores
readonly := false

// Retrieve
driver, _ := database.From(lite.DriverName)
db, err := driver.Open(path, protocol.DevNet, readonly)

if err != nil {
	...
}

defer db.Close()

blocks := make([]*block.Block, 100)
// Populate blocks here ...

// a managed read-write DB Tx to store all blocks via atomic update
err = db.Update(func(tx database.Tx) error {
	for _, block := range blocks {
		err := tx.StoreBlock(block)
		if err != nil {
			return err
		}
	}
	return nil
})

if err != nil {
	fmt.Printf("Transaction failed. No blocks stored")
	return
}

 // a managed read-only DB Tx to check if all blocks have been stored
_ = db.View(func(tx database.Tx) error {
	for _, block := range blocks {
		exists, err := tx.FetchBlockExists(block.Header.Hash)
		if err != nil {
			fmt.Printf(err.Error())
			return nil
		}

		if !exists {
			fmt.Printf("Block with Height %d was not found", block.Header.Height)
			return nil
		}
	}
	return nil
})

Additional features

Additional features that can be provided by a Driver:

• Read-only transactions/connections
• Validated storage connection
• Profiling: Collect transactions and connections statistics
• Traces: Log transactions data
• Alarming: Trigger an event in case of critical backend or storage failure
• Iterators/Cursors
• Safe Concurrency model: One read-write transaction, many read-only transactions
• Redundancy: blockchain data stored in a secondary in-memory or on-disk structure

Documentation

Index

• Variables
• func Drivers() []string
• func Register(driver Driver) error
• type DB
• type Driver
  • func From(name string) (Driver, error)
• type State
• type Transaction

Variables

var (

	// ErrKeyImageNotFound returned on a keyImage lookup
	ErrKeyImageNotFound = errors.New("database: keyImage not found")
	// ErrTxNotFound returned on a tx lookup by hash
	ErrTxNotFound = errors.New("database: transaction not found")
	// ErrBlockNotFound returned on a block lookup by hash or height
	ErrBlockNotFound = errors.New("database: block not found")
	// ErrStateNotFound returned on missing state db entry
	ErrStateNotFound = errors.New("database: state not found")

	// AnyTxType is used as a filter value on FetchBlockTxByHash
	AnyTxType = transactions.TxType(math.MaxUint8)
)

Functions

func Drivers

func Drivers() []string

Drivers returns a sorted list of the names of the registered drivers.

func Register

func Register(driver Driver) error

Register makes a database driver available by the provided name. If Register is called twice with the same name or if driver is nil, it returns error.

Types

type DB

type DB interface {

	// View provides a managed execution of a read-only transaction
	View(fn func(t Transaction) error) error

	// Update provides a managed execution of a read-write atomic transaction.
	//
	// An atomic transaction is an indivisible and irreducible series of
	// database operations such that either all occur, or nothing occurs.
	//
	// Transaction commit will happen only if no error is returned by `fn`
	// and no panic is raised on `fn` execution.
	Update(fn func(t Transaction) error) error

	Close() error
}

DB is a thin layer on top of Transaction providing a manageable execution

type Driver

type Driver interface {
	// Open returns a new connection to a blockchain database. The path is a
	// string in a driver-specific format.
	Open(path string, network protocol.Magic, readonly bool) (DB, error)

	// Close terminates all DB connections and closes underlying storage
	Close() error

	// Name returns a unique identifier that can be used to register the driver
	Name() string
}

A Driver represents an application programming interface for accessing blockchain database management systems.

It is conceptually similar to ODBC for DBMS

func From

func From(name string) (Driver, error)

From returns a registered Driver by name

type State

type State struct {
	TipHash []byte
}

State represents a single db entry that provides chain metadata. This includes currently only chain tip hash but could be extended at later stage

type Transaction

type Transaction interface {
	FetchBlockHeader(hash []byte) (*block.Header, error)
	// Fetch all of the Txs that belong to a block with this header.hash
	FetchBlockTxs(hash []byte) ([]transactions.Transaction, error)
	// Fetch tx by txID. If succeeds, it returns tx data, tx index and
	// hash of the block it belongs to.
	FetchBlockTxByHash(txID []byte) (tx transactions.Transaction, txIndex uint32, blockHeaderHash []byte, err error)
	FetchBlockHashByHeight(height uint64) ([]byte, error)
	FetchBlockExists(hash []byte) (bool, error)
	// Fetch chain state information (chain tip hash)
	FetchState() (*State, error)

	// Check if an input keyImage is already stored. If succeeds, it returns
	// also txID the input belongs to
	FetchKeyImageExists(keyImage []byte) (exists bool, txID []byte, err error)

	// Fetch a candidate block by hash
	FetchCandidateBlock(hash []byte) (*block.Block, error)

	// Read-write transactions
	// Store the next chain block in a append-only manner
	// Overwrites only if block with same hash already stored
	StoreBlock(block *block.Block) error

	// StoreCandidateBlock stores a candidate block to be proposed in next
	// consensus round.
	StoreCandidateBlock(block *block.Block) error

	// DeleteCandidateBlocks deletes all candidate blocks. If maxHeight is not
	// 0, it deletes only blocks with a height lower than maxHeight or equal. It
	// returns number of deleted candidate blocks
	DeleteCandidateBlocks(maxHeight uint64) (uint32, error)

	FetchBlock(hash []byte) (*block.Block, error)

	FetchCurrentHeight() (uint64, error)

	FetchDecoys(numDecoys int) []ristretto.Point

	FetchOutputExists(destkey []byte) (bool, error)

	// Atomic storage
	Commit() error
	Rollback() error
	Close()
}

Transaction represents transaction layer. Transaction should provide basic transactions to fetch and store blockchain data.

To simplify code reading with database pkg we should use 'Tx' to refer to blockchain transaction and 'Transaction' to refer to database transaction