walletdb

README

walletdb

[] (https://travis-ci.org/btcsuite/btcwallet)

Package walletdb provides a namespaced database interface for btcwallet.

A wallet essentially consists of a multitude of stored data such as private and public keys, key derivation bits, pay-to-script-hash scripts, and various metadata. One of the issues with many wallets is they are tightly integrated. Designing a wallet with loosely coupled components that provide specific functionality is ideal, however it presents a challenge in regards to data storage since each component needs to store its own data without knowing the internals of other components or breaking atomicity.

This package solves this issue by providing a namespaced database interface that is intended to be used by the main wallet daemon. This allows the potential for any backend database type with a suitable driver. Each component, which will typically be a package, can then implement various functionality such as address management, voting pools, and colored coin metadata in their own namespace without having to worry about conflicts with other packages even though they are sharing the same database that is managed by the wallet.

A suite of tests is provided to ensure proper functionality. See test_coverage.txt for the gocov coverage report. Alternatively, if you are running a POSIX OS, you can run the cov_report.sh script for a real-time report. Package walletdb is licensed under the copyfree ISC license.

This interfaces provided by this package were heavily inspired by the excellent boltdb project at https://github.com/boltdb/bolt by Ben B. Johnson.

Feature Overview

• Key/value store
• Namespace support
  • Allows multiple packages to have their own area in the database without worrying about conflicts
• Read-only and read-write transactions with both manual and managed modes
• Nested buckets
• Supports registration of backend databases
• Comprehensive test coverage

Documentation

[] (http://godoc.org/git.parallelcoin.io/dev/9/walletmain/walletdb)

Full go doc style documentation for the project can be viewed online without installing this package by using the GoDoc site here: http://godoc.org/git.parallelcoin.io/dev/9/walletmain/walletdb

You can also view the documentation locally once the package is installed with the godoc tool by running godoc -http=":6060" and pointing your browser to http://localhost:6060/pkg/git.parallelcoin.io/dev/9/walletmain/walletdb

Installation

$ go get git.parallelcoin.io/dev/9/walletmain/walletdb

Examples

• [Basic Usage Example] (http://godoc.org/git.parallelcoin.io/dev/9/walletmain/walletdb#example-package--BasicUsage)
  Demonstrates creating a new database, getting a namespace from it, and using a managed read-write transaction against the namespace to store and retrieve data.

License

Package walletdb is licensed under the copyfree ISC License.

Documentation

Overview

Package walletdb provides a namespaced database interface for btcwallet.

Overview

A wallet essentially consists of a multitude of stored data such as private and public keys, key derivation bits, pay-to-script-hash scripts, and various metadata. One of the issues with many wallets is they are tightly integrated. Designing a wallet with loosely coupled components that provide specific functionality is ideal, however it presents a challenge in regards to data storage since each component needs to store its own data without knowing the internals of other components or breaking atomicity.

This package solves this issue by providing a pluggable driver, namespaced database interface that is intended to be used by the main wallet daemon. This allows the potential for any backend database type with a suitable driver. Each component, which will typically be a package, can then implement various functionality such as address management, voting pools, and colored coin metadata in their own namespace without having to worry about conflicts with other packages even though they are sharing the same database that is managed by the wallet.

A quick overview of the features walletdb provides are as follows:

• Key/value store
• Namespace support
• Allows multiple packages to have their own area in the database without worrying about conflicts
• Read-only and read-write transactions with both manual and managed modes
• Nested buckets
• Supports registration of backend databases
• Comprehensive test coverage

Database

The main entry point is the DB interface. It exposes functionality for creating, retrieving, and removing namespaces. It is obtained via the Create and Open functions which take a database type string that identifies the specific database driver (backend) to use as well as arguments specific to the specified driver.

Namespaces

The Namespace interface is an abstraction that provides facilities for obtaining transactions (the Tx interface) that are the basis of all database reads and writes. Unlike some database interfaces that support reading and writing without transactions, this interface requires transactions even when only reading or writing a single key.

The Begin function provides an unmanaged transaction while the View and Update functions provide a managed transaction. These are described in more detail below.

Transactions

The Tx interface provides facilities for rolling back or commiting changes that took place while the transaction was active. It also provides the root bucket under which all keys, values, and nested buckets are stored. A transaction can either be read-only or read-write and managed or unmanaged.

Managed versus Unmanaged Transactions

A managed transaction is one where the caller provides a function to execute within the context of the transaction and the commit or rollback is handled automatically depending on whether or not the provided function returns an error. Attempting to manually call Rollback or Commit on the managed transaction will result in a panic.

An unmanaged transaction, on the other hand, requires the caller to manually call Commit or Rollback when they are finished with it. Leaving transactions open for long periods of time can have several adverse effects, so it is recommended that managed transactions are used instead.

Buckets

The Bucket interface provides the ability to manipulate key/value pairs and nested buckets as well as iterate through them.

The Get, Put, and Delete functions work with key/value pairs, while the Bucket, CreateBucket, CreateBucketIfNotExists, and DeleteBucket functions work with buckets. The ForEach function allows the caller to provide a function to be called with each key/value pair and nested bucket in the current bucket.

Root Bucket

As discussed above, all of the functions which are used to manipulate key/value pairs and nested buckets exist on the Bucket interface. The root bucket is the upper-most bucket in a namespace under which data is stored and is created at the same time as the namespace. Use the RootBucket function on the Tx interface to retrieve it.

Nested Buckets

The CreateBucket and CreateBucketIfNotExists functions on the Bucket interface provide the ability to create an arbitrary number of nested buckets. It is a good idea to avoid a lot of buckets with little data in them as it could lead to poor page utilization depending on the specific driver in use.

Example (BasicUsage)

This example demonstrates creating a new database, getting a namespace from it, and using a managed read-write transaction against the namespace to store and retrieve data.

package main

import (
	"bytes"
	"fmt"
	"os"
	"path/filepath"

	walletdb "git.parallelcoin.io/dev/9/pkg/wallet/db"
	_ "git.parallelcoin.io/dev/9/pkg/wallet/db/bdb"
)

func main() {

	// This example assumes the bdb (bolt db) driver is imported.

	//

	// import (

	// 	"git.parallelcoin.io/dev/9/pkg/wallet/db"

	// 	_ "git.parallelcoin.io/dev/9/pkg/wallet/db/bdb"

	// )

	// Create a database and schedule it to be closed and removed on exit.

	// Typically you wouldn't want to remove the database right away like

	// this, but it's done here in the example to ensure the example cleans

	// up after itself.
	dbPath := filepath.Join(os.TempDir(), "exampleusage.db")
	db, err := walletdb.Create("bdb", dbPath)

	if err != nil {

		fmt.Println(err)
		return
	}
	defer os.Remove(dbPath)
	defer db.Close()

	// Get or create a bucket in the database as needed.  This bucket

	// is what is typically passed to specific sub-packages so they have

	// their own area to work in without worrying about conflicting keys.
	bucketKey := []byte("walletsubpackage")
	err = walletdb.Update(db, func(tx walletdb.ReadWriteTx) error {

		bucket := tx.ReadWriteBucket(bucketKey)

		if bucket == nil {

			_, err = tx.CreateTopLevelBucket(bucketKey)

			if err != nil {

				return err
			}
		}
		return nil
	})

	if err != nil {

		fmt.Println(err)
		return
	}

	// Use the Update function of the namespace to perform a managed

	// read-write transaction.  The transaction will automatically be rolled

	// back if the supplied inner function returns a non-nil error.
	err = walletdb.Update(db, func(tx walletdb.ReadWriteTx) error {

		// All data is stored against the root bucket of the namespace,
		// or nested buckets of the root bucket.  It's not really
		// necessary to store it in a separate variable like this, but
		// it has been done here for the purposes of the example to
		// illustrate.
		rootBucket := tx.ReadWriteBucket(bucketKey)

		// Store a key/value pair directly in the root bucket.
		key := []byte("mykey")
		value := []byte("myvalue")

		if err := rootBucket.Put(key, value); err != nil {

			return err
		}

		// Read the key back and ensure it matches.

		if !bytes.Equal(rootBucket.Get(key), value) {

			return fmt.Errorf("unexpected value for key '%s'", key)
		}

		// Create a new nested bucket under the root bucket.
		nestedBucketKey := []byte("mybucket")
		nestedBucket, err := rootBucket.CreateBucket(nestedBucketKey)

		if err != nil {

			return err
		}

		// The key from above that was set in the root bucket does not
		// exist in this new nested bucket.

		if nestedBucket.Get(key) != nil {

			return fmt.Errorf("key '%s' is not expected nil", key)
		}

		return nil
	})

	if err != nil {

		fmt.Println(err)
		return
	}

}

Index

• Variables
• func BucketIsEmpty(bucket ReadBucket) bool
• func RegisterDriver(driver Driver) error
• func SupportedDrivers() []string
• func Update(db DB, f func(tx ReadWriteTx) error) error
• func View(db DB, f func(tx ReadTx) error) error
• type DB
  • func Create(dbType string, args ...interface{}) (DB, error)
  • func Open(dbType string, args ...interface{}) (DB, error)
• type Driver
• type ReadBucket
• type ReadCursor
• type ReadTx
• type ReadWriteBucket
• type ReadWriteCursor
• type ReadWriteTx

Examples

• Package (BasicUsage)
• Create
• DB (CreateTopLevelBucket)

Variables

var (

	// the specified database type.
	ErrDbUnknownType = errors.New("unknown database type")

	// does not exist.
	ErrDbDoesNotExist = errors.New("database does not exist")

	// already exists.
	ErrDbExists = errors.New("database already exists")

	// it is opened or after it is closed.
	ErrDbNotOpen = errors.New("database not open")

	// is already open.
	ErrDbAlreadyOpen = errors.New("database already open")

	// ErrInvalid is returned if the specified database is not valid.
	ErrInvalid = errors.New("invalid database")
)

Errors that the various database functions may return.

var (

	// transaction that has already had one of those operations performed.
	ErrTxClosed = errors.New("tx closed")

	// access to the database is attempted against a read-only transaction.
	ErrTxNotWritable = errors.New("tx not writable")
)

Errors that can occur when beginning or committing a transaction.

var (

	// not been created yet.
	ErrBucketNotFound = errors.New("bucket not found")

	// ErrBucketExists is returned when creating a bucket that already exists.
	ErrBucketExists = errors.New("bucket already exists")

	// ErrBucketNameRequired is returned when creating a bucket with a blank name.
	ErrBucketNameRequired = errors.New("bucket name required")

	// ErrKeyRequired is returned when inserting a zero-length key.
	ErrKeyRequired = errors.New("key required")

	// ErrKeyTooLarge is returned when inserting a key that is larger than MaxKeySize.
	ErrKeyTooLarge = errors.New("key too large")

	// ErrValueTooLarge is returned when inserting a value that is larger than MaxValueSize.
	ErrValueTooLarge = errors.New("value too large")

	// delete a non-bucket key on an existing bucket key.
	ErrIncompatibleValue = errors.New("incompatible value")
)

Errors that can occur when putting or deleting a value or bucket.

var (

	// attempt to register with the name database type.
	ErrDbTypeRegistered = errors.New("database type already registered")
)

Errors that can occur during driver registration.

Functions

func BucketIsEmpty

func BucketIsEmpty(
	bucket ReadBucket) bool

BucketIsEmpty returns whether the bucket is empty, that is, whether there are no key/value pairs or nested buckets.

func RegisterDriver

func RegisterDriver(
	driver Driver) error

RegisterDriver adds a backend database driver to available interfaces. ErrDbTypeRegistered will be retruned if the database type for the driver has already been registered.

func SupportedDrivers

func SupportedDrivers() []string

SupportedDrivers returns a slice of strings that represent the database drivers that have been registered and are therefore supported.

func Update

func Update(
	db DB, f func(tx ReadWriteTx) error) error

Update opens a database read/write transaction and executes the function f with the transaction passed as a parameter. After f exits, if f did not error, the transaction is committed. Otherwise, if f did error, the transaction is rolled back. If the rollback fails, the original error returned by f is still returned. If the commit fails, the commit error is returned.

func View

func View(
	db DB, f func(tx ReadTx) error) error

View opens a database read transaction and executes the function f with the transaction passed as a parameter. After f exits, the transaction is rolled back. If f errors, its error is returned, not a rollback error (if any occur).

Types

type DB

type DB interface {

	// BeginReadTx opens a database read transaction.
	BeginReadTx() (ReadTx, error)

	// BeginReadWriteTx opens a database read+write transaction.
	BeginReadWriteTx() (ReadWriteTx, error)

	// call will start a read-only transaction to perform all operations.
	Copy(w io.Writer) error

	// Close cleanly shuts down the database and syncs all data.
	Close() error
}

Example (CreateTopLevelBucket)

This example demonstrates creating a new top level bucket.

package main

import (
	"fmt"
	"os"
	"path/filepath"

	walletdb "git.parallelcoin.io/dev/9/pkg/wallet/db"
	_ "git.parallelcoin.io/dev/9/pkg/wallet/db/bdb"
)

// exampleNum is used as a counter in the exampleLoadDB function to provided
// a unique database name for each example.
var exampleNum = 0

// exampleLoadDB is used in the examples to elide the setup code.
func exampleLoadDB() (walletdb.DB, func(), error) {

	dbName := fmt.Sprintf("exampleload%d.db", exampleNum)
	dbPath := filepath.Join(os.TempDir(), dbName)
	db, err := walletdb.Create("bdb", dbPath)

	if err != nil {

		return nil, nil, err
	}
	teardownFunc := func() {

		db.Close()
		os.Remove(dbPath)
	}
	exampleNum++

	return db, teardownFunc, err
}

func main() {

	// Load a database for the purposes of this example and schedule it to

	// be closed and removed on exit. See the Create example for more

	// details on what this step is doing.
	db, teardownFunc, err := exampleLoadDB()

	if err != nil {

		fmt.Println(err)
		return
	}
	defer teardownFunc()

	dbtx, err := db.BeginReadWriteTx()

	if err != nil {

		fmt.Println(err)
		return
	}
	defer dbtx.Commit()

	// Get or create a bucket in the database as needed.  This bucket

	// is what is typically passed to specific sub-packages so they have

	// their own area to work in without worrying about conflicting keys.
	bucketKey := []byte("walletsubpackage")
	bucket, err := dbtx.CreateTopLevelBucket(bucketKey)

	if err != nil {

		fmt.Println(err)
		return
	}

	// Prevent unused error.
	_ = bucket

}

func Create

func Create(
	dbType string, args ...interface{}) (DB, error)

Create intializes and opens a database for the specified type. The arguments are specific to the database type driver. See the documentation for the database driver for further details.

ErrDbUnknownType will be returned if the the database type is not registered.

Example

This example demonstrates creating a new database.

package main

import (
	"fmt"
	"os"
	"path/filepath"

	walletdb "git.parallelcoin.io/dev/9/pkg/wallet/db"
	_ "git.parallelcoin.io/dev/9/pkg/wallet/db/bdb"
)

func main() {

	// This example assumes the bdb (bolt db) driver is imported.

	//

	// import (

	// 	"git.parallelcoin.io/dev/9/pkg/wallet/db"

	// 	_ "git.parallelcoin.io/dev/9/pkg/wallet/db/bdb"

	// )

	// Create a database and schedule it to be closed and removed on exit.

	// Typically you wouldn't want to remove the database right away like

	// this, but it's done here in the example to ensure the example cleans

	// up after itself.
	dbPath := filepath.Join(os.TempDir(), "examplecreate.db")
	db, err := walletdb.Create("bdb", dbPath)

	if err != nil {

		fmt.Println(err)
		return
	}
	defer os.Remove(dbPath)
	defer db.Close()

}

func Open

func Open(dbType string, args ...interface{}) (DB, error)

Open opens an existing database for the specified type. The arguments are specific to the database type driver. See the documentation for the database driver for further details.

ErrDbUnknownType will be returned if the the database type is not registered.

type Driver

type Driver struct {

	// database driver.  There can be only one driver with the same name.
	DbType string

	// ErrDbExists if the database already exists.
	Create func(args ...interface{}) (DB, error)

	// ErrDbDoesNotExist if the database has not already been created.
	Open func(args ...interface{}) (DB, error)
}

type ReadBucket

type ReadBucket interface {

	// Returns nil if the bucket does not exist.
	NestedReadBucket(key []byte) ReadBucket

	// implementations.
	ForEach(func(k, v []byte) error) error

	// implementations.
	Get(key []byte) []byte

	ReadCursor() ReadCursor
}

type ReadCursor

type ReadCursor interface {

	// the pair.
	First() (key, value []byte)

	// pair.
	Last() (key, value []byte)

	// pair.
	Next() (key, value []byte)

	// pair.
	Prev() (key, value []byte)

	// the new pair.
	Seek(seek []byte) (key, value []byte)
}

type ReadTx

type ReadTx interface {

	// described by the key does not exist, nil is returned.
	ReadBucket(key []byte) ReadBucket

	// database was modified by a write transaction.
	Rollback() error
}

type ReadWriteBucket

type ReadWriteBucket interface {
	ReadBucket

	// Returns nil if the bucket does not exist.
	NestedReadWriteBucket(key []byte) ReadWriteBucket

	// implementation.
	CreateBucket(key []byte) (ReadWriteBucket, error)

	// backend.  Other errors are possible depending on the implementation.
	CreateBucketIfNotExists(key []byte) (ReadWriteBucket, error)

	// and ErrBucketNotFound if the specified bucket does not exist.
	DeleteNestedBucket(key []byte) error

	// read-only transaction.
	Put(key, value []byte) error

	// ErrTxNotWritable if attempted against a read-only transaction.
	Delete(key []byte) error

	// key/value pairs and nested buckets in forward or backward order.
	ReadWriteCursor() ReadWriteCursor
}

type ReadWriteCursor

type ReadWriteCursor interface {
	ReadCursor

	// when the cursor points to a nested bucket.
	Delete() error
}

type ReadWriteTx

type ReadWriteTx interface {
	ReadTx

	// bucket described by the key does not exist, nil is returned.
	ReadWriteBucket(key []byte) ReadWriteBucket

	// does not exist.  The newly-created bucket it returned.
	CreateTopLevelBucket(key []byte) (ReadWriteBucket, error)

	// instead of a bucket.
	DeleteTopLevelBucket(key []byte) error

	// buckets and all of their sub-buckets to persistent storage.
	Commit() error
}