database

Documentation

Index

• Constants
• Variables
• type Batch
• type BlockKey
  • func NewBlockKey(hash *common.Hash, blockType BlockType) *BlockKey
  • func NewNormalBlockKey(hash *common.Hash) *BlockKey
  • func NewVirtualBlockKey(hash *common.Hash) *BlockKey
  • func (blockkey *BlockKey) IsVirtual() bool
  • func (blockkey *BlockKey) String() string
• type BlockRegion
• type BlockType
• type Bucket
• type Cursor
• type Database
• type Deleter
• type Error
  • func ConvertErr(desc string, ldbErr error) Error
  • func MakeError(c ErrorCode, desc string, err error) Error
  • func (e Error) Error() string
• type ErrorCode
  • func (e ErrorCode) String() string
• type Putter
• type Queryer
• type Transactor
• type Tx

Constants

const (
	FFLDB = "ffldb"
	ETHDB = "ethdb"
	// blockKeyLength is the size of block key
	BlockKeyLength = 33

	BlockNormal  BlockType = 0
	BlockVirtual BlockType = 1
)

const IdealBatchSize = 100 * 1024

Code using batches should try to add this much data to the batch. The value was determined empirically.

Variables

var ErrUnimplement = errors.New("method is unimplement")

Types

type Batch

type Batch interface {
	Putter
	Deleter
	ValueSize() int // amount of data in the batch
	Write() error
	// Reset resets the batch for reuse
	Reset()
}

Batch is a write-only database that commits changes to its host database when Write is called. Batch cannot be used concurrently.

type BlockKey

type BlockKey [BlockKeyLength]byte

func NewBlockKey

func NewBlockKey(hash *common.Hash, blockType BlockType) *BlockKey

func NewNormalBlockKey

func NewNormalBlockKey(hash *common.Hash) *BlockKey

func NewVirtualBlockKey

func NewVirtualBlockKey(hash *common.Hash) *BlockKey

func (*BlockKey) IsVirtual

func (blockkey *BlockKey) IsVirtual() bool

func (*BlockKey) String

func (blockkey *BlockKey) String() string

type BlockRegion

type BlockRegion struct {
	Key    *BlockKey
	Offset uint32
	Len    uint32
}

BlockRegion specifies a particular region of a block identified by the specified key, given an offset and length.

type BlockType

type BlockType byte

type Bucket

type Bucket interface {
	// Bucket retrieves a nested bucket with the given key.  Returns nil if
	// the bucket does not exist.
	Bucket(key []byte) Bucket

	// CreateBucket creates and returns a new nested bucket with the given
	// key.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrBucketExists if the bucket already exists
	//   - ErrBucketNameRequired if the key is empty
	//   - ErrIncompatibleValue if the key is otherwise invalid for the
	//     particular implementation
	//   - ErrTxNotWritable if attempted against a read-only transaction
	//   - ErrTxClosed if the transaction has already been closed
	CreateBucket(key []byte) (Bucket, error)

	// CreateBucketIfNotExists creates and returns a new nested bucket with
	// the given key if it does not already exist.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrBucketNameRequired if the key is empty
	//   - ErrIncompatibleValue if the key is otherwise invalid for the
	//     particular implementation
	//   - ErrTxNotWritable if attempted against a read-only transaction
	//   - ErrTxClosed if the transaction has already been closed
	CreateBucketIfNotExists(key []byte) (Bucket, error)

	// DeleteBucket removes a nested bucket with the given key.  This also
	// includes removing all nested buckets and keys under the bucket being
	// deleted.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrBucketNotFound if the specified bucket does not exist
	//   - ErrTxNotWritable if attempted against a read-only transaction
	//   - ErrTxClosed if the transaction has already been closed
	DeleteBucket(key []byte) error

	// ForEach invokes the passed function with every key/value pair in the
	// bucket.  This does not include nested buckets or the key/value pairs
	// within those nested buckets.
	//
	// WARNING: It is not safe to mutate data while iterating with this
	// method.  Doing so may cause the underlying cursor to be invalidated
	// and return unexpected keys and/or values.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrTxClosed if the transaction has already been closed
	//
	// NOTE: The slices returned by this function are only valid during a
	// transaction.  Attempting to access them after a transaction has ended
	// results in undefined behavior.  Additionally, the slices must NOT
	// be modified by the caller.  These constraints prevent additional data
	// copies and allows support for memory-mapped database implementations.
	ForEach(func(k, v []byte) error) error

	// ForEachBucket invokes the passed function with the key of every
	// nested bucket in the current bucket.  This does not include any
	// nested buckets within those nested buckets.
	//
	// WARNING: It is not safe to mutate data while iterating with this
	// method.  Doing so may cause the underlying cursor to be invalidated
	// and return unexpected keys and/or values.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrTxClosed if the transaction has already been closed
	//
	// NOTE: The keys returned by this function are only valid during a
	// transaction.  Attempting to access them after a transaction has ended
	// results in undefined behavior.  This constraint prevents additional
	// data copies and allows support for memory-mapped database
	// implementations.
	ForEachBucket(func(k []byte) error) error

	// Cursor returns a new cursor, allowing for iteration over the bucket's
	// key/value pairs and nested buckets in forward or backward order.
	//
	// You must seek to a position using the First, Last, or Seek functions
	// before calling the Next, Prev, Key, or Value functions.  Failure to
	// do so will result in the same return values as an exhausted cursor,
	// which is false for the Prev and Next functions and nil for Key and
	// Value functions.
	Cursor() Cursor

	// Writable returns whether or not the bucket is writable.
	Writable() bool

	// Put saves the specified key/value pair to the bucket.  Keys that do
	// not already exist are added and keys that already exist are
	// overwritten.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrKeyRequired if the key is empty
	//   - ErrIncompatibleValue if the key is the same as an existing bucket
	//   - ErrTxNotWritable if attempted against a read-only transaction
	//   - ErrTxClosed if the transaction has already been closed
	//
	// NOTE: The slices passed to this function must NOT be modified by the
	// caller.  This constraint prevents the requirement for additional data
	// copies and allows support for memory-mapped database implementations.
	Put(key, value []byte) error

	// Get returns the value for the given key.  Returns nil if the key does
	// not exist in this bucket.  An empty slice is returned for keys that
	// exist but have no value assigned.
	//
	// NOTE: The value returned by this function is only valid during a
	// transaction.  Attempting to access it after a transaction has ended
	// results in undefined behavior.  Additionally, the value must NOT
	// be modified by the caller.  These constraints prevent additional data
	// copies and allows support for memory-mapped database implementations.
	Get(key []byte) []byte

	// Delete removes the specified key from the bucket.  Deleting a key
	// that does not exist does not return an error.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrKeyRequired if the key is empty
	//   - ErrIncompatibleValue if the key is the same as an existing bucket
	//   - ErrTxNotWritable if attempted against a read-only transaction
	//   - ErrTxClosed if the transaction has already been closed
	Delete(key []byte) error
}

Bucket represents a collection of key/value pairs.

type Cursor

type Cursor interface {
	// Bucket returns the bucket the cursor was created for.
	Bucket() Bucket

	// Delete removes the current key/value pair the cursor is at without
	// invalidating the cursor.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrIncompatibleValue if attempted when the cursor points to a
	//     nested bucket
	//   - ErrTxNotWritable if attempted against a read-only transaction
	//   - ErrTxClosed if the transaction has already been closed
	Delete() error

	// First positions the cursor at the first key/value pair and returns
	// whether or not the pair exists.
	First() bool

	// Last positions the cursor at the last key/value pair and returns
	// whether or not the pair exists.
	Last() bool

	// Next moves the cursor one key/value pair forward and returns whether
	// or not the pair exists.
	Next() bool

	// Prev moves the cursor one key/value pair backward and returns whether
	// or not the pair exists.
	Prev() bool

	// Seek positions the cursor at the first key/value pair that is greater
	// than or equal to the passed seek key.  Returns whether or not the
	// pair exists.
	Seek(seek []byte) bool

	// Key returns the current key the cursor is pointing to.
	Key() []byte

	// Value returns the current value the cursor is pointing to.  This will
	// be nil for nested buckets.
	Value() []byte
}

Cursor represents a cursor over key/value pairs and nested buckets of a bucket.

Note that open cursors are not tracked on bucket changes and any modifications to the bucket, with the exception of Cursor.Delete, invalidates the cursor. After invalidation, the cursor must be repositioned, or the keys and values returned may be unpredictable.

type Database

type Database interface {
	Type() string

	Putter
	Deleter
	Queryer

	Transactor

	NewBatch() Batch
}

Database wraps all database operations. All methods are safe for concurrent use.

type Deleter

type Deleter interface {
	Delete(key []byte) error
}

Deleter wraps the database delete operation supported by both batches and regular databases.

type Error

type Error struct {
	ErrorCode   ErrorCode // Describes the kind of error
	Description string    // Human readable description of the issue
	Err         error     // Underlying error
}

Error provides a single type for errors that can happen during database operation. It is used to indicate several types of failures including errors with caller requests such as specifying invalid block regions or attempting to access data against closed database transactions, driver errors, errors retrieving data, and errors communicating with database servers.

The caller can use type assertions to determine if an error is an Error and access the ErrorCode field to ascertain the specific reason for the failure.

The ErrDriverSpecific error code will also have the Err field set with the underlying error. Depending on the backend driver, the Err field might be set to the underlying error for other error codes as well.

func ConvertErr

func ConvertErr(desc string, ldbErr error) Error

convertErr converts the passed leveldb error into a database error with an equivalent error code and the passed description. It also sets the passed error as the underlying error.

func MakeError

func MakeError(c ErrorCode, desc string, err error) Error

makeError creates an Error given a set of arguments. The error code must be one of the error codes provided by this package.

func (Error) Error

func (e Error) Error() string

Error satisfies the error interface and prints human-readable errors.

type ErrorCode

type ErrorCode int

ErrorCode identifies a kind of error.

const (

	// ErrDbTypeRegistered indicates two different database drivers
	// ErrDbTypeRegistered indicates two different database drivers
	// attempt to register with the name database type.
	ErrDbTypeRegistered ErrorCode = iota

	// ErrDbUnknownType indicates there is no driver registered for
	// the specified database type.
	ErrDbUnknownType

	// ErrDbDoesNotExist indicates open is called for a database that
	// does not exist.
	ErrDbDoesNotExist

	// ErrDbExists indicates create is called for a database that
	// already exists.
	ErrDbExists

	// ErrDbNotOpen indicates a database instance is accessed before
	// it is opened or after it is closed.
	ErrDbNotOpen

	// ErrDbAlreadyOpen indicates open was called on a database that
	// is already open.
	ErrDbAlreadyOpen

	// ErrInvalid indicates the specified database is not valid.
	ErrInvalid

	// ErrCorruption indicates a checksum failure occurred which invariably
	// means the database is corrupt.
	ErrCorruption

	// ErrTxClosed indicates an attempt was made to commit or rollback a
	// transaction that has already had one of those operations performed.
	ErrTxClosed

	// ErrTxNotWritable indicates an operation that requires write access to
	// the database was attempted against a read-only transaction.
	ErrTxNotWritable

	// ErrBucketNotFound indicates an attempt to access a bucket that has
	// not been created yet.
	ErrBucketNotFound

	// ErrBucketExists indicates an attempt to create a bucket that already
	// exists.
	ErrBucketExists

	// ErrBucketNameRequired indicates an attempt to create a bucket with a
	// blank name.
	ErrBucketNameRequired

	// ErrKeyRequired indicates at attempt to insert a zero-length key.
	ErrKeyRequired

	// ErrKeyTooLarge indicates an attmempt to insert a key that is larger
	// than the max allowed key size.  The max key size depends on the
	// specific backend driver being used.  As a general rule, key sizes
	// should be relatively, so this should rarely be an issue.
	ErrKeyTooLarge

	// ErrValueTooLarge indicates an attmpt to insert a value that is larger
	// than max allowed value size.  The max key size depends on the
	// specific backend driver being used.
	ErrValueTooLarge

	// ErrIncompatibleValue indicates the value in question is invalid for
	// the specific requested operation.  For example, trying create or
	// delete a bucket with an existing non-bucket key, attempting to create
	// or delete a non-bucket key with an existing bucket key, or trying to
	// delete a value via a cursor when it points to a nested bucket.
	ErrIncompatibleValue

	// ErrBlockNotFound indicates a block with the provided hash does not
	// exist in the database.
	ErrBlockNotFound

	// ErrBlockExists indicates a block with the provided hash already
	// exists in the database.
	ErrBlockExists

	// ErrBlockRegionInvalid indicates a region that exceeds the bounds of
	// the specified block was requested.  When the hash provided by the
	// region does not correspond to an existing block, the error will be
	// ErrBlockNotFound instead.
	ErrBlockRegionInvalid

	// ErrDriverSpecific indicates the Err field is a driver-specific error.
	// This provides a mechanism for drivers to plug-in their own custom
	// errors for any situations which aren't already covered by the error
	// codes provided by this package.
	ErrDriverSpecific
)

These constants are used to identify a specific database Error.

func (ErrorCode) String

func (e ErrorCode) String() string

String returns the ErrorCode as a human-readable name.

type Putter

type Putter interface {
	Put(key []byte, value []byte) error
}

Putter wraps the database write operation supported by both batches and regular databases.

type Queryer

type Queryer interface {
	Get(key []byte) ([]byte, error)
	Has(key []byte) (bool, error)
}

type Transactor

type Transactor interface {
	Begin(writable bool) (Tx, error)
	View(fn func(tx Tx) error) error
	Update(fn func(tx Tx) error) error
	Close() error
}

type Tx

type Tx interface {
	// StoreBlock stores the provided block into the database.  There are no
	// checks to ensure the block connects to a previous block, contains
	// double spends, or any additional functionality such as transaction
	// indexing.  It simply stores the block in the database.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrBlockExists when the block key already exists
	//   - ErrTxNotWritable if attempted against a read-only transaction
	//   - ErrTxClosed if the transaction has already been closed
	//
	// Other errors are possible depending on the implementation.
	StoreBlock(blockKey *BlockKey, blockBytes []byte) error

	// HasBlock returns whether or not a block with the given key exists
	// in the database.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrTxClosed if the transaction has already been closed
	//
	// Other errors are possible depending on the implementation.
	HasBlock(key *BlockKey) (bool, error)

	// HasBlocks returns whether or not the blocks with the provided keys
	// exist in the database.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrTxClosed if the transaction has already been closed
	//
	// Other errors are possible depending on the implementation.
	HasBlocks(keys []BlockKey) ([]bool, error)

	// FetchBlockHeader returns the raw serialized bytes for the block
	// header identified by the given key.  The raw bytes are in the format
	// returned by Serialize on a protos.BlockHeader.
	//
	// It is highly recommended to use this function (or FetchBlockHeaders)
	// to obtain block headers over the FetchBlockRegion(s) functions since
	// it provides the backend drivers the freedom to perform very specific
	// optimizations which can result in significant speed advantages when
	// working with headers.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrBlockNotFound if the requested block key does not exist
	//   - ErrTxClosed if the transaction has already been closed
	//   - ErrCorruption if the database has somehow become corrupted
	//
	// NOTE: The data returned by this function is only valid during a
	// database transaction.  Attempting to access it after a transaction
	// has ended results in undefined behavior.  This constraint prevents
	// additional data copies and allows support for memory-mapped database
	// implementations.
	FetchBlockHeader(key *BlockKey) ([]byte, error)

	// FetchBlockHeaders returns the raw serialized bytes for the block
	// headers identified by the given keys.  The raw bytes are in the
	// format returned by Serialize on a protos.BlockHeader.
	//
	// It is highly recommended to use this function (or FetchBlockHeader)
	// to obtain block headers over the FetchBlockRegion(s) functions since
	// it provides the backend drivers the freedom to perform very specific
	// optimizations which can result in significant speed advantages when
	// working with headers.
	//
	// Furthermore, depending on the specific implementation, this function
	// can be more efficient for bulk loading multiple block headers than
	// loading them one-by-one with FetchBlockHeader.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrBlockNotFound if any of the request block keys do not exist
	//   - ErrTxClosed if the transaction has already been closed
	//   - ErrCorruption if the database has somehow become corrupted
	//
	// NOTE: The data returned by this function is only valid during a
	// database transaction.  Attempting to access it after a transaction
	// has ended results in undefined behavior.  This constraint prevents
	// additional data copies and allows support for memory-mapped database
	// implementations.
	FetchBlockHeaders(keys []BlockKey) ([][]byte, error)

	// FetchBlock returns the raw serialized bytes for the block identified
	// by the given key.  The raw bytes are in the format returned by
	// Serialize on a protos.MsgBlock.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrBlockNotFound if the requested block key does not exist
	//   - ErrTxClosed if the transaction has already been closed
	//   - ErrCorruption if the database has somehow become corrupted
	//
	// NOTE: The data returned by this function is only valid during a
	// database transaction.  Attempting to access it after a transaction
	// has ended results in undefined behavior.  This constraint prevents
	// additional data copies and allows support for memory-mapped database
	// implementations.
	FetchBlock(key *BlockKey) ([]byte, error)

	// FetchBlocks returns the raw serialized bytes for the blocks
	// identified by the given keys.  The raw bytes are in the format
	// returned by Serialize on a protos.MsgBlock.
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrBlockNotFound if the any of the requested block keys do not
	//     exist
	//   - ErrTxClosed if the transaction has already been closed
	//   - ErrCorruption if the database has somehow become corrupted
	//
	// NOTE: The data returned by this function is only valid during a
	// database transaction.  Attempting to access it after a transaction
	// has ended results in undefined behavior.  This constraint prevents
	// additional data copies and allows support for memory-mapped database
	// implementations.
	FetchBlocks(keys []BlockKey) ([][]byte, error)

	// FetchBlockRegion returns the raw serialized bytes for the given
	// block region.
	//
	// For example, it is possible to directly extract Bitcoin transactions
	// and/or scripts from a block with this function.  Depending on the
	// backend implementation, this can provide significant savings by
	// avoiding the need to load entire blocks.
	//
	// The raw bytes are in the format returned by Serialize on a
	// protos.MsgBlock and the Offset field in the provided BlockRegion is
	// zero-based and relative to the start of the block (byte 0).
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrBlockNotFound if the requested block key does not exist
	//   - ErrBlockRegionInvalid if the region exceeds the bounds of the
	//     associated block
	//   - ErrTxClosed if the transaction has already been closed
	//   - ErrCorruption if the database has somehow become corrupted
	//
	// NOTE: The data returned by this function is only valid during a
	// database transaction.  Attempting to access it after a transaction
	// has ended results in undefined behavior.  This constraint prevents
	// additional data copies and allows support for memory-mapped database
	// implementations.
	FetchBlockRegion(region *BlockRegion) ([]byte, error)

	// FetchBlockRegions returns the raw serialized bytes for the given
	// block regions.
	//
	// For example, it is possible to directly extract Bitcoin transactions
	// and/or scripts from various blocks with this function.  Depending on
	// the backend implementation, this can provide significant savings by
	// avoiding the need to load entire blocks.
	//
	// The raw bytes are in the format returned by Serialize on a
	// protos.MsgBlock and the Offset fields in the provided BlockRegions are
	// zero-based and relative to the start of the block (byte 0).
	//
	// The interface contract guarantees at least the following errors will
	// be returned (other implementation-specific errors are possible):
	//   - ErrBlockNotFound if any of the requested block do not
	//     exist
	//   - ErrBlockRegionInvalid if one or more region exceed the bounds of
	//     the associated block
	//   - ErrTxClosed if the transaction has already been closed
	//   - ErrCorruption if the database has somehow become corrupted
	//
	// NOTE: The data returned by this function is only valid during a
	// database transaction.  Attempting to access it after a transaction
	// has ended results in undefined behavior.  This constraint prevents
	// additional data copies and allows support for memory-mapped database
	// implementations.
	FetchBlockRegions(regions []BlockRegion) ([][]byte, error)

	// Metadata returns the top-most bucket for all metadata storage.
	Metadata() Bucket

	// Commit commits all changes that have been made to the metadata or
	// block storage.  Depending on the backend implementation this could be
	// to a cache that is periodically synced to persistent storage or
	// directly to persistent storage.  In any case, all transactions which
	// are started after the commit finishes will include all changes made
	// by this transaction.  Calling this function on a managed transaction
	// will result in a panic.
	Commit() error

	// Rollback undoes all changes that have been made to the metadata or
	// block storage.  Calling this function on a managed transaction will
	// result in a panic.
	Rollback() error
}

Tx represents a database transaction. It can either by read-only or read-write. The transaction provides a metadata bucket against which all read and writes occur.

As would be expected with a transaction, no changes will be saved to the database until it has been committed. The transaction will only provide a view of the database at the time it was created. Transactions should not be long running operations.