btcchain

package module
v0.0.0-...-4151416 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 4, 2013 License: ISC Imports: 15 Imported by: 0

README

btcchain

Package btcchain implements bitcoin block handling and chain selection rules. The test coverage is currently only around 60%, but will be increasing over time. See test_coverage.txt for the gocov coverage report. Alternatively, if you are running a POSIX OS, you can run the cov_report.sh script for a real-time report. Package btcchain is licensed under the liberal ISC license.

There is an associated blog post about the release of this package here.

This package is one of the core packages from btcd, an alternative full-node implementation of bitcoin which is under active development by Conformal. Although it was primarily written for btcd, this package has intentionally been designed so it can be used as a standalone package for any projects needing to handle processing of blocks into the bitcoin block chain.

Documentation

Full go doc style documentation for the project can be viewed online without installing this package by using the GoDoc site here: http://godoc.org/github.com/conformal/btcchain

You can also view the documentation locally once the package is installed with the godoc tool by running godoc -http=":6060" and pointing your browser to http://localhost:6060/pkg/github.com/conformal/btcchain

Installation

$ go get github.com/conformal/btcchain

Bitcoin Chain Processing Overview

Before a block is allowed into the block chain, it must go through an intensive series of validation rules. The following list serves as a general outline of those rules to provide some intuition into what is going on under the hood, but is by no means exhaustive:

  • Reject duplicate blocks
  • Perform a series of sanity checks on the block and its transactions such as verifying proof of work, timestamps, number and character of transactions, transaction amounts, script complexity, and merkle root calculations
  • Compare the block against predetermined checkpoints for expected timestamps and difficulty based on elapsed time since the checkpoint
  • Save the most recent orphan blocks for a limited time in case their parent blocks become available
  • Stop processing if the block is an orphan as the rest of the processing depends on the block's position within the block chain
  • Perform a series of more thorough checks that depend on the block's position within the block chain such as verifying block difficulties adhere to difficulty retarget rules, timestamps are after the median of the last several blocks, all transactions are finalized, checkpoint blocks match, and block versions are in line with the previous blocks
  • Determine how the block fits into the chain and perform different actions accordingly in order to ensure any side chains which have higher difficulty than the main chain become the new main chain
  • When a block is being connected to the main chain (either through reorganization of a side chain to the main chain or just extending the main chain), perform further checks on the block's transactions such as verifying transaction duplicates, script complexity for the combination of connected scripts, coinbase maturity, double spends, and connected transaction values
  • Run the transaction scripts to verify the spender is allowed to spend the coins
  • Insert the block into the block database

Block Processing Example

The following example program demonstrates processing a block. This example intentionally causes an error by attempting to process a duplicate block.

	package main

	import (
		"fmt"
		"github.com/conformal/btcchain"
		"github.com/conformal/btcdb"
		_ "github.com/conformal/btcdb/sqlite3"
		"github.com/conformal/btcutil"
		"github.com/conformal/btcwire"
		"os"
	)

	func main() {
		// Create a new database to store the accepted blocks into.  Typically
		// this would be opening an existing database, but we create a new db
		// here so this is a complete working example.  Also, typically the
		// calls to os.Remove would not be used either, but again, we want
		// a complete working example here, so we make sure to remove the
		// database.
		dbName := "example.db"
		_ = os.Remove(dbName)
		db, err := btcdb.CreateDB("sqlite", dbName)
		if err != nil {
			fmt.Printf("Failed to create database: %v\n", err)
			return
		}
		defer os.Remove(dbName) // Ignore error.
		defer db.Close()

		// Insert the main network genesis block.  This is part of the initial
		// database setup.  Like above, this typically would not be needed when
		// opening an existing database.
		genesisBlock := btcutil.NewBlock(&btcwire.GenesisBlock)
		_, err = db.InsertBlock(genesisBlock)
		if err != nil {
			fmt.Printf("Failed to insert genesis block: %v\n", err)
			return
		}

		// Create a new BlockChain instance using the underlying database for
		// the main bitcoin network and ignore notifications.
		chain := btcchain.New(db, btcwire.MainNet, nil)

		// Process a block.  For this example, we are going to intentionally
		// cause an error by trying to process the genesis block which already
		// exists.
		err = chain.ProcessBlock(genesisBlock)
		if err != nil {
			fmt.Printf("Failed to process block: %v\n", err)
			return
		}
	}

TODO

  • Increase test coverage
  • Expose some APIs for block verification (without actually inserting it)

GPG Verification Key

All official release tags are signed by Conformal so users can ensure the code has not been tampered with and is coming from Conformal. To verify the signature perform the following:

  • Download the public key from the Conformal website at https://opensource.conformal.com/GIT-GPG-KEY-conformal.txt

  • Import the public key into your GPG keyring:

    gpg --import GIT-GPG-KEY-conformal.txt

  • Verify the release tag with the following command where TAG_NAME is a placeholder for the specific tag:

    git tag -v TAG_NAME

License

Package btcchain is licensed under the liberal ISC License.

Documentation

Overview

Package btcchain implements bitcoin block handling and chain selection rules.

The bitcoin block handling and chain selection rules are an integral, and quite likely the most important, part of bitcoin. Unfortunately, at the time of this writing, these rules are also largely undocumented and had to be ascertained from the bitcoind source code. At its core, bitcoin is a distributed consensus of which blocks are valid and which ones will comprise the main block chain (public ledger) that ultimately determines accepted transactions, so it is extremely important that fully validating nodes agree on all rules.

At a high level, this package provides support for inserting new blocks into the block chain according to the aforementioned rules. It includes functionality such as rejecting duplicate blocks, ensuring blocks and transactions follow all rules, orphan handling, and best chain selection along with reorganization.

Since this package does not deal with other bitcoin specifics such as network communication or wallets, it provides a notification system which gives the caller a high level of flexibility in how they want to react to certain events such as orphan blocks which need their parents requested and newly connected main chain blocks which might result in wallet updates.

Bitcoin Chain Processing Overview

Before a block is allowed into the block chain, it must go through an intensive series of validation rules. The following list serves as a general outline of those rules to provide some intuition into what is going on under the hood, but is by no means exhaustive:

  • Reject duplicate blocks
  • Perform a series of sanity checks on the block and its transactions such as verifying proof of work, timestamps, number and character of transactions, transaction amounts, script complexity, and merkle root calculations
  • Compare the block against predetermined checkpoints for expected timestamps and difficulty based on elapsed time since the checkpoint
  • Save the most recent orphan blocks for a limited time in case their parent blocks become available
  • Stop processing if the block is an orphan as the rest of the processing depends on the block's position within the block chain
  • Perform a series of more thorough checks that depend on the block's position within the block chain such as verifying block difficulties adhere to difficulty retarget rules, timestamps are after the median of the last several blocks, all transactions are finalized, checkpoint blocks match, and block versions are in line with the previous blocks
  • Determine how the block fits into the chain and perform different actions accordingly in order to ensure any side chains which have higher difficulty than the main chain become the new main chain
  • When a block is being connected to the main chain (either through reorganization of a side chain to the main chain or just extending the main chain), perform further checks on the block's transactions such as verifying transaction duplicates, script complexity for the combination of connected scripts, coinbase maturity, double spends, and connected transaction values
  • Run the transaction scripts to verify the spender is allowed to spend the coins
  • Insert the block into the block database

Block Processing Example

The following example program demonstrates processing a block. This example intentionally causes an error by attempting to process a duplicate block. 

package main

import (
	"fmt"
	"github.com/conformal/btcchain"
	"github.com/conformal/btcdb"
	_ "github.com/conformal/btcdb/sqlite3"
	"github.com/conformal/btcutil"
	"github.com/conformal/btcwire"
	"os"
)

func main() {
	// Create a new database to store the accepted blocks into.  Typically
	// this would be opening an existing database, but we create a new db
	// here so this is a complete working example.  Also, typically the
	// calls to os.Remove would not be used either, but again, we want
	// a complete working example here, so we make sure to remove the
	// database.
	dbName := "example.db"
	_ = os.Remove(dbName)
	db, err := btcdb.CreateDB("sqlite", dbName)
	if err != nil {
		fmt.Printf("Failed to create database: %v\n", err)
		return
	}
	defer os.Remove(dbName) // Ignore error.
	defer db.Close()

	// Insert the main network genesis block.  This is part of the initial
	// database setup.  Like above, this typically would not be needed when
	// opening an existing database.
	genesisBlock := btcutil.NewBlock(&btcwire.GenesisBlock)
	_, err = db.InsertBlock(genesisBlock)
	if err != nil {
		fmt.Printf("Failed to insert genesis block: %v\n", err)
		return
	}

	// Create a new BlockChain instance using the underlying database for
	// the main bitcoin network and ignore notifications.
	chain := btcchain.New(db, btcwire.MainNet, nil)

	// Process a block.  For this example, we are going to intentionally
	// cause an error by trying to process the genesis block which already
	// exists.
	err = chain.ProcessBlock(genesisBlock)
	if err != nil {
		fmt.Printf("Failed to process block: %v\n", err)
		return
	}
}

Errors

Errors returned by this package are either the raw errors provided by underlying calls or of type btcchain.RuleError. This allows the caller to differentiate between unexpected errors, such as database errors, versus errors due to rule violations through type assertions.

Bitcoin Improvement Proposals

This package includes spec changes outlined by the following BIPs: 

BIP0016 (https://en.bitcoin.it/wiki/BIP_0016)
BIP0030 (https://en.bitcoin.it/wiki/BIP_0030)
BIP0034 (https://en.bitcoin.it/wiki/BIP_0034)

Index

Constants

View Source 
const CheckpointConfirmations = 2016

CheckpointConfirmations is the number of blocks before the end of the current best block chain that a good checkpoint candidate must be.

Variables

View Source 
var ErrIndexAlreadyInitialized = errors.New("the block index can only be " +
	"initialized before it has been modified")

ErrIndexAlreadyInitialized describes an error that indicates the block index is already initialized.

Functions

func BigToCompact 

func BigToCompact(n *big.Int) uint32

BigToCompact converts a whole number N to a compact representation using an unsigned 32-bit number. The compact representation only provides 23 bits of precision, so values larger than (2^23 - 1) only encode the most significant digits of the number. See CompactToBig for details.

func BuildMerkleTreeStore 

func BuildMerkleTreeStore(block *btcutil.Block) []*btcwire.ShaHash

BuildMerkleTreeStore creates a merkle tree from block, stores it using a linear array, and returns a slice of the backing array. A linear array was chosen as opposed to an actual tree structure since it uses about half as much memory. The following describes a merkle tree and how it is stored in a linear array.

A merkle tree is a tree in which every non-leaf node is the hash of its children nodes. A diagram depicting how this works for bitcoin transactions where h(x) is a double sha256 follows: 

         root = h1234 = h(h12 + h34)
        /                           \
  h12 = h(h1 + h2)            h34 = h(h3 + h4)
   /            \              /            \
h1 = h(tx1)  h2 = h(tx2)    h3 = h(tx3)  h4 = h(tx4)

The above stored as a linear array is as follows: 

[h1 h2 h3 h4 h12 h34 root]

As the above shows, the merkle root is always the last element in the array.

The number of inputs is not always a power of two which results in a balanced tree structure as above. In that case, parent nodes with no children are also zero and parent nodes with only a single left node are calculated by concatenating the left node with itself before hashing. Since this function uses nodes that are pointers to the hashes, empty nodes will be nil.

func CheckTransactionInputs 

func CheckTransactionInputs(tx *btcwire.MsgTx, txHeight int64, txStore TxStore) (int64, error)

CheckTransactionInputs performs a series of checks on the inputs to a transaction to ensure they are valid. An example of some of the checks include verifying all inputs exist, ensuring the coinbase seasoning requirements are met, detecting double spends, validating all values and fees are in the legal range and the total output amount doesn't exceed the input amount, and verifying the signatures to prove the spender was the owner of the bitcoins and therefore allowed to spend them. As it checks the inputs, it also calculates the total fees for the transaction and returns that value.

func CheckTransactionSanity 

func CheckTransactionSanity(tx *btcwire.MsgTx) error

CheckTransactionSanity performs some preliminary checks on a transaction to ensure it is sane. These checks are context free.

func CompactToBig 

func CompactToBig(compact uint32) *big.Int

CompactToBig converts a compact representation of a whole number N to an unsigned 32-bit number. The representation is similar to IEEE754 floating point numbers.

Like IEEE754 floating point, there are three basic components: the sign, the exponent, and the mantissa. They are broken out as follows:

  • the most significant 8 bits represent the unsigned base 256 exponent

  • bit 23 (the 24th bit) represents the sign bit

  • the least significant 23 bits represent the mantissa

    ------------------------------------------------- | Exponent | Sign | Mantissa | ------------------------------------------------- | 8 bits [31-24] | 1 bit [23] | 23 bits [22-00] | -------------------------------------------------

The formula to calculate N is: 

N = (-1^sign) * mantissa * 256^(exponent-3)

This compact form is only used in bitcoin to encode unsigned 256-bit numbers which represent difficulty targets, thus there really is not a need for a sign bit, but it is implemented here to stay consistent with bitcoind.

func DisableLog 

func DisableLog()

DisableLog disables all library log output. Logging output is disabled by default until either UseLogger or SetLogWriter are called.

func IsCoinBase 

func IsCoinBase(msgTx *btcwire.MsgTx) bool

IsCoinBase determines whether or not a transaction is a coinbase. A coinbase is a special transaction created by miners that has no inputs. This is represented in the block chain by a transaction with a single input that has a previous output transaction index set to the maximum value along with a zero hash.

func IsFinalizedTransaction 

func IsFinalizedTransaction(msgTx *btcwire.MsgTx, blockHeight int64, blockTime time.Time) bool

IsFinalizedTransaction determines whether or not a transaction is finalized.

func SetLogWriter 

func SetLogWriter(w io.Writer) error

SetLogWriter uses a specified io.Writer to output package logging info. This allows a caller to direct package logging output without needing a dependency on seelog. If the caller is also using seelog, UseLogger should be used instead.

func ShaHashToBig 

func ShaHashToBig(hash *btcwire.ShaHash) *big.Int

ShaHashToBig converts a btcwire.ShaHash into a big.Int that can be used to perform math comparisons.

func UseLogger 

func UseLogger(logger seelog.LoggerInterface)

UseLogger uses a specified Logger to output package logging info. This should be used in preference to SetLogWriter if the caller is also using seelog.

func ValidateTransactionScripts 

func ValidateTransactionScripts(tx *btcwire.MsgTx, txHash *btcwire.ShaHash, timestamp time.Time, txStore TxStore) (err error)

ValidateTransactionScripts validates the scripts for the passed transaction using multiple goroutines.

Types

type BlockChain 

type BlockChain struct {
	// contains filtered or unexported fields
}

BlockChain provides functions for working with the bitcoin block chain. It includes functionality such as rejecting duplicate blocks, ensuring blocks follow all rules, orphan handling, checkpoint handling, and best chain selection with reorganization.

func New 

func New(db btcdb.Db, btcnet btcwire.BitcoinNet, c NotificationCallback) *BlockChain

New returns a BlockChain instance for the passed bitcoin network using the provided backing database. It accepts a callback on which notifications will be sent when various events take place. See the documentation for Notification and NotificationType for details on the types and contents of notifications. The provided callback can be nil if the caller is not interested in receiving notifications.

func (*BlockChain) BlockLocatorFromHash 

func (b *BlockChain) BlockLocatorFromHash(hash *btcwire.ShaHash) BlockLocator

BlockLocatorFromHash returns a block locator for the passed block hash. See BlockLocator for details on the algotirhm used to create a block locator.

In addition to the general algorithm referenced above, there are a couple of special cases which are handled:

  • If the genesis hash is passed, there are no previous hashes to add and therefore the block locator will only consist of the genesis hash
  • If the passed hash is not currently known, the block locator will only consist of the passed hash

func (*BlockChain) DisableCheckpoints 

func (b *BlockChain) DisableCheckpoints(disable bool)

DisableCheckpoints provides a mechanism to disable validation against checkpoints which you DO NOT want to do in production. It is provided only for debug purposes.

func (*BlockChain) DisableVerify 

func (b *BlockChain) DisableVerify(disable bool)

DisableVerify provides a mechanism to disable transaction script validation which you DO NOT want to do in production as it could allow double spends and othe undesirable things. It is provided only for debug purposes since script validation is extremely intensive and when debugging it is sometimes nice to quickly get the chain.

func (*BlockChain) FetchTransactionStore 

func (b *BlockChain) FetchTransactionStore(tx *btcwire.MsgTx) (TxStore, error)

FetchTransactionStore fetches the input transactions referenced by the passed transaction from the point of view of the end of the main chain. It also attempts to fetch the transaction itself so the returned TxStore can be examined for duplicate transactions.

func (*BlockChain) GenerateInitialIndex 

func (b *BlockChain) GenerateInitialIndex() error

GenerateInitialIndex is an optional function which generates the required number of initial block nodes in an optimized fashion. This is optional because the memory block index is sparse and previous nodes are dynamically loaded as needed. However, during initial startup (when there are no nodes in memory yet), dynamically loading all of the required nodes on the fly in the usual way is much slower than preloading them.

This function can only be called once and it must be called before any nodes are added to the block index. ErrIndexAlreadyInitialized is returned if the former is not the case. In practice, this means the function should be called directly after New.

func (*BlockChain) GetOrphanRoot 

func (b *BlockChain) GetOrphanRoot(hash *btcwire.ShaHash) *btcwire.ShaHash

GetOrphanRoot returns the head of the chain for the provided hash from the map of orphan blocks.

This function is safe for concurrent access.

func (*BlockChain) HaveInventory 

func (b *BlockChain) HaveInventory(inventoryVector *btcwire.InvVect) bool

HaveInventory returns whether or not the chain instance has the inventory represented by the passed inventory vector. This includes checking all of the various places inventory can be when it is in different states such as part of the main chain, on a side chain, and orphans.

This function is safe for concurrent access.

func (*BlockChain) IsCheckpointCandidate 

func (b *BlockChain) IsCheckpointCandidate(block *btcutil.Block) (bool, error)

IsCheckpointCandidate returns whether or not the passed block is a good checkpoint candidate.

The factors used to determine a good checkpoint are:

  • The block must be in the main chain
  • The block must be at least 'CheckpointConfirmations' blocks prior to the current end of the main chain
  • The timestamps for the blocks before and after the checkpoint must have timestamps which are also before and after the checkpoint, respectively (due to the median time allowance this is not always the case)
  • The block must not contain any strange transaction such as those with nonstandard scripts

func (*BlockChain) IsCurrent 

func (b *BlockChain) IsCurrent() bool

IsCurrent returns whether or not the chain believes it is current. Several factors are used to guess, but the key factors that allow the chain to believe it is current are:

  • Latest block height is after the latest checkpoint (if enabled)
  • Latest block has a timestamp newer than 24 hours ago

This function is NOT safe for concurrent access.

func (*BlockChain) IsKnownOrphan 

func (b *BlockChain) IsKnownOrphan(hash *btcwire.ShaHash) bool

IsKnownOrphan returns whether the passed hash is currently a known orphan. Keep in mind that only a limited number of orphans are held onto for a limited amount of time, so this function must not be used as an absolute way to test if a block is an orphan block. A full block (as opposed to just its hash) must be passed to ProcessBlock for that purpose. However, calling ProcessBlock with an orphan that already exists results in an error, so this function provides a mechanism for a caller to intelligently detect *recent* duplicate orphans and react accordingly.

This function is safe for concurrent access.

func (*BlockChain) LatestBlockLocator 

func (b *BlockChain) LatestBlockLocator() (BlockLocator, error)

LatestBlockLocator returns a block locator for the latest known tip of the main (best) chain.

func (*BlockChain) LatestCheckpoint 

func (b *BlockChain) LatestCheckpoint() *Checkpoint

LatestCheckpoint returns the most recent checkpoint (regardless of whether it is already known). When checkpoints are disabled or there are no checkpoints for the active network, it will return nil.

func (*BlockChain) ProcessBlock 

func (b *BlockChain) ProcessBlock(block *btcutil.Block) error

ProcessBlock is the main workhorse for handling insertion of new blocks into the block chain. It includes functionality such as rejecting duplicate blocks, ensuring blocks follow all rules, orphan handling, and insertion into the block chain along with best chain selection and reorganization.

type BlockLocator 

type BlockLocator []*btcwire.ShaHash

BlockLocator is used to help locate a specific block. The algorithm for building the block locator is to add the hashes in reverse order until the genesis block is reached. In order to keep the list of locator hashes to a reasonable number of entries, first the most recent previous 10 block hashes are added, then the step is doubled each loop iteration to exponentially decrease the number of hashes as a function of the distance from the block being located.

type Checkpoint 

type Checkpoint struct {
	Height int64
	Hash   *btcwire.ShaHash
}

Checkpoint identifies a known good point in the block chain. Using checkpoints allows a few optimizations for old blocks during initial download and also prevents forks from old blocks.

Each checkpoint is selected by the core developers based upon several factors. See the documentation for IsCheckpointCandidate for details on the selection criteria.

As alluded to above, this package provides an IsCheckpointCandidate function which programatically identifies a block as a checkpoint candidate. The idea is that candidates are reviewed by a developer to make the final decision and then manually added to the list of checkpoints.

type Notification 

type Notification struct {
	Type NotificationType
	Data interface{}
}

Notification defines notification that is sent to the caller via the callback function provided during the call to New and consists of a notification type as well as associated data that depends on the type as follows:

  • NTOrphanBlock: *btcwire.ShaHash
  • NTBlockAccepted: *btcutil.Block
  • NTBlockConnected: *btcutil.Block
  • NTBlockDisconnected: *btcutil.Block

type NotificationCallback 

type NotificationCallback func(*Notification)

NotificationCallback is used for a caller to provide a callback for notifications about various chain events.

type NotificationType 

type NotificationType int

NotificationType represents the type of a notification message. 

const (
	// NTOrphanBlock indicates an orphan block was processed and the
	// associated block hash should be passed to the GetOrphanRoot function
	// to find the root of all known orphans which should then be used to
	// request the missing blocks.
	NTOrphanBlock NotificationType = iota

	// NTBlockAccepted indicates the associated block was accepted into
	// the block chain.  Note that this does not necessarily mean it was
	// added to the main chain.  For that, use NTBlockConnected.
	NTBlockAccepted

	// NTBlockConnected indicates the associated block was connected to the
	// main chain.
	NTBlockConnected

	// NTBlockDisconnected indicates the associated block was disconnected
	// from the main chain.
	NTBlockDisconnected
)

Constants for the type of a notification message.

func (NotificationType) String 

func (n NotificationType) String() string

String returns the NotificationType in human-readable form.

type Params 

type Params struct {
	// GenesisBlock is the genesis block for the specific network.
	GenesisBlock *btcwire.MsgBlock

	// GenesisHash is the genesis block hash for the specific network.
	GenesisHash *btcwire.ShaHash

	// PowLimit is the highest proof of work value a bitcoin block can have
	// for the specific network.
	PowLimit *big.Int

	// PowLimitBits is the highest proof of work value a bitcoin block can
	// have represented in compact form.  See CompactToBig for more details
	// on compact form.
	PowLimitBits uint32
}

Params houses parameters unique to various bitcoin networks such as the main network and test networks. See ChainParams.

func ChainParams 

func ChainParams(btcnet btcwire.BitcoinNet) *Params

ChainParams returns chain parameters specific to the passed bitcoin network. It returns the parameters for btcwire.MainNet if the passed network is not supported.

type RuleError 

type RuleError string

RuleError identifies a rule violation. It is used to indicate that processing of a block or transaction failed due to one of the many validation rules. The caller can use type assertions to determine if a failure was specifically due to a rule violation.

func (RuleError) Error 

func (e RuleError) Error() string

Error satisfies the error interface to print human-readable errors.

type TxData 

type TxData struct {
	Tx          *btcwire.MsgTx
	Hash        *btcwire.ShaHash
	BlockHeight int64
	Spent       []bool
	Err         error
}

TxData contains contextual information about transactions such as which block they were found in and whether or not the outputs are spent.

type TxStore 

type TxStore map[btcwire.ShaHash]*TxData

TxStore is used to store transactions needed by other transactions for things such as script validation and double spend prevention. This also allows the transaction data to be treated as a view since it can contain the information from the point-of-view of different points in the chain.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.