statedb

package
v1.9.0-rc.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 8, 2022 License: GPL-3.0 Imports: 31 Imported by: 24

Documentation

Overview

Package statedb implements the Merkle Patricia Trie structure used for state object trie. This package is used to read/write data from/to the state object trie.

Overview of statedb package

There are 3 key struct in this package: Trie, SecureTrie and Database.

Trie struct represents a Merkle Patricia Trie.

SecureTrie is a basically same as Trie but it wraps a trie with key hashing. In a SecureTrie, all access operations hash the key using keccak256. This prevents calling code from creating long chains of nodes that increase the access time.

Database is an intermediate write layer between the trie data structures and the disk database. The aim is to accumulate trie writes in-memory and only periodically flush a couple tries to disk, garbage collecting the remainder.

Source Files

Related functions and variables are defined in the files listed below

  • database.go : Implementation of Database struct
  • db_migration.go : Implementation of DB migration
  • derive_sha.go : Implementation of DeriveShaOrig used in Klaytn
  • encoding.go : Implementation of 3 encodings: KEYBYTES, HEX and COMPACT
  • errors.go : Errors used in this package
  • hasher.go : Implementation of recursive and bottom-up hashing
  • iterator.go : Implementation of key-value trie iterator that traverses a Trie
  • node.go : Implementation of 4 types of nodes, used in Merkle Patricia Trie
  • proof.go : Functions which construct a Merkle Patricia Proof for the given key
  • secure_trie.go : Implementation of Merkle Patricia Trie with key hashing
  • sync.go : Implementation of state trie sync
  • trie.go : Implementation of Merkle Patricia Trie

Copyright 2019 The go-ethereum Authors This file is part of the go-ethereum library.

The go-ethereum library is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

The go-ethereum library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.

This file is derived from trie/sync_test.go (2020/05/20). Modified and improved for the klaytn development.

Index

Constants

View Source 
const (
	// Available trie node cache types
	CacheTypeLocal  TrieNodeCacheType = "LocalCache"
	CacheTypeRedis                    = "RemoteCache"
	CacheTypeHybrid                   = "HybridCache"
)
View Source 
const AutoScaling = -1

AutoScaling is for auto-scaling cache size. If cacheSize is set to this value, cache size is set scaling to physical memeory

Variables

View Source 
var ErrAlreadyProcessed = errors.New("already processed")

ErrAlreadyProcessed is returned by the trie sync when it's requested to process a node it already processed previously.

View Source 
var ErrNotRequested = errors.New("not requested")

ErrNotRequested is returned by the trie sync when it's requested to process a node it did not request.

View Source 
var ErrZeroHashNode = errors.New("cannot retrieve a node which has 0x00 hash value")

Functions

func GetHashAndHexKey 

func GetHashAndHexKey(key []byte) ([]byte, []byte)

func HexPathToString added in v1.5.0 

func HexPathToString(path []byte) string

HexPathToString turns hex nibbles of trie path key into string of the trie path key.

func VerifyProof 

func VerifyProof(rootHash common.Hash, key []byte, proofDB database.DBManager) (value []byte, err error, nodes int)

VerifyProof checks merkle proofs. The given proof must contain the value for key in a trie with the given root hash. VerifyProof returns an error if the proof contains invalid trie nodes or the wrong value.

func VerifyRangeProof added in v1.8.0 

func VerifyRangeProof(rootHash common.Hash, firstKey []byte, lastKey []byte, keys [][]byte, values [][]byte, proof ProofDBReader) (bool, error)

VerifyRangeProof checks whether the given leaf nodes and edge proof can prove the given trie leaves range is matched with the specific root. Besides, the range should be consecutive (no gap inside) and monotonic increasing.

Note the given proof actually contains two edge proofs. Both of them can be non-existent proofs. For example the first proof is for a non-existent key 0x03, the last proof is for a non-existent key 0x10. The given batch leaves are [0x04, 0x05, .. 0x09]. It's still feasible to prove the given batch is valid.

The firstKey is paired with firstProof, not necessarily the same as keys[0] (unless firstProof is an existent proof). Similarly, lastKey and lastProof are paired.

Expect the normal case, this function can also be used to verify the following range proofs:

  • All elements proof. In this case the proof can be nil, but the range should be all the leaves in the trie.
  • One element proof. In this case no matter the edge proof is a non-existent proof or not, we can always verify the correctness of the proof.
  • Zero element proof. In this case a single non-existent proof is enough to prove. Besides, if there are still some other leaves available on the right side, then an error will be returned.

Except returning the error to indicate the proof is valid or not, the function will also return a flag to indicate whether there exists more accounts/slots in the trie.

Note: This method does not verify that the proof is of minimal form. If the input proofs are 'bloated' with neighbour leaves or random data, aside from the 'useful' data, then the proof will still be accepted.

Types

type BlockPubSub added in v1.5.3 

type BlockPubSub interface {
	PublishBlock(msg string) error
	SubscribeBlockCh() <-chan *redis.Message
	UnsubscribeBlock() error
}

type Database 

type Database struct {
	// contains filtered or unexported fields
}

Database is an intermediate write layer between the trie data structures and the disk database. The aim is to accumulate trie writes in-memory and only periodically flush a couple tries to disk, garbage collecting the remainder.

func NewDatabase 

func NewDatabase(diskDB database.DBManager) *Database

NewDatabase creates a new trie database to store ephemeral trie content before its written out to disk or garbage collected.

func NewDatabaseWithExistingCache added in v1.5.0 

func NewDatabaseWithExistingCache(diskDB database.DBManager, cache TrieNodeCache) *Database

NewDatabaseWithExistingCache creates a new trie database to store ephemeral trie content before its written out to disk or garbage collected. It also acts as a read cache for nodes loaded from disk.

func NewDatabaseWithNewCache added in v1.5.0 

func NewDatabaseWithNewCache(diskDB database.DBManager, cacheConfig *TrieNodeCacheConfig) *Database

NewDatabaseWithNewCache creates a new trie database to store ephemeral trie content before its written out to disk or garbage collected. It also acts as a read cache for nodes loaded from disk.

func (*Database) CanSaveTrieNodeCacheToFile added in v1.6.0 

func (db *Database) CanSaveTrieNodeCacheToFile() error

func (*Database) Cap 

func (db *Database) Cap(limit common.StorageSize) error

Cap iteratively flushes old but still referenced trie nodes until the total memory usage goes below the given threshold.

func (*Database) CollectChildrenStats added in v1.6.0 

func (db *Database) CollectChildrenStats(node common.Hash, depth int, resultCh chan<- NodeInfo)

CollectChildrenStats collects the depth of the trie recursively

func (*Database) Commit 

func (db *Database) Commit(node common.Hash, report bool, blockNum uint64) error

Commit iterates over all the children of a particular node, writes them out to disk, forcefully tearing down all references in both directions.

As a side effect, all pre-images accumulated up to this point are also written.

func (*Database) Dereference 

func (db *Database) Dereference(root common.Hash)

Dereference removes an existing reference from a root node.

func (*Database) DiskDB 

func (db *Database) DiskDB() database.DBManager

DiskDB retrieves the persistent database backing the trie database.

func (*Database) DoesExistCachedNode added in v1.4.0 

func (db *Database) DoesExistCachedNode(hash common.Hash) bool

DoesExistCachedNode returns if the node exists on cached trie node in memory.

func (*Database) DoesExistNodeInPersistent added in v1.5.0 

func (db *Database) DoesExistNodeInPersistent(hash common.Hash) bool

DoesExistNodeInPersistent returns if the node exists on the persistent database or its cache.

func (*Database) GetTrieNodeCacheConfig added in v1.5.3 

func (db *Database) GetTrieNodeCacheConfig() *TrieNodeCacheConfig

GetTrieNodeCacheConfig returns the configuration of TrieNodeCache.

func (*Database) GetTrieNodeLocalCacheByteLimit added in v1.5.3 

func (db *Database) GetTrieNodeLocalCacheByteLimit() uint64

GetTrieNodeLocalCacheByteLimit returns the byte size of trie node cache.

func (*Database) Node 

func (db *Database) Node(hash common.Hash) ([]byte, error)

Node retrieves an encoded cached trie node from memory. If it cannot be found cached, the method queries the persistent database for the content.

func (*Database) NodeChildren added in v1.5.0 

func (db *Database) NodeChildren(hash common.Hash) ([]common.Hash, error)

NodeChildren retrieves the children of the given hash trie

func (*Database) NodeFromOld added in v1.5.0 

func (db *Database) NodeFromOld(hash common.Hash) ([]byte, error)

NodeFromOld retrieves an encoded cached trie node from memory. If it cannot be found cached, the method queries the old persistent database for the content.

func (*Database) Nodes 

func (db *Database) Nodes() []common.Hash

Nodes retrieves the hashes of all the nodes cached within the memory database. This method is extremely expensive and should only be used to validate internal states in test code.

func (*Database) RLockGCCachedNode added in v1.4.0 

func (db *Database) RLockGCCachedNode()

RLockGCCachedNode locks the GC lock of CachedNode.

func (*Database) RUnlockGCCachedNode added in v1.4.0 

func (db *Database) RUnlockGCCachedNode()

RUnlockGCCachedNode unlocks the GC lock of CachedNode.

func (*Database) Reference 

func (db *Database) Reference(child common.Hash, parent common.Hash)

Reference adds a new reference from a parent node to a child node. This function is used to add reference between internal trie node and external node(e.g. storage trie root), all internal trie nodes are referenced together by database itself.

func (*Database) SaveCachePeriodically added in v1.6.0 

func (db *Database) SaveCachePeriodically(c *TrieNodeCacheConfig, stopCh <-chan struct{})

DumpPeriodically atomically saves fast cache data to the given dir with the specified interval.

func (*Database) SaveTrieNodeCacheToFile added in v1.5.3 

func (db *Database) SaveTrieNodeCacheToFile(filePath string, concurrency int)

SaveTrieNodeCacheToFile saves the current cached trie nodes to file to reuse when the node restarts

func (*Database) Size 

func (db *Database) Size() (common.StorageSize, common.StorageSize, common.StorageSize)

Size returns the current database size of the memory cache in front of the persistent database layer.

func (*Database) TrieNodeCache added in v1.5.0 

func (db *Database) TrieNodeCache() TrieNodeCache

TrieNodeCache retrieves the trieNodeCache of the trie database.

func (*Database) UpdateMetricNodes 

func (db *Database) UpdateMetricNodes()

UpdateMetricNodes updates the size of Database.nodes

type DatabaseReader 

type DatabaseReader interface {
	// Get retrieves the value associated with key from the database.
	Get(key []byte) (value []byte, err error)

	// Has retrieves whether a key is present in the database.
	Has(key []byte) (bool, error)
}

type DeriveShaOrig 

type DeriveShaOrig struct{}

func (DeriveShaOrig) DeriveSha 

func (d DeriveShaOrig) DeriveSha(list types.DerivableList) common.Hash

type FastCache added in v1.5.3 

type FastCache struct {
	// contains filtered or unexported fields
}

func (*FastCache) Close added in v1.5.3 

func (cache *FastCache) Close() error

func (*FastCache) Get added in v1.5.3 

func (cache *FastCache) Get(k []byte) []byte

func (*FastCache) Has added in v1.5.3 

func (cache *FastCache) Has(k []byte) ([]byte, bool)

func (*FastCache) SaveToFile added in v1.5.3 

func (cache *FastCache) SaveToFile(filePath string, concurrency int) error

func (*FastCache) Set added in v1.5.3 

func (cache *FastCache) Set(k, v []byte)

func (*FastCache) UpdateStats added in v1.5.3 

func (cache *FastCache) UpdateStats() interface{}

type HybridCache added in v1.5.3 

type HybridCache struct {
	// contains filtered or unexported fields
}

HybridCache integrates two kinds of caches: local, remote. Local cache uses memory of the local machine and remote cache uses memory of the remote machine. When it sets data to both caches, only remote cache is set asynchronously

func (*HybridCache) Close added in v1.5.3 

func (cache *HybridCache) Close() error

func (*HybridCache) Get added in v1.5.3 

func (cache *HybridCache) Get(k []byte) []byte

func (*HybridCache) Has added in v1.5.3 

func (cache *HybridCache) Has(k []byte) ([]byte, bool)

func (*HybridCache) Local added in v1.7.0 

func (cache *HybridCache) Local() TrieNodeCache

func (*HybridCache) PublishBlock added in v1.5.3 

func (cache *HybridCache) PublishBlock(msg string) error

func (*HybridCache) Remote added in v1.7.0 

func (cache *HybridCache) Remote() *RedisCache

func (*HybridCache) SaveToFile added in v1.5.3 

func (cache *HybridCache) SaveToFile(filePath string, concurrency int) error

func (*HybridCache) Set added in v1.5.3 

func (cache *HybridCache) Set(k, v []byte)

Set writes data to local cache synchronously and to remote cache asynchronously.

func (*HybridCache) SubscribeBlockCh added in v1.5.3 

func (cache *HybridCache) SubscribeBlockCh() <-chan *redis.Message

func (*HybridCache) UnsubscribeBlock added in v1.5.3 

func (cache *HybridCache) UnsubscribeBlock() error

func (*HybridCache) UpdateStats added in v1.5.3 

func (cache *HybridCache) UpdateStats() interface{}

type Iterator 

type Iterator struct {
	Key   []byte // Current data key on which the iterator is positioned on
	Value []byte // Current data value on which the iterator is positioned on
	Err   error
	// contains filtered or unexported fields
}

Iterator is a key-value trie iterator that traverses a Trie.

func NewIterator 

func NewIterator(it NodeIterator) *Iterator

NewIterator creates a new key-value iterator from a node iterator

func (*Iterator) Next 

func (it *Iterator) Next() bool

Next moves the iterator forward one key-value entry.

type KeccakState added in v1.9.0 

type KeccakState interface {
	hash.Hash
	Read([]byte) (int, error)
}

KeccakState wraps sha3.state. In addition to the usual hash methods, it also supports Read to get a variable amount of data from the hash state. Read is faster than Sum because it doesn't copy the internal state, but also modifies the internal state.

type LeafCallback 

type LeafCallback func(paths [][]byte, hexpath []byte, leaf []byte, parent common.Hash, parentDepth int) error

LeafCallback is a callback type invoked when a trie operation reaches a leaf node.

The paths is a path tuple identifying a particular trie node either in a single trie (account) or a layered trie (account -> storage). Each path in the tuple is in the raw format(32 bytes).

The hexpath is a composite hexary path identifying the trie node. All the key bytes are converted to the hexary nibbles and composited with the parent path if the trie node is in a layered trie.

It's used by state sync and commit to allow handling external references between account and storage tries. And also it's used in the state healing for extracting the raw states(leaf nodes) with corresponding paths.

type MissingNodeError 

type MissingNodeError struct {
	NodeHash common.Hash // hash of the missing node
	Path     []byte      // hex-encoded path to the missing node
}

MissingNodeError is returned by the trie functions (TryGet, TryUpdate, TryDelete) in the case where a trie node is not present in the local database. It contains information necessary for retrieving the missing node.

func (*MissingNodeError) Error 

func (err *MissingNodeError) Error() string

type NodeInfo added in v1.6.0 

type NodeInfo struct {
	Depth    int  // 0 if not a leaf node
	Finished bool // true if the uppermost call is finished
}

NodeInfo is a struct used for collecting trie statistics

type NodeIterator 

type NodeIterator interface {
	// Next moves the iterator to the next node. If the parameter is false, any child
	// nodes will be skipped.
	Next(bool) bool
	// Error returns the error status of the iterator.
	Error() error

	// Hash returns the hash of the current node.
	Hash() common.Hash
	// Parent returns the hash of the parent of the current node. The hash may be the one
	// grandparent if the immediate parent is an internal node with no hash.
	Parent() common.Hash
	// Path returns the hex-encoded path to the current node.
	// Callers must not retain references to the return value after calling Next.
	// For leaf nodes, the last element of the path is the 'terminator symbol' 0x10.
	Path() []byte

	// Leaf returns true iff the current node is a leaf node.
	// LeafBlob, LeafKey return the contents and key of the leaf node. These
	// method panic if the iterator is not positioned at a leaf.
	// Callers must not retain references to their return value after calling Next
	Leaf() bool

	// LeafKey returns the key of the leaf. The method panics if the iterator is not
	// positioned at a leaf. Callers must not retain references to the value after
	// calling Next.
	LeafKey() []byte

	// LeafBlob returns the content of the leaf. The method panics if the iterator
	// is not positioned at a leaf. Callers must not retain references to the value
	// after calling Next.
	LeafBlob() []byte

	// LeafProof returns the Merkle proof of the leaf. The method panics if the
	// iterator is not positioned at a leaf. Callers must not retain references
	// to the value after calling Next.
	LeafProof() [][]byte

	// AddResolver sets an intermediate database to use for looking up trie nodes
	// before reaching into the real persistent layer.
	//
	// This is not required for normal operation, rather is an optimization for
	// cases where trie nodes can be recovered from some external mechanism without
	// reading from disk. In those cases, this resolver allows short circuiting
	// accesses and returning them from memory.
	//
	// Before adding a similar mechanism to any other place in Geth, consider
	// making trie.Database an interface and wrapping at that level. It's a huge
	// refactor, but it could be worth it if another occurrence arises.
	AddResolver(database.DBManager)
}

NodeIterator is an iterator to traverse the trie pre-order.

func NewDifferenceIterator 

func NewDifferenceIterator(a, b NodeIterator) (NodeIterator, *int)

NewDifferenceIterator constructs a NodeIterator that iterates over elements in b that are not in a. Returns the iterator, and a pointer to an integer recording the number of nodes seen.

func NewUnionIterator 

func NewUnionIterator(iters []NodeIterator) (NodeIterator, *int)

NewUnionIterator constructs a NodeIterator that iterates over elements in the union of the provided NodeIterators. Returns the iterator, and a pointer to an integer recording the number of nodes visited.

type ProofDBReader added in v1.9.0 

type ProofDBReader interface {
	ReadCachedTrieNode(hash common.Hash) ([]byte, error)
}

type ProofDBWriter added in v1.9.0 

type ProofDBWriter interface {
	WriteMerkleProof(key, value []byte)
}

type RedisCache added in v1.5.3 

type RedisCache struct {
	// contains filtered or unexported fields
}

func (*RedisCache) Close added in v1.5.3 

func (cache *RedisCache) Close() error

func (*RedisCache) Get added in v1.5.3 

func (cache *RedisCache) Get(k []byte) []byte

func (*RedisCache) Has added in v1.5.3 

func (cache *RedisCache) Has(k []byte) ([]byte, bool)

func (*RedisCache) PublishBlock added in v1.5.3 

func (cache *RedisCache) PublishBlock(msg string) error

func (*RedisCache) SaveToFile added in v1.5.3 

func (cache *RedisCache) SaveToFile(filePath string, concurrency int) error

func (*RedisCache) Set added in v1.5.3 

func (cache *RedisCache) Set(k, v []byte)

Set writes data synchronously. To write data asynchronously, use SetAsync instead.

func (*RedisCache) SetAsync added in v1.6.1 

func (cache *RedisCache) SetAsync(k, v []byte)

SetAsync writes data asynchronously. Not all data is written if a setItemCh is full. To write data synchronously, use Set instead.

func (*RedisCache) SubscribeBlockCh added in v1.5.3 

func (cache *RedisCache) SubscribeBlockCh() <-chan *redis.Message

func (*RedisCache) UnsubscribeBlock added in v1.5.3 

func (cache *RedisCache) UnsubscribeBlock() error

func (*RedisCache) UpdateStats added in v1.5.3 

func (cache *RedisCache) UpdateStats() interface{}

type SecureTrie 

type SecureTrie struct {
	// contains filtered or unexported fields
}

SecureTrie wraps a trie with key hashing. In a secure trie, all access operations hash the key using keccak256. This prevents calling code from creating long chains of nodes that increase the access time.

Contrary to a regular trie, a SecureTrie can only be created with NewTrie and must have an attached database. The database also stores the preimage of each key.

SecureTrie is not safe for concurrent use.

func NewSecureTrie 

func NewSecureTrie(root common.Hash, db *Database) (*SecureTrie, error)

NewSecureTrie creates a trie with an existing root node from a backing database and optional intermediate in-memory node pool.

If root is the zero hash or the sha3 hash of an empty string, the trie is initially empty. Otherwise, NewTrie will panic if db is nil and returns MissingNodeError if the root node cannot be found.

Accessing the trie loads nodes from the database or node pool on demand. Loaded nodes are kept around until their 'cache generation' expires. A new cache generation is created by each call to Commit. cachelimit sets the number of past cache generations to keep.

func NewSecureTrieForPrefetching added in v1.6.1 

func NewSecureTrieForPrefetching(root common.Hash, db *Database) (*SecureTrie, error)

func (*SecureTrie) Commit 

func (t *SecureTrie) Commit(onleaf LeafCallback) (root common.Hash, err error)

Commit writes all nodes and the secure hash pre-images to the trie's database. Nodes are stored with their sha3 hash as the key.

Committing flushes nodes from memory. Subsequent Get calls will load nodes from the database.

func (*SecureTrie) Copy 

func (t *SecureTrie) Copy() *SecureTrie

func (*SecureTrie) Delete 

func (t *SecureTrie) Delete(key []byte)

Delete removes any existing value for key from the trie.

func (*SecureTrie) Get 

func (t *SecureTrie) Get(key []byte) []byte

Get returns the value for key stored in the trie. The value bytes must not be modified by the caller.

func (*SecureTrie) GetKey 

func (t *SecureTrie) GetKey(shaKey []byte) []byte

GetKey returns the sha3 preimage of a hashed key that was previously used to store a value.

func (*SecureTrie) Hash 

func (t *SecureTrie) Hash() common.Hash

func (*SecureTrie) NodeIterator 

func (t *SecureTrie) NodeIterator(start []byte) NodeIterator

NodeIterator returns an iterator that returns nodes of the underlying trie. Iteration starts at the key after the given start key.

func (*SecureTrie) Prove 

func (t *SecureTrie) Prove(key []byte, fromLevel uint, proofDB database.DBManager) error

NOTE-Klaytn-RemoveLater Below Prove is only used in tests, not in core codes. Prove constructs a merkle proof for key. The result contains all encoded nodes on the path to the value at key. The value itself is also included in the last node and can be retrieved by verifying the proof.

If the trie does not contain a value for key, the returned proof contains all nodes of the longest existing prefix of the key (at least the root node), ending with the node that proves the absence of the key.

func (*SecureTrie) TryDelete 

func (t *SecureTrie) TryDelete(key []byte) error

TryDelete removes any existing value for key from the trie. If a node was not found in the database, a MissingNodeError is returned.

func (*SecureTrie) TryGet 

func (t *SecureTrie) TryGet(key []byte) ([]byte, error)

TryGet returns the value for key stored in the trie. The value bytes must not be modified by the caller. If a node was not found in the database, a MissingNodeError is returned.

func (*SecureTrie) TryGetNode added in v1.9.0 

func (t *SecureTrie) TryGetNode(path []byte) ([]byte, int, error)

TryGetNode attempts to retrieve a trie node by compact-encoded path. It is not possible to use keybyte-encoding as the path might contain odd nibbles.

func (*SecureTrie) TryUpdate 

func (t *SecureTrie) TryUpdate(key, value []byte) error

TryUpdate associates key with value in the trie. Subsequent calls to Get will return value. If value has length zero, any existing value is deleted from the trie and calls to Get will return nil.

The value bytes must not be modified by the caller while they are stored in the trie.

If a node was not found in the database, a MissingNodeError is returned.

func (*SecureTrie) TryUpdateWithKeys 

func (t *SecureTrie) TryUpdateWithKeys(key, hashKey, hexKey, value []byte) error

TryUpdateWithKeys does basically same thing that TryUpdate does. Only difference is that it uses pre-encoded hashKey and hexKey.

func (*SecureTrie) Update 

func (t *SecureTrie) Update(key, value []byte)

Update associates key with value in the trie. Subsequent calls to Get will return value. If value has length zero, any existing value is deleted from the trie and calls to Get will return nil.

The value bytes must not be modified by the caller while they are stored in the trie.

type StateTrieReadDB added in v1.5.0 

type StateTrieReadDB interface {
	ReadStateTrieNode(key []byte) ([]byte, error)
	HasStateTrieNode(key []byte) (bool, error)
	HasCodeWithPrefix(hash common.Hash) bool
}

type SyncBloom added in v1.5.0 

type SyncBloom struct {
	// contains filtered or unexported fields
}

SyncBloom is a bloom filter used during fast sync to quickly decide if a trie node or contract code already exists on disk or not. It self populates from the provided disk database on creation in a background thread and will only start returning live results once that's finished.

func NewSyncBloom added in v1.5.0 

func NewSyncBloom(memory uint64, database database.Iteratee) *SyncBloom

NewSyncBloom creates a new bloom filter of the given size (in megabytes) and initializes it from the database. The bloom is hard coded to use 3 filters.

func (*SyncBloom) Add added in v1.5.0 

func (b *SyncBloom) Add(hash []byte)

Add inserts a new trie node hash into the bloom filter.

func (*SyncBloom) Close added in v1.5.0 

func (b *SyncBloom) Close() error

Close terminates any background initializer still running and releases all the memory allocated for the bloom.

func (*SyncBloom) Contains added in v1.5.0 

func (b *SyncBloom) Contains(hash []byte) bool

Contains tests if the bloom filter contains the given hash:

  • false: the bloom definitely does not contain hash
  • true: the bloom maybe contains hash

While the bloom is being initialized, any query will return true.

type SyncPath added in v1.9.0 

type SyncPath [][]byte

SyncPath is a path tuple identifying a particular trie node either in a single trie (account) or a layered trie (account -> storage).

Content wise the tuple either has 1 element if it addresses a node in a single trie or 2 elements if it addresses a node in a stacked trie.

To support aiming arbitrary trie nodes, the path needs to support odd nibble lengths. To avoid transferring expanded hex form over the network, the last part of the tuple (which needs to index into the middle of a trie) is compact encoded. In case of a 2-tuple, the first item is always 32 bytes so that is simple binary encoded.

Examples:

  • Path 0x9 -> {0x19}
  • Path 0x99 -> {0x0099}
  • Path 0x01234567890123456789012345678901012345678901234567890123456789019 -> {0x0123456789012345678901234567890101234567890123456789012345678901, 0x19}
  • Path 0x012345678901234567890123456789010123456789012345678901234567890199 -> {0x0123456789012345678901234567890101234567890123456789012345678901, 0x0099}

type SyncResult 

type SyncResult struct {
	Hash common.Hash // Hash of the originally unknown trie node
	Data []byte      // Data content of the retrieved node
	Err  error
}

SyncResult is a response with requested data along with it's hash.

type Trie 

type Trie struct {
	// contains filtered or unexported fields
}

Trie is a Merkle Patricia Trie. The zero value is an empty trie with no database. Use NewTrie to create a trie that sits on top of a database.

Trie is not safe for concurrent use.

func NewTrie 

func NewTrie(root common.Hash, db *Database) (*Trie, error)

NewTrie creates a trie with an existing root node from db.

If root is the zero hash or the sha3 hash of an empty string, the trie is initially empty and does not require a database. Otherwise, NewTrie will panic if db is nil and returns a MissingNodeError if root does not exist in the database. Accessing the trie loads nodes from db on demand.

func NewTrieForPrefetching added in v1.6.1 

func NewTrieForPrefetching(root common.Hash, db *Database) (*Trie, error)

func (*Trie) Commit 

func (t *Trie) Commit(onleaf LeafCallback) (root common.Hash, err error)

Commit writes all nodes to the trie's memory database, tracking the internal and external (for account tries) references.

func (*Trie) Delete 

func (t *Trie) Delete(key []byte)

Delete removes any existing value for key from the trie.

func (*Trie) Get 

func (t *Trie) Get(key []byte) []byte

Get returns the value for key stored in the trie. The value bytes must not be modified by the caller.

func (*Trie) Hash 

func (t *Trie) Hash() common.Hash

Hash returns the root hash of the trie. It does not write to the database and can be used even if the trie doesn't have one.

func (*Trie) NodeIterator 

func (t *Trie) NodeIterator(start []byte) NodeIterator

NodeIterator returns an iterator that returns nodes of the trie. Iteration starts at the key after the given start key.

func (*Trie) Prove 

func (t *Trie) Prove(key []byte, fromLevel uint, proofDB ProofDBWriter) error

Prove constructs a merkle proof for key. The result contains all encoded nodes on the path to the value at key. The value itself is also included in the last node and can be retrieved by verifying the proof.

If the trie does not contain a value for key, the returned proof contains all nodes of the longest existing prefix of the key (at least the root node), ending with the node that proves the absence of the key.

func (*Trie) TryDelete 

func (t *Trie) TryDelete(key []byte) error

TryDelete removes any existing value for key from the trie. If a node was not found in the database, a MissingNodeError is returned.

func (*Trie) TryGet 

func (t *Trie) TryGet(key []byte) ([]byte, error)

TryGet returns the value for key stored in the trie. The value bytes must not be modified by the caller. If a node was not found in the database, a MissingNodeError is returned.

func (*Trie) TryGetNode added in v1.9.0 

func (t *Trie) TryGetNode(path []byte) ([]byte, int, error)

TryGetNode attempts to retrieve a trie node by compact-encoded path. It is not possible to use keybyte-encoding as the path might contain odd nibbles.

func (*Trie) TryUpdate 

func (t *Trie) TryUpdate(key, value []byte) error

TryUpdate associates key with value in the trie. Subsequent calls to Get will return value. If value has length zero, any existing value is deleted from the trie and calls to Get will return nil.

The value bytes must not be modified by the caller while they are stored in the trie.

If a node was not found in the database, a MissingNodeError is returned.

func (*Trie) TryUpdateWithHexKey 

func (t *Trie) TryUpdateWithHexKey(hexKey, value []byte) error

TryUpdateWithHexKey uses pre-generated hexKey. It is both called from TryUpdate and SecureTrie.TryUpdateWithKeys.

func (*Trie) Update 

func (t *Trie) Update(key, value []byte)

Update associates key with value in the trie. Subsequent calls to Get will return value. If value has length zero, any existing value is deleted from the trie and calls to Get will return nil.

The value bytes must not be modified by the caller while they are stored in the trie.

type TrieNodeCache added in v1.5.3 

type TrieNodeCache interface {
	Set(k, v []byte)
	Get(k []byte) []byte
	Has(k []byte) ([]byte, bool)
	UpdateStats() interface{}
	SaveToFile(filePath string, concurrency int) error
	Close() error
}

TrieNodeCache interface the cache of stateDB

func NewTrieNodeCache added in v1.5.3 

func NewTrieNodeCache(config *TrieNodeCacheConfig) (TrieNodeCache, error)

NewTrieNodeCache creates one type of any supported trie node caches. NOTE: It returns (nil, nil) when the cache type is CacheTypeLocal and its size is set to zero.

type TrieNodeCacheConfig added in v1.5.3 

type TrieNodeCacheConfig struct {
	CacheType                 TrieNodeCacheType
	NumFetcherPrefetchWorker  int           // Number of workers used to prefetch a block when fetcher works
	UseSnapshotForPrefetch    bool          // Enable snapshot functionality while prefetching
	LocalCacheSizeMiB         int           // Memory allowance (MiB) to use for caching trie nodes in fast cache
	FastCacheFileDir          string        // Directory where the persistent fastcache data is stored
	FastCacheSavePeriod       time.Duration // Period of saving in memory trie cache to file if fastcache is used
	RedisEndpoints            []string      // Endpoints of redis cache
	RedisClusterEnable        bool          // Enable cluster-enabled mode of redis cache
	RedisPublishBlockEnable   bool          // Enable publishing every inserted block to the redis server
	RedisSubscribeBlockEnable bool          // Enable subscribing blocks from the redis server
}

TrieNodeCacheConfig contains configuration values of all TrieNodeCache.

func GetEmptyTrieNodeCacheConfig added in v1.5.3 

func GetEmptyTrieNodeCacheConfig() *TrieNodeCacheConfig

func (*TrieNodeCacheConfig) DumpPeriodically added in v1.6.0 

func (c *TrieNodeCacheConfig) DumpPeriodically() bool

type TrieNodeCacheType added in v1.5.3 

type TrieNodeCacheType string

func (TrieNodeCacheType) ToValid added in v1.5.3 

func (cacheType TrieNodeCacheType) ToValid() TrieNodeCacheType

type TrieSync 

type TrieSync struct {
	// contains filtered or unexported fields
}

TrieSync is the main state trie synchronisation scheduler, which provides yet unknown trie hashes to retrieve, accepts node data associated with said hashes and reconstructs the trie step by step until all is done.

func NewTrieSync 

func NewTrieSync(root common.Hash, database StateTrieReadDB, callback LeafCallback, bloom *SyncBloom, lruCache *lru.Cache) *TrieSync

NewTrieSync creates a new trie data download scheduler. If both bloom and cache are set, only cache is used.

func (*TrieSync) AddCodeEntry added in v1.9.0 

func (s *TrieSync) AddCodeEntry(hash common.Hash, path []byte, depth int, parent common.Hash)

AddCodeEntry schedules the direct retrieval of a contract code that should not be interpreted as a trie node, but rather accepted and stored into the database as is.

func (*TrieSync) AddSubTrie 

func (s *TrieSync) AddSubTrie(root common.Hash, path []byte, depth int, parent common.Hash, callback LeafCallback)

AddSubTrie registers a new trie to the sync code, rooted at the designated parent.

func (*TrieSync) CalcProgressPercentage added in v1.5.0 

func (s *TrieSync) CalcProgressPercentage() float64

CalcProgressPercentage returns the progress percentage.

func (*TrieSync) Commit 

func (s *TrieSync) Commit(dbw database.Batch) (int, error)

Commit flushes the data stored in the internal membatch out to persistent storage, returning the number of items written and any occurred error.

func (*TrieSync) CommittedByDepth added in v1.5.0 

func (s *TrieSync) CommittedByDepth(depth int) int

CommittedByDepth returns the committed trie count by given depth.

func (*TrieSync) Missing 

func (s *TrieSync) Missing(max int) (nodes []common.Hash, paths []SyncPath, codes []common.Hash)

Missing retrieves the known missing nodes from the trie for retrieval. To aid both klay/6x style fast sync and snap/1x style state sync, the paths of trie nodes are returned too, as well as separate hash list for codes.

func (*TrieSync) Pending 

func (s *TrieSync) Pending() int

Pending returns the number of state entries currently pending for download.

func (*TrieSync) Process 

func (s *TrieSync) Process(result SyncResult) error

Process injects the received data for requested item. Note it can happen that the single response commits two pending requests(e.g. there are two requests one for code and one for node but the hash is same). In this case the second response for the same hash will be treated as "non-requested" item or "already-processed" item but there is no downside.

func (*TrieSync) RetrievedByDepth added in v1.5.0 

func (s *TrieSync) RetrievedByDepth(depth int) int

RetrievedByDepth returns the retrieved trie count by given depth. This number is same as the number of nodes that needs to be committed to complete trie sync.

Source Files

View all Source files

Directories

Path Synopsis
Package mock_statedb is a generated GoMock package.
 Package mock_statedb is a generated GoMock package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.