stagedsync

package
v0.0.0-...-92d349b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 15, 2023 License: GPL-3.0 Imports: 104 Imported by: 0

README

Staged Sync

Staged Sync is a version of Go-Ethereum's Full Sync that was rearchitected for better performance.

It is I/O intensive and even though we have a goal on being able to sync the node on an HDD, we still recommend using fast SSDs.

Staged Sync, as its name suggests, consists of 10 stages that are executed in order, one after another.

How The Sync Works

For each peer Erigon learns what the HEAD blocks is and it executes each stage in order for the missing blocks between the local HEAD block and the peer's head blocks.

The first stage (downloading headers) sets the local HEAD block.

Each stage is executed in order and a stage N does not stop until the local head is reached for it.

That mean, that in the ideal scenario (no network interruptions, the app isn't restarted, etc), for the full initial sync, each stage will be executed exactly once.

After the last stage is finished, the process starts from the beginning, by looking for the new headers to download.

If the app is restarted in between stages, it restarts from the first stage.

If the app is restarted in the middle of the stage execution, it restarts from that stage, giving it the opportunity to complete.

How long do the stages take?

Here is a pie chart showing the proportional time spent on each stage (it was taken from the full sync). It is by all means just an estimation, but it gives an idea.

Reorgs / Unwinds

Sometimes the chain makes a reorg and we need to "undo" some parts of our sync.

This happens backward from the last stage to the first one with one caveat that tx pool is updated after we already unwound the execution so we know the new nonces.

That is the example of stages order to be unwound (unwind happens from right to left).

state.unwindOrder = []*Stage{
		// Unwinding of tx pool (reinjecting transactions into the pool needs to happen after unwinding execution)
		stages[0], stages[1], stages[2], stages[9], stages[3], stages[4], stages[5], stages[6], stages[7], stages[8],
	}

Preprocessing with ETL

Some stages use our ETL framework to sort data by keys before inserting it into the database.

That allows to reduce db write amplification significantly.

So, when we are generating indexes or hashed state, we do a multi-step process.

  1. We write the processed data into a couple of temp files in your data directory;
  2. We then use a heap to insert data from the temp files into the database, in the order that minimizes db write amplification.

This optimization sometimes leads to dramatic (orders of magnitude) write speed improvements.

What happens after the Merge?

In the Proof-of-Stake world staged sync becomes somewhat more complicated, as the following diagram shows.

Stages (for the up to date list see stages.go and stagebuilder.go):

Each stage consists of 2 functions ExecFunc that progesses the stage forward and UnwindFunc that unwinds the stage backwards.

Most of the stages can work offline though it isn't implemented in the current version.

We can add/remove stages, so exact stage numbers may change - but order and names stay the same.

Stage 1: Snapshots

Download Snapshots

Stage 2: Download Headers Stage

During this stage we download all the headers between the local HEAD and our peer's head.

This stage is CPU intensive and can benefit from a multicore processor due to verifying PoW of the headers.

Most of the unwinds are initiated on this stage due to the chain reorgs.

This stage promotes local HEAD pointer.

Stage 3: Cumulative Index

Calculate how much gas has been used up to each block.

Stage 4: Block Hashes

Creates an index of blockHash -> blockNumber extracted from the headers for faster lookups and making the sync friendlier for HDDs.

Stage 5: Download Block Bodies Stage

At that stage, we download bodies for block headers that we already downloaded.

That is the most intensive stage for the network connection, the vast majority of data is downloaded here.

Stage 6: Recover Senders Stage

This stage recovers and stores senders for each transaction in each downloaded block.

This is also a CPU intensive stage and also benefits from multi-core CPUs.

This stage doesn't use any network connection.

Stage 7: Execute Blocks Stage

During this stage, we execute block-by-block everything that we downloaded before.

One important point there, that we don't check root hashes during this execution, we don't even build a merkle trie here.

This stage is single threaded.

This stage doesn't use internet connection.

This stage is disk intensive.

This stage can spawn unwinds if the block execution fails.

Stage 8: Transpile marked VM contracts to TEVM

Translation each marked for translation contract (from EVM to TEVM)

Stage 9: VerkleTrie

[TODO]

Stage 10: Compute State Root Stage

This stage build the Merkle trie and checks the root hash for the current state.

It also builds Intermediate Hashes along the way and stores them into the database.

If there were no intermediate hashes stored before (that could happend during the first initial sync), it builds the full Merkle Trie and its root hash.

If there are intermediate hashes in the database, it uses the block history to figure out which ones are outdated and which ones are still up to date. Then it builds a partial Merkle trie using the up-to-date hashes and only rebuilding the outdated ones.

If the root hash doesn't match, it initiates an unwind one block backwards.

This stage doesn't use a network connection.

Stage 11: Generate Hashed State Stage

Erigon during execution uses Plain state storage.

Plain State: Instead of the normal (we call it "Hashed State") where accounts and storage items are addressed as keccak256(address), in the plain state them are addressed by the address itself.

Though, to make sure that some APIs work and keep the compatibility with the other clients, we generate Hashed state as well.

If the hashed state is not empty, then we are looking at the History ChangeSets and update only the items that were changed.

This stage doesn't use a network connection.

Stages 12, 13, 14, 15 and 16: Generate Indexes

There are 5 indexes that are generated during sync.

They might be disabled because they aren't used for all the APIs.

These stages do not use a network connection.

Account History Index

This index stores the mapping from the account address to the list of blocks where this account was changed in some way.

Storage History Index

This index stores the mapping from the storage item address to the list of blocks where this storage item was changed in some way.

Log Index

This index sets up a link from the [TODO] to [TODO].

Call traces index

[TODO]

Tx Lookup Index

This index sets up a link from the transaction hash to the block number.

Stage 16: Transaction Pool Stage

During this stage we start the transaction pool or update its state. For instance, we remove the transactions from the blocks we have downloaded from the pool.

On unwinds, we add the transactions from the blocks we unwind, back to the pool.

This stage doesn't use a network connection.

Stage 17: Finish

This stage sets the current block number that is then used by RPC calls, such as eth_blockNumber.

Documentation

Index

Constants

View Source 
const (
	GLOBAL_EXIT_ROOT_STORAGE_POS        = 0
	ADDRESS_GLOBAL_EXIT_ROOT_MANAGER_L2 = "0xa40D5f56745a118D0906a34E69aeC8C0Db1cB8fA"
)
View Source 
const ShortPoSReorgThresholdBlocks = 10

The number of blocks we should be able to re-org sub-second on commodity hardware. See https://hackmd.io/TdJtNs0dS56q-In8h-ShSg

Variables

View Source 
var DefaultForwardOrder = sync_stages.UnwindOrder{
	sync_stages.Snapshots,
	sync_stages.Headers,
	sync_stages.BlockHashes,
	sync_stages.Bodies,

	sync_stages.Senders,
	sync_stages.Execution,
	sync_stages.Translation,
	sync_stages.HashState,
	sync_stages.IntermediateHashes,
	sync_stages.CallTraces,
	sync_stages.AccountHistoryIndex,
	sync_stages.StorageHistoryIndex,
	sync_stages.LogIndex,
	sync_stages.TxLookup,
	sync_stages.Finish,
}
View Source 
var DefaultPruneOrder = sync_stages.PruneOrder{}
View Source 
var DefaultUnwindOrder = sync_stages.UnwindOrder{}
View Source 
var ExecRepeats = metrics.NewCounter(`exec_repeats`) //nolint
View Source 
var ExecStepsInDB = metrics.NewCounter(`exec_steps_in_db`) //nolint
View Source 
var ExecTriggers = metrics.NewCounter(`exec_triggers`) //nolint
View Source 
var MiningPruneOrder = sync_stages.PruneOrder{} // nothing to unwind in mining - because mining does not commit db changes
View Source 
var MiningUnwindOrder = sync_stages.UnwindOrder{} // nothing to unwind in mining - because mining does not commit db changes
View Source 
var StateUnwindOrder = sync_stages.UnwindOrder{}
View Source 
var ZkUnwindOrder = sync_stages.UnwindOrder{
	sync_stages.L1Syncer,
	sync_stages.Batches,
	sync_stages.BlockHashes,

	sync_stages.IntermediateHashes,
	sync_stages.Execution,
	sync_stages.HashState,
	sync_stages.Senders,
	sync_stages.Translation,
	sync_stages.CallTraces,
	sync_stages.AccountHistoryIndex,
	sync_stages.StorageHistoryIndex,
	sync_stages.LogIndex,
	sync_stages.TxLookup,
	sync_stages.Finish,
}

Functions

func BodiesForward 

func BodiesForward(
	s *sync_stages.StageState,
	u sync_stages.Unwinder,
	ctx context.Context,
	tx kv.RwTx,
	cfg BodiesCfg,
	test bool,
	firstCycle bool,
	quiet bool,
) error

BodiesForward progresses Bodies stage in the forward direction

func DefaultStages 

func DefaultStages(ctx context.Context, snapshots SnapshotsCfg, headers HeadersCfg, cumulativeIndex CumulativeIndexCfg, blockHashCfg BlockHashesCfg, bodies BodiesCfg, senders SendersCfg, exec ExecuteBlockCfg, hashState HashStateCfg, trieCfg TrieCfg, history HistoryCfg, logIndex LogIndexCfg, callTraces CallTracesCfg, txLookup TxLookupCfg, finish FinishCfg, test bool) []*sync_stages.Stage

func DefaultZkStages 

func DefaultZkStages(
	ctx context.Context,
	snapshots SnapshotsCfg,
	l1SyncerCfg zkStages.L1SyncerCfg,
	batchesCfg zkStages.BatchesCfg,
	cumulativeIndex CumulativeIndexCfg,
	blockHashCfg BlockHashesCfg,
	senders SendersCfg,
	exec ExecuteBlockCfg,
	hashState HashStateCfg,
	zkInterHashesCfg zkStages.ZkInterHashesCfg,
	history HistoryCfg,
	logIndex LogIndexCfg,
	callTraces CallTracesCfg,
	txLookup TxLookupCfg,
	finish FinishCfg,
	test bool,
) []*sync_stages.Stage

func DoUnwindCallTraces 

func DoUnwindCallTraces(logPrefix string, db kv.RwTx, from, to uint64, ctx context.Context, tmpdir string) error

func DownloadAndIndexSnapshotsIfNeed 

func DownloadAndIndexSnapshotsIfNeed(s *sync_stages.StageState, ctx context.Context, tx kv.RwTx, cfg SnapshotsCfg, initialCycle bool) error

func ExecBlockV3 

func ExecBlockV3(s *sync_stages.StageState, u sync_stages.Unwinder, tx kv.RwTx, toBlock uint64, ctx context.Context, cfg ExecuteBlockCfg, initialCycle bool) (err error)

func ExecV3 

func ExecV3(ctx context.Context,
	execStage *sync_stages.StageState, u sync_stages.Unwinder, workerCount int, cfg ExecuteBlockCfg, applyTx kv.RwTx,
	parallel bool, logPrefix string,
	maxBlockNum uint64,
) error

func FillDBFromSnapshots 

func FillDBFromSnapshots(logPrefix string, ctx context.Context, tx kv.RwTx, dirs datadir.Dirs, sn *snapshotsync.RoSnapshots, blockReader services.FullBlockReader, chainConfig chain.Config, engine consensus.Engine, agg *state.AggregatorV3) error

func FinishForward 

func FinishForward(s *sync_stages.StageState, tx kv.RwTx, cfg FinishCfg, initialCycle bool) error

func HeadersPOS 

func HeadersPOS(
	s *sync_stages.StageState,
	u sync_stages.Unwinder,
	ctx context.Context,
	tx kv.RwTx,
	cfg HeadersCfg,
	initialCycle bool,
	test bool,
	useExternalTx bool,
	preProgress uint64,
) error

HeadersPOS processes Proof-of-Stake requests (newPayload, forkchoiceUpdated). It also saves PoS headers downloaded by (*HeaderDownload)StartPoSDownloader into the DB.

func HeadersPOW 

func HeadersPOW(
	s *sync_stages.StageState,
	u sync_stages.Unwinder,
	ctx context.Context,
	tx kv.RwTx,
	cfg HeadersCfg,
	initialCycle bool,
	test bool,
	useExternalTx bool,
) error

HeadersPOW progresses Headers stage for Proof-of-Work headers

func HeadersPrune 

func HeadersPrune(p *sync_stages.PruneState, tx kv.RwTx, cfg HeadersCfg, ctx context.Context) (err error)

func HeadersUnwind 

func HeadersUnwind(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, cfg HeadersCfg, test bool) (err error)

func MiningStages 

func MiningStages(
	ctx context.Context,
	createBlockCfg MiningCreateBlockCfg,
	execCfg MiningExecCfg,
	hashStateCfg HashStateCfg,
	trieCfg TrieCfg,
	finish MiningFinishCfg,
) []*sync_stages.Stage

func NotifyNewHeaders 

func NotifyNewHeaders(ctx context.Context, finishStageBeforeSync uint64, finishStageAfterSync uint64, unwindTo *uint64, notifier ChainEventNotifier, tx kv.Tx) error

func NotifyPendingLogs 

func NotifyPendingLogs(logPrefix string, notifier ChainEventNotifier, logs types.Logs)

func PromoteHashedStateCleanly 

func PromoteHashedStateCleanly(logPrefix string, tx kv.RwTx, cfg HashStateCfg, ctx context.Context) error

func PruneAccountHistoryIndex 

func PruneAccountHistoryIndex(s *sync_stages.PruneState, tx kv.RwTx, cfg HistoryCfg, ctx context.Context) (err error)

func PruneBlockHashStage 

func PruneBlockHashStage(p *sync_stages.PruneState, tx kv.RwTx, cfg BlockHashesCfg, ctx context.Context) (err error)

func PruneBodiesStage 

func PruneBodiesStage(s *sync_stages.PruneState, tx kv.RwTx, cfg BodiesCfg, ctx context.Context) (err error)

func PruneCallTraces 

func PruneCallTraces(s *sync_stages.PruneState, tx kv.RwTx, cfg CallTracesCfg, ctx context.Context) (err error)

func PruneCumulativeIndexStage 

func PruneCumulativeIndexStage(p *sync_stages.PruneState, tx kv.RwTx, ctx context.Context) (err error)

func PruneExecutionStage 

func PruneExecutionStage(s *sync_stages.PruneState, tx kv.RwTx, cfg ExecuteBlockCfg, ctx context.Context, initialCycle bool) (err error)

func PruneFinish 

func PruneFinish(u *sync_stages.PruneState, tx kv.RwTx, cfg FinishCfg, ctx context.Context) (err error)

func PruneHashStateStage 

func PruneHashStateStage(s *sync_stages.PruneState, tx kv.RwTx, cfg HashStateCfg, ctx context.Context) (err error)

func PruneIntermediateHashesStage 

func PruneIntermediateHashesStage(s *sync_stages.PruneState, tx kv.RwTx, cfg TrieCfg, ctx context.Context) (err error)

func PruneLogIndex 

func PruneLogIndex(s *sync_stages.PruneState, tx kv.RwTx, cfg LogIndexCfg, ctx context.Context) (err error)

func PruneSendersStage 

func PruneSendersStage(s *sync_stages.PruneState, tx kv.RwTx, cfg SendersCfg, ctx context.Context) (err error)

func PruneStorageHistoryIndex 

func PruneStorageHistoryIndex(s *sync_stages.PruneState, tx kv.RwTx, cfg HistoryCfg, ctx context.Context) (err error)

func PruneTxLookup 

func PruneTxLookup(s *sync_stages.PruneState, tx kv.RwTx, cfg TxLookupCfg, ctx context.Context, initialCycle bool) (err error)

func PruneVerkleTries 

func PruneVerkleTries(s *sync_stages.PruneState, tx kv.RwTx, cfg TrieCfg, ctx context.Context) (err error)

func ReadLogs 

func ReadLogs(tx kv.Tx, from uint64, isUnwind bool) ([]*remote.SubscribeLogsReply, error)

func ReconstituteState 

func ReconstituteState(ctx context.Context, s *sync_stages.StageState, dirs datadir.Dirs, workerCount int, batchSize datasize.ByteSize, chainDb kv.RwDB,
	blockReader services.FullBlockReader,
	logger log.Logger, agg *state2.AggregatorV3, engine consensus.Engine,
	chainConfig *chain.Config, genesis *types.Genesis) (err error)

func RegenerateIntermediateHashes 

func RegenerateIntermediateHashes(logPrefix string, db kv.RwTx, cfg TrieCfg, expectedRootHash libcommon.Hash, ctx context.Context) (libcommon.Hash, error)

func SnapshotsPrune 

func SnapshotsPrune(s *sync_stages.PruneState, cfg SnapshotsCfg, ctx context.Context, tx kv.RwTx) (err error)
====== PRUNING ======

snapshots pruning sections works more as a retiring of blocks retiring blocks means moving block data from db into snapshots

func SpawnAccountHistoryIndex 

func SpawnAccountHistoryIndex(s *sync_stages.StageState, tx kv.RwTx, cfg HistoryCfg, ctx context.Context) error

func SpawnBlockHashStage 

func SpawnBlockHashStage(s *sync_stages.StageState, tx kv.RwTx, cfg BlockHashesCfg, ctx context.Context) (err error)

func SpawnCallTraces 

func SpawnCallTraces(s *sync_stages.StageState, tx kv.RwTx, cfg CallTracesCfg, ctx context.Context) error

func SpawnExecuteBlocksStage 

func SpawnExecuteBlocksStage(s *sync_stages.StageState, u sync_stages.Unwinder, tx kv.RwTx, toBlock uint64, ctx context.Context, cfg ExecuteBlockCfg, initialCycle bool, quiet bool) (err error)

func SpawnHashStateStage 

func SpawnHashStateStage(s *sync_stages.StageState, tx kv.RwTx, cfg HashStateCfg, ctx context.Context, quiet bool) error

func SpawnIntermediateHashesStage 

func SpawnIntermediateHashesStage(s *sync_stages.StageState, u sync_stages.Unwinder, tx kv.RwTx, cfg TrieCfg, ctx context.Context, quiet bool) (libcommon.Hash, error)

func SpawnLogIndex 

func SpawnLogIndex(s *sync_stages.StageState, tx kv.RwTx, cfg LogIndexCfg, ctx context.Context, prematureEndBlock uint64) error

func SpawnMiningCreateBlockStage 

func SpawnMiningCreateBlockStage(s *sync_stages.StageState, tx kv.RwTx, cfg MiningCreateBlockCfg, quit <-chan struct{}) (err error)

SpawnMiningCreateBlockStage TODO: - resubmitAdjustCh - variable is not implemented

func SpawnMiningExecStage 

func SpawnMiningExecStage(s *sync_stages.StageState, tx kv.RwTx, cfg MiningExecCfg, quit <-chan struct{}) error

SpawnMiningExecStage TODO: - resubmitAdjustCh - variable is not implemented

func SpawnMiningFinishStage 

func SpawnMiningFinishStage(s *sync_stages.StageState, tx kv.RwTx, cfg MiningFinishCfg, quit <-chan struct{}) error

func SpawnPostExecStage 

func SpawnPostExecStage(s *sync_stages.StageState, tx kv.RwTx, cfg PostExecCfg, ctx context.Context) error

func SpawnRecoverSendersStage 

func SpawnRecoverSendersStage(cfg SendersCfg, s *sync_stages.StageState, u sync_stages.Unwinder, tx kv.RwTx, toBlock uint64, ctx context.Context, quiet bool) error

func SpawnStageCumulativeIndex 

func SpawnStageCumulativeIndex(cfg CumulativeIndexCfg, s *sync_stages.StageState, tx kv.RwTx, ctx context.Context) error

func SpawnStageHeaders 

func SpawnStageHeaders(
	s *sync_stages.StageState,
	u sync_stages.Unwinder,
	ctx context.Context,
	tx kv.RwTx,
	cfg HeadersCfg,
	initialCycle bool,
	test bool,
) error

func SpawnStageSnapshots 

func SpawnStageSnapshots(
	s *sync_stages.StageState,
	ctx context.Context,
	tx kv.RwTx,
	cfg SnapshotsCfg,
	initialCycle bool,
) (err error)

func SpawnStorageHistoryIndex 

func SpawnStorageHistoryIndex(s *sync_stages.StageState, tx kv.RwTx, cfg HistoryCfg, ctx context.Context) error

func SpawnTxLookup 

func SpawnTxLookup(s *sync_stages.StageState, tx kv.RwTx, toBlock uint64, cfg TxLookupCfg, ctx context.Context) (err error)

func SpawnVerkleTrie 

func SpawnVerkleTrie(s *sync_stages.StageState, u sync_stages.Unwinder, tx kv.RwTx, cfg TrieCfg, ctx context.Context) (libcommon.Hash, error)

func StateStages 

func StateStages(ctx context.Context, headers HeadersCfg, bodies BodiesCfg, blockHashCfg BlockHashesCfg, senders SendersCfg, exec ExecuteBlockCfg, hashState HashStateCfg, trieCfg TrieCfg) []*sync_stages.Stage

StateStages are all stages necessary for basic unwind and stage computation, it is primarily used to process side forks and memory execution.

func UnwindAccountHistoryIndex 

func UnwindAccountHistoryIndex(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, cfg HistoryCfg, ctx context.Context) (err error)

func UnwindBlockHashStage 

func UnwindBlockHashStage(u *sync_stages.UnwindState, tx kv.RwTx, cfg BlockHashesCfg, ctx context.Context) (err error)

func UnwindBodiesStage 

func UnwindBodiesStage(u *sync_stages.UnwindState, tx kv.RwTx, cfg BodiesCfg, ctx context.Context) (err error)

func UnwindCallTraces 

func UnwindCallTraces(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, cfg CallTracesCfg, ctx context.Context) (err error)

func UnwindCumulativeIndexStage 

func UnwindCumulativeIndexStage(u *sync_stages.UnwindState, cfg CumulativeIndexCfg, tx kv.RwTx, ctx context.Context) (err error)

func UnwindExecutionStage 

func UnwindExecutionStage(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, ctx context.Context, cfg ExecuteBlockCfg, initialCycle bool) (err error)

func UnwindFinish 

func UnwindFinish(u *sync_stages.UnwindState, tx kv.RwTx, cfg FinishCfg, ctx context.Context) (err error)

func UnwindHashStateStage 

func UnwindHashStateStage(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, cfg HashStateCfg, ctx context.Context) (err error)

func UnwindIntermediateHashesForTrieLoader 

func UnwindIntermediateHashesForTrieLoader(logPrefix string, rl *trie.RetainList, u *sync_stages.UnwindState, s *sync_stages.StageState, db kv.RwTx, cfg TrieCfg, accTrieCollectorFunc trie.HashCollector2, stTrieCollectorFunc trie.StorageHashCollector2, quit <-chan struct{}) (*trie.FlatDBTrieLoader, error)

func UnwindIntermediateHashesStage 

func UnwindIntermediateHashesStage(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, cfg TrieCfg, ctx context.Context) (err error)

func UnwindLogIndex 

func UnwindLogIndex(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, cfg LogIndexCfg, ctx context.Context) (err error)

func UnwindPostExecStage 

func UnwindPostExecStage(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, cfg PostExecCfg, ctx context.Context) (err error)

func UnwindSendersStage 

func UnwindSendersStage(s *sync_stages.UnwindState, tx kv.RwTx, cfg SendersCfg, ctx context.Context) (err error)

func UnwindStorageHistoryIndex 

func UnwindStorageHistoryIndex(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, cfg HistoryCfg, ctx context.Context) (err error)

func UnwindTxLookup 

func UnwindTxLookup(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, cfg TxLookupCfg, ctx context.Context) (err error)

func UnwindVerkleTrie 

func UnwindVerkleTrie(u *sync_stages.UnwindState, s *sync_stages.StageState, tx kv.RwTx, cfg TrieCfg, ctx context.Context) (err error)

func WaitForDownloader 

func WaitForDownloader(s *sync_stages.StageState, ctx context.Context, cfg SnapshotsCfg, tx kv.RwTx) error

WaitForDownloader - wait for Downloader service to download all expected snapshots for MVP we sync with Downloader only once, in future will send new snapshots also

Types

type BlockHashesCfg 

type BlockHashesCfg struct {
	// contains filtered or unexported fields
}

func StageBlockHashesCfg 

func StageBlockHashesCfg(db kv.RwDB, tmpDir string, cc *chain.Config) BlockHashesCfg

type BodiesCfg 

type BodiesCfg struct {
	// contains filtered or unexported fields
}

func StageBodiesCfg 

func StageBodiesCfg(db kv.RwDB, bd *bodydownload.BodyDownload, bodyReqSend func(context.Context, *bodydownload.BodyRequest) ([64]byte, bool), penalise func(context.Context, []headerdownload.PenaltyItem), blockPropagator adapter.BlockPropagator, timeout int, chanConfig chain.Config, snapshots *snapshotsync.RoSnapshots, blockReader services.FullBlockReader, historyV3 bool, transactionsV3 bool) BodiesCfg

type CallTracesCfg 

type CallTracesCfg struct {
	ToBlock uint64 // not setting this params means no limit
	// contains filtered or unexported fields
}

func StageCallTracesCfg 

func StageCallTracesCfg(
	db kv.RwDB,
	prune prune.Mode,
	toBlock uint64,
	tmpdir string,
) CallTracesCfg

type ChainEventNotifier 

type ChainEventNotifier interface {
	OnNewHeader(newHeadersRlp [][]byte)
	OnNewPendingLogs(types.Logs)
	OnLogs([]*remote.SubscribeLogsReply)
	HasLogSubsriptions() bool
}

type ChainReader 

type ChainReader struct {
	Cfg chain.Config

	Db kv.Getter
}

Implements consensus.ChainReader

func (ChainReader) Config 

func (cr ChainReader) Config() *chain.Config

Config retrieves the blockchain's chain configuration.

func (ChainReader) CurrentHeader 

func (cr ChainReader) CurrentHeader() *types.Header

CurrentHeader retrieves the current header from the local chain.

func (ChainReader) GetBlock 

func (cr ChainReader) GetBlock(hash libcommon.Hash, number uint64) *types.Block

GetBlock retrieves a block from the database by hash and number.

func (ChainReader) GetHeader 

func (cr ChainReader) GetHeader(hash libcommon.Hash, number uint64) *types.Header

GetHeader retrieves a block header from the database by hash and number.

func (ChainReader) GetHeaderByHash 

func (cr ChainReader) GetHeaderByHash(hash libcommon.Hash) *types.Header

GetHeaderByHash retrieves a block header from the database by its hash.

func (ChainReader) GetHeaderByNumber 

func (cr ChainReader) GetHeaderByNumber(number uint64) *types.Header

GetHeaderByNumber retrieves a block header from the database by number.

func (ChainReader) GetTd 

func (cr ChainReader) GetTd(hash libcommon.Hash, number uint64) *big.Int

GetTd retrieves the total difficulty from the database by hash and number.

func (ChainReader) HasBlock 

func (cr ChainReader) HasBlock(hash libcommon.Hash, number uint64) bool

HasBlock retrieves a block from the database by hash and number.

type ChainReaderImpl 

type ChainReaderImpl struct {
	// contains filtered or unexported fields
}

func NewChainReaderImpl 

func NewChainReaderImpl(config *chain.Config, tx kv.Getter, blockReader services.FullBlockReader) *ChainReaderImpl

func (ChainReaderImpl) Config 

func (cr ChainReaderImpl) Config() *chain.Config

func (ChainReaderImpl) CurrentHeader 

func (cr ChainReaderImpl) CurrentHeader() *types.Header

func (ChainReaderImpl) GetHeader 

func (cr ChainReaderImpl) GetHeader(hash libcommon.Hash, number uint64) *types.Header

func (ChainReaderImpl) GetHeaderByHash 

func (cr ChainReaderImpl) GetHeaderByHash(hash libcommon.Hash) *types.Header

func (ChainReaderImpl) GetHeaderByNumber 

func (cr ChainReaderImpl) GetHeaderByNumber(number uint64) *types.Header

func (ChainReaderImpl) GetTd 

func (cr ChainReaderImpl) GetTd(hash libcommon.Hash, number uint64) *big.Int

type ChangeSetHook 

type ChangeSetHook func(blockNum uint64, wr *state.ChangeSetWriter)

type CumulativeIndexCfg 

type CumulativeIndexCfg struct {
	// contains filtered or unexported fields
}

func StageCumulativeIndexCfg 

func StageCumulativeIndexCfg(db kv.RwDB) CumulativeIndexCfg

type DownloaderGlue 

type DownloaderGlue interface {
	SpawnHeaderDownloadStage([]func() error, *sync_stages.StageState, sync_stages.Unwinder) error
	SpawnBodyDownloadStage(string, string, *sync_stages.StageState, sync_stages.Unwinder, *bodydownload.PrefetchedBlocks) (bool, error)
}

type ExecuteBlockCfg 

type ExecuteBlockCfg struct {
	// contains filtered or unexported fields
}

func StageExecuteBlocksCfg 

func StageExecuteBlocksCfg(
	db kv.RwDB,
	pm prune.Mode,
	batchSize datasize.ByteSize,
	changeSetHook ChangeSetHook,
	chainConfig *chain.Config,
	engine consensus.Engine,
	vmConfig *vm.Config,
	accumulator *shards.Accumulator,
	stateStream bool,
	badBlockHalt bool,

	historyV3 bool,
	dirs datadir.Dirs,
	blockReader services.FullBlockReader,
	hd headerDownloader,
	genesis *types.Genesis,
	syncCfg ethconfig.Sync,
	agg *libstate.AggregatorV3,
	zk *ethconfig.Zk,
) ExecuteBlockCfg

type FinishCfg 

type FinishCfg struct {
	// contains filtered or unexported fields
}

func StageFinishCfg 

func StageFinishCfg(db kv.RwDB, tmpDir string, forkValidator *engineapi.ForkValidator) FinishCfg

type HasChangeSetWriter 

type HasChangeSetWriter interface {
	ChangeSetWriter() *state.ChangeSetWriter
}

type HashPromoter 

type HashPromoter struct {
	ChangeSetBufSize uint64
	TempDir          string
	// contains filtered or unexported fields
}

func NewHashPromoter 

func NewHashPromoter(db kv.RwTx, tempDir string, quitCh <-chan struct{}, logPrefix string) *HashPromoter

func (*HashPromoter) Promote 

func (p *HashPromoter) Promote(logPrefix string, from, to uint64, storage bool, load etl.LoadFunc) error

func (*HashPromoter) PromoteOnHistoryV3 

func (p *HashPromoter) PromoteOnHistoryV3(logPrefix string, agg *state.AggregatorV3, from, to uint64, storage bool, load func(k []byte, v []byte) error) error

func (*HashPromoter) Unwind 

func (p *HashPromoter) Unwind(logPrefix string, s *sync_stages.StageState, u *sync_stages.UnwindState, storage bool, load etl.LoadFunc) error

func (*HashPromoter) UnwindOnHistoryV3 

func (p *HashPromoter) UnwindOnHistoryV3(logPrefix string, agg *state.AggregatorV3, unwindFrom, unwindTo uint64, storage bool, load func(k []byte, v []byte)) error

type HashStateCfg 

type HashStateCfg struct {
	// contains filtered or unexported fields
}

func StageHashStateCfg 

func StageHashStateCfg(db kv.RwDB, dirs datadir.Dirs, historyV3 bool, agg *state.AggregatorV3) HashStateCfg

type HeadersCfg 

type HeadersCfg struct {
	// contains filtered or unexported fields
}

func StageHeadersCfg 

func StageHeadersCfg(
	db kv.RwDB,
	headerDownload *headerdownload.HeaderDownload,
	bodyDownload *bodydownload.BodyDownload,
	chainConfig chain.Config,
	headerReqSend func(context.Context, *headerdownload.HeaderRequest) ([64]byte, bool),
	announceNewHashes func(context.Context, []headerdownload.Announce),
	penalize func(context.Context, []headerdownload.PenaltyItem),
	batchSize datasize.ByteSize,
	noP2PDiscovery bool,
	snapshots *snapshotsync.RoSnapshots,
	blockReader services.FullBlockReader,
	tmpdir string,
	notifications *shards.Notifications,
	forkValidator *engineapi.ForkValidator) HeadersCfg

type HermezDb 

type HermezDb interface {
	GetBlockGlobalExitRoot(l2BlockNo uint64) (common.Hash, error)
}

type HistoryCfg 

type HistoryCfg struct {
	// contains filtered or unexported fields
}

func StageHistoryCfg 

func StageHistoryCfg(db kv.RwDB, prune prune.Mode, tmpDir string) HistoryCfg

type LogIndexCfg 

type LogIndexCfg struct {
	// contains filtered or unexported fields
}

func StageLogIndexCfg 

func StageLogIndexCfg(db kv.RwDB, prune prune.Mode, tmpDir string) LogIndexCfg

type MiningBlock 

type MiningBlock struct {
	Header      *types.Header
	Uncles      []*types.Header
	Txs         types.Transactions
	Receipts    types.Receipts
	Withdrawals []*types.Withdrawal
	PreparedTxs types.TransactionsStream
}

type MiningCreateBlockCfg 

type MiningCreateBlockCfg struct {
	// contains filtered or unexported fields
}

func StageMiningCreateBlockCfg 

func StageMiningCreateBlockCfg(db kv.RwDB, miner MiningState, chainConfig chain.Config, engine consensus.Engine, txPool2 *txpool.TxPool, txPool2DB kv.RoDB, blockBuilderParameters *core.BlockBuilderParameters, tmpdir string) MiningCreateBlockCfg

type MiningExecCfg 

type MiningExecCfg struct {
	// contains filtered or unexported fields
}

func StageMiningExecCfg 

func StageMiningExecCfg(
	db kv.RwDB,
	miningState MiningState,
	notifier ChainEventNotifier,
	chainConfig chain.Config,
	engine consensus.Engine,
	vmConfig *vm.Config,
	tmpdir string,
	interrupt *int32,
	payloadId uint64,
	txPool2 *txpool.TxPool,
	txPool2DB kv.RoDB,
	snapshots *snapshotsync.RoSnapshots,
	transactionsV3 bool,
) MiningExecCfg

type MiningFinishCfg 

type MiningFinishCfg struct {
	// contains filtered or unexported fields
}

func StageMiningFinishCfg 

func StageMiningFinishCfg(
	db kv.RwDB,
	chainConfig chain.Config,
	engine consensus.Engine,
	miningState MiningState,
	sealCancel chan struct{},
) MiningFinishCfg

type MiningState 

type MiningState struct {
	MiningConfig      *params.MiningConfig
	PendingResultCh   chan *types.Block
	MiningResultCh    chan *types.Block
	MiningResultPOSCh chan *types.BlockWithReceipts
	MiningBlock       *MiningBlock
}

func NewMiningState 

func NewMiningState(cfg *params.MiningConfig) MiningState

func NewProposingState 

func NewProposingState(cfg *params.MiningConfig) MiningState

type OldestAppearedLoad 

type OldestAppearedLoad struct {
	// contains filtered or unexported fields
}

func (*OldestAppearedLoad) LoadFunc 

func (l *OldestAppearedLoad) LoadFunc(k, v []byte, table etl.CurrentTableReader, next etl.LoadNextFunc) error

type PostExecCfg 

type PostExecCfg struct {
	// contains filtered or unexported fields
}

func StagePostExecCfg 

func StagePostExecCfg(db kv.RwDB, borDb kv.RwDB) PostExecCfg

type Progress 

type Progress struct {
	// contains filtered or unexported fields
}

func NewProgress 

func NewProgress(prevOutputBlockNum, commitThreshold uint64, workersCount int, logPrefix string) *Progress

func (*Progress) Log 

func (p *Progress) Log(rs *state.StateV3, in *exec22.QueueWithRetry, rws *exec22.ResultsQueue, doneCount, inputBlockNum, outputBlockNum, outTxNum, repeatCount uint64, idxStepsAmountInDB float64)

type Promoter 

type Promoter struct {
	ChangeSetBufSize uint64
	// contains filtered or unexported fields
}

func NewPromoter 

func NewPromoter(db kv.RwTx, dirs datadir.Dirs, ctx context.Context) *Promoter

func (*Promoter) Promote 

func (p *Promoter) Promote(logPrefix string, from, to uint64, storage, codes bool, quiet bool) error

func (*Promoter) PromoteOnHistoryV3 

func (p *Promoter) PromoteOnHistoryV3(logPrefix string, agg *state.AggregatorV3, from, to uint64, storage, quiet bool) error

func (*Promoter) Unwind 

func (p *Promoter) Unwind(logPrefix string, s *sync_stages.StageState, u *sync_stages.UnwindState, storage bool, codes bool) error

func (*Promoter) UnwindOnHistoryV3 

func (p *Promoter) UnwindOnHistoryV3(logPrefix string, agg *state.AggregatorV3, unwindFrom, unwindTo uint64, storage, codes bool) error

type SendersCfg 

type SendersCfg struct {
	// contains filtered or unexported fields
}

func StageSendersCfg 

func StageSendersCfg(db kv.RwDB, chainCfg *chain.Config, badBlockHalt bool, tmpdir string, prune prune.Mode, br *snapshotsync.BlockRetire, hd *headerdownload.HeaderDownload) SendersCfg

type SnapshotsCfg 

type SnapshotsCfg struct {
	// contains filtered or unexported fields
}

func StageSnapshotsCfg 

func StageSnapshotsCfg(
	db kv.RwDB,
	chainConfig chain.Config,
	dirs datadir.Dirs,
	snapshots *snapshotsync.RoSnapshots,
	blockRetire *snapshotsync.BlockRetire,
	snapshotDownloader proto_downloader.DownloaderClient,
	blockReader services.FullBlockReader,
	dbEventNotifier snapshotsync.DBEventNotifier,
	engine consensus.Engine,
	historyV3 bool,
	agg *state.AggregatorV3,
) SnapshotsCfg

type TrieCfg 

type TrieCfg struct {
	// contains filtered or unexported fields
}

func StageTrieCfg 

func StageTrieCfg(db kv.RwDB, checkRoot, saveNewHashesToDB, badBlockHalt bool, tmpDir string, blockReader services.FullBlockReader, hd *headerdownload.HeaderDownload, historyV3 bool, agg *state.AggregatorV3) TrieCfg

type TxLookupCfg 

type TxLookupCfg struct {
	// contains filtered or unexported fields
}

func StageTxLookupCfg 

func StageTxLookupCfg(
	db kv.RwDB,
	prune prune.Mode,
	tmpdir string,
	snapshots *snapshotsync.RoSnapshots,
	borConfig *erigonchain.BorConfig,
) TxLookupCfg

type WithSnapshots 

type WithSnapshots interface {
	Snapshots() *snapshotsync.RoSnapshots
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.