beacon

package
v1.12.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 25, 2024 License: GPL-3.0 Imports: 38 Imported by: 0

README

Dora: beaconchain indexer

Welcome to the documentation for the indexer package of Dora the Beacon Chain Explorer. This package is responsible for indexing the beacon chain, including blocks, duties, and the current chain state. It consumes events from one or more pool.consensus.Client clients, updates database indexes, and provides functions to gain insights into the current network state.

Overview

The indexer package is divided into several components:

  • Initialization routine
  • Input processor
  • Caching subsystems
    • Block cache
    • Epoch cache
    • Fork cache
  • Processing routines
    • Finalization
    • Pruning
    • Synchronization

Initialization Routine

The initialization routine prefills all caching subsystems with the latest preserved state from the database. It uses the currently finalized epoch as a boundary for loading entities, so only objects between the current wallclock epoch and the current finalized epoch are restored. Entities older than inMemoryEpochs are directly pruned to save memory. This process runs before the indexer starts to receive events from the attached clients. For networks with large unfinality periods, this might take a few seconds to minutes.

Input Processor

The input processor consumes events from an underlying consensus pool client and feeds the caching subsystems with new data. It:

  • Attaches to "block" and "head" events.
  • Processes each received block, loading its block header and body.
  • Ensures the completeness of the block cache by backfilling parent blocks.
  • Dumps fully received blocks into the unfinalized blocks table for restoration after restart.
  • Tracks the latest head of the client and detects chain reorgs.

An instance of the input processor is created for every client attached to the indexer via indexer.AddClient.

Caching Subsystems

Block Cache

The block cache subsystem keeps an overview of all unfinalized blocks in the network. It:

  • Provides block accessors by various block properties.
  • Checks block relationships and calculates distances between blocks.
  • Might hold pruned blocks (without block bodies), but bodies can be restored on purpose.
Epoch Cache

The epoch cache subsystem maintains an overview of unfinalized epoch stats. It:

  • Holds duty and epoch statistics for each epoch.
  • Identifies epoch stats by the epoch's dependent root and number.
  • Might hold pruned epoch stats, keeping only proposer + sync committee duties and aggregations in memory
  • Can restores full epoch stats from the database, which is an expensive operation.
  • Can pre-calculate the next epoch stats, however this might not be accurate all the time.
Fork Cache

The fork cache subsystem keeps an overview of all unfinalized forks in the network. It:

  • Identifies forks by the base block root and leaf block root.
  • Numbers forks with an ascending internal counter.
  • Holds the fork detection system to identify current fork IDs and detect possible forks.
  • Uses forkId as a placeholder for unknown or canonical finalized forks.

Processing Routines

Finalization Routine

The finalization routine processes unfinalized blocks from the block cache after finalization. It:

  • Writes blocks, epoch aggregations, and child objects to the database.
  • Stores orphaned blocks in the orphaned blocks table.
  • Wipes processed blocks and epoch stats from caches.
Pruning Routine

The pruning routine pre-finalizes old unfinalized epochs and prunes memory-heavy data from cache. It:

  • Is controlled by the inMemoryEpochs setting.
  • Generates non-final epoch voting aggregations.
  • Writes blocks and child objects to the database.
  • Stores pruned epoch aggregations in cache and the unfinalized epochs table for restoration.
Synchronization Routine

The synchronization routine processes all missed historic data. It:

  • Ensures each epoch is properly indexed.
  • Loads canonical blocks and dependent states from a ready node.
  • Computes epoch aggregations and writes them, along with canonical blocks and child objects, to the database.
  • Is triggered by failed finalization or the initialization routine.

Documentation

Index

Constants

View Source
const EtherGweiFactor = 1_000_000_000
View Source
const FarFutureEpoch = phase0.Epoch(math.MaxUint64)

Variables

This section is empty.

Functions

func LoadBeaconBlock

func LoadBeaconBlock(ctx context.Context, client *Client, root phase0.Root) (*spec.VersionedSignedBeaconBlock, error)

LoadBeaconBlock loads the block body from the RPC client.

func LoadBeaconHeader

func LoadBeaconHeader(ctx context.Context, client *Client, root phase0.Root) (*phase0.SignedBeaconBlockHeader, error)

LoadBeaconHeader loads the block header from the client.

func LoadBeaconHeaderBySlot

func LoadBeaconHeaderBySlot(ctx context.Context, client *Client, slot phase0.Slot) (*phase0.SignedBeaconBlockHeader, phase0.Root, bool, error)

LoadBeaconHeaderBySlot loads the block header with given slot number from the client.

func LoadBeaconState

func LoadBeaconState(ctx context.Context, client *Client, root phase0.Root) (*spec.VersionedBeaconState, error)

LoadBeaconState loads the beacon state from the client.

Types

type Block

type Block struct {
	Root phase0.Root
	Slot phase0.Slot
	// contains filtered or unexported fields
}

Block represents a beacon block.

func (*Block) AwaitBlock

func (block *Block) AwaitBlock(ctx context.Context, timeout time.Duration) *spec.VersionedSignedBeaconBlock

AwaitBlock waits for the versioned signed beacon block of this block to be available.

func (*Block) AwaitHeader

func (block *Block) AwaitHeader(ctx context.Context, timeout time.Duration) *phase0.SignedBeaconBlockHeader

AwaitHeader waits for the signed beacon block header of this block to be available.

func (*Block) EnsureBlock

func (block *Block) EnsureBlock(loadBlock func() (*spec.VersionedSignedBeaconBlock, error)) (bool, error)

EnsureBlock ensures that the versioned signed beacon block of this block is available.

func (*Block) EnsureHeader

func (block *Block) EnsureHeader(loadHeader func() (*phase0.SignedBeaconBlockHeader, error)) error

EnsureHeader ensures that the signed beacon block header of this block is available.

func (*Block) GetBlock

func (block *Block) GetBlock() *spec.VersionedSignedBeaconBlock

GetBlock returns the versioned signed beacon block of this block.

func (*Block) GetBlockIndex

func (block *Block) GetBlockIndex() *BlockBodyIndex

GetBlockIndex returns the block index of this block.

func (*Block) GetDbBlock

func (block *Block) GetDbBlock(indexer *Indexer) *dbtypes.Slot

GetDbBlock returns the database representation of this block.

func (*Block) GetDbConsolidationRequests

func (block *Block) GetDbConsolidationRequests(indexer *Indexer) []*dbtypes.ConsolidationRequest

GetDbConsolidationRequests returns the database representation of the consolidation requests in this block.

func (*Block) GetDbDeposits

func (block *Block) GetDbDeposits(indexer *Indexer, depositIndex *uint64) []*dbtypes.Deposit

GetDbDeposits returns the database representation of the deposits in this block.

func (*Block) GetDbSlashings

func (block *Block) GetDbSlashings(indexer *Indexer) []*dbtypes.Slashing

GetDbSlashings returns the database representation of the slashings in this block.

func (*Block) GetDbVoluntaryExits

func (block *Block) GetDbVoluntaryExits(indexer *Indexer) []*dbtypes.VoluntaryExit

GetDbVoluntaryExits returns the database representation of the voluntary exits in this block.

func (*Block) GetDbWithdrawalRequests added in v1.12.0

func (block *Block) GetDbWithdrawalRequests(indexer *Indexer) []*dbtypes.WithdrawalRequest

GetDbWithdrawalRequests returns the database representation of the withdrawal requests in this block.

func (*Block) GetForkId

func (block *Block) GetForkId() ForkKey

GetForkId returns the fork ID of this block.

func (*Block) GetHeader

func (block *Block) GetHeader() *phase0.SignedBeaconBlockHeader

GetHeader returns the signed beacon block header of this block.

func (*Block) GetParentRoot

func (block *Block) GetParentRoot() *phase0.Root

GetParentRoot returns the parent root of this block.

func (*Block) GetSeenBy

func (block *Block) GetSeenBy() []*Client

GetSeenBy returns a list of clients that have seen this block.

func (*Block) SetBlock

func (block *Block) SetBlock(body *spec.VersionedSignedBeaconBlock)

SetBlock sets the versioned signed beacon block of this block.

func (*Block) SetHeader

func (block *Block) SetHeader(header *phase0.SignedBeaconBlockHeader)

SetHeader sets the signed beacon block header of this block.

func (*Block) SetSeenBy

func (block *Block) SetSeenBy(client *Client)

SetSeenBy sets the client that has seen this block.

type BlockBodyIndex

type BlockBodyIndex struct {
	Graffiti           [32]byte
	ExecutionExtraData []byte
	ExecutionHash      phase0.Hash32
	ExecutionNumber    uint64
}

BlockBodyIndex holds important block propoerties that are used as index for cache lookups. this structure should be preserved after pruning, so the block is still identifiable.

type ChainHead

type ChainHead struct {
	HeadBlock             *Block      // The head block of the chain.
	AggregatedHeadVotes   phase0.Gwei // The aggregated votes of the last 2 epochs for the head block.
	PerEpochVotingPercent []float64   // The voting percentage in the last epochs (ascendeing order).
}

ChainHead represents a head block of the chain.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client represents a consensus pool client that should be used for indexing beacon blocks.

func (*Client) GetClient

func (c *Client) GetClient() *consensus.Client

func (*Client) GetIndex

func (c *Client) GetIndex() uint16

func (*Client) GetPriority

func (c *Client) GetPriority() int

type EpochStats

type EpochStats struct {
	// contains filtered or unexported fields
}

EpochStats holds the epoch-specific information based on the underlying dependent beacon state.

func (*EpochStats) GetDbEpoch

func (es *EpochStats) GetDbEpoch(indexer *Indexer, headBlock *Block) *dbtypes.Epoch

GetDbEpoch returns the database Epoch representaion for the EpochStats.

func (*EpochStats) GetDependentRoot added in v1.12.1

func (es *EpochStats) GetDependentRoot() phase0.Root

func (*EpochStats) GetEpoch

func (es *EpochStats) GetEpoch() phase0.Epoch

func (*EpochStats) GetEpochVotes

func (es *EpochStats) GetEpochVotes(indexer *Indexer, headBlock *Block) *EpochVotes

GetEpochVotes aggregates & returns the EpochVotes for the EpochStats.

func (*EpochStats) GetOrLoadValues

func (es *EpochStats) GetOrLoadValues(indexer *Indexer, withPrecalc bool, keepInCache bool) *EpochStatsValues

GetOrLoadValues returns the EpochStats values, loading them from the database if necessary.

func (*EpochStats) GetValues

func (es *EpochStats) GetValues(withPrecalc bool) *EpochStatsValues

GetValues returns the EpochStats values.

type EpochStatsPacked

type EpochStatsPacked struct {
	ActiveValidators    []EpochStatsPackedValidator
	SyncCommitteeDuties []phase0.ValidatorIndex
	RandaoMix           phase0.Hash32
	NextRandaoMix       phase0.Hash32
	TotalBalance        phase0.Gwei
	ActiveBalance       phase0.Gwei
	FirstDepositIndex   uint64
}

EpochStatsPacked holds the packed values for the epoch-specific information.

type EpochStatsPackedValidator

type EpochStatsPackedValidator struct {
	ValidatorIndexOffset uint32 // offset to the previous index in the list (this is smaller than storing the full validator index)
	EffectiveBalanceEth  uint16 // effective balance in full ETH
}

EpochStatsPackedValidator holds the packed values for an active validator.

type EpochStatsValues

type EpochStatsValues struct {
	RandaoMix           phase0.Hash32
	NextRandaoMix       phase0.Hash32
	ActiveIndices       []phase0.ValidatorIndex
	EffectiveBalances   []uint16
	ProposerDuties      []phase0.ValidatorIndex
	AttesterDuties      [][][]duties.ActiveIndiceIndex
	SyncCommitteeDuties []phase0.ValidatorIndex
	ActiveValidators    uint64
	TotalBalance        phase0.Gwei
	ActiveBalance       phase0.Gwei
	EffectiveBalance    phase0.Gwei
	FirstDepositIndex   uint64
}

EpochStatsValues holds the values for the epoch-specific information.

func (*EpochStatsValues) GetEffectiveBalance

func (v *EpochStatsValues) GetEffectiveBalance(index duties.ActiveIndiceIndex) phase0.Gwei

GetEffectiveBalance returns the effective balance for the given active validator indice.

type EpochVotes

type EpochVotes struct {
	CurrentEpoch struct {
		TargetVoteAmount phase0.Gwei
		HeadVoteAmount   phase0.Gwei
		TotalVoteAmount  phase0.Gwei
	}
	NextEpoch struct {
		TargetVoteAmount phase0.Gwei
		HeadVoteAmount   phase0.Gwei
		TotalVoteAmount  phase0.Gwei
	}
	TargetVotePercent float64
	HeadVotePercent   float64
	TotalVotePercent  float64
	AmountIsCount     bool
	ActivityBitfield  bitfield.Bitfield
}

EpochVotes represents the aggregated votes for an epoch.

type Fork

type Fork struct {
	// contains filtered or unexported fields
}

Fork represents a fork in the beacon chain.

func (*Fork) GetBase

func (fork *Fork) GetBase() (phase0.Slot, phase0.Root)

func (*Fork) GetLeaf

func (fork *Fork) GetLeaf() (phase0.Slot, phase0.Root)

func (*Fork) GetParent

func (fork *Fork) GetParent() ForkKey

type ForkHead

type ForkHead struct {
	ForkId ForkKey
	Fork   *Fork
	Block  *Block
}

ForkHead represents a fork head with its ID, fork, and block.

type ForkKey

type ForkKey uint64

ForkKey represents a key used for indexing forks.

type Indexer

type Indexer struct {
	// contains filtered or unexported fields
}

Indexer is responsible for indexing the ethereum beacon chain.

func NewIndexer

func NewIndexer(logger logrus.FieldLogger, consensusPool *consensus.Pool) *Indexer

NewIndexer creates a new instance of the Indexer.

func (*Indexer) AddClient

func (indexer *Indexer) AddClient(index uint16, client *consensus.Client, priority int, archive bool, skipValidators bool) *Client

AddClient adds a new consensus pool client to the indexer.

func (*Indexer) GetAllClients

func (indexer *Indexer) GetAllClients() []*Client

GetAllClients returns a slice of all clients in the indexer.

func (*Indexer) GetBlockByParentRoot

func (indexer *Indexer) GetBlockByParentRoot(blockRoot phase0.Root) []*Block

GetBlockByParentRoot returns a slice of blocks with the given parent root.

func (*Indexer) GetBlockByRoot

func (indexer *Indexer) GetBlockByRoot(blockRoot phase0.Root) *Block

GetBlockByRoot returns the block with the given block root.

func (*Indexer) GetBlockByStateRoot

func (indexer *Indexer) GetBlockByStateRoot(stateRoot phase0.Root) *Block

GetBlockByStateRoot returns the block with the given state root.

func (*Indexer) GetBlockCacheState

func (indexer *Indexer) GetBlockCacheState() (finalizedEpoch phase0.Epoch, prunedEpoch phase0.Epoch)

GetBlockCacheState returns the state of the block cache, including the last finalized epoch and the last pruned epoch. this represents the internal cache state and might be behind the actual finalization checkpoint.

func (*Indexer) GetBlockDistance

func (indexer *Indexer) GetBlockDistance(baseRoot phase0.Root, headRoot phase0.Root) (bool, uint64)

GetBlockDistance returns whether the base root is in the canonical chain defined by the head root and the distance between both blocks.

func (*Indexer) GetBlocksByExecutionBlockHash

func (indexer *Indexer) GetBlocksByExecutionBlockHash(blockHash phase0.Hash32) []*Block

GetBlocksByExecutionBlockHash returns a slice of blocks with the given execution block hash.

func (*Indexer) GetBlocksByExecutionBlockNumber

func (indexer *Indexer) GetBlocksByExecutionBlockNumber(blockNumber uint64) []*Block

GetBlocksByExecutionBlockNumber returns a slice of blocks with the given execution block number.

func (*Indexer) GetBlocksBySlot

func (indexer *Indexer) GetBlocksBySlot(slot phase0.Slot) []*Block

GetBlocksBySlot returns a slice of blocks with the given slot.

func (*Indexer) GetCanonicalHead

func (indexer *Indexer) GetCanonicalHead(overrideForkId *ForkKey) *Block

GetCanonicalHead returns the canonical head block of the chain.

func (*Indexer) GetCanonicalValidatorSet

func (indexer *Indexer) GetCanonicalValidatorSet(overrideForkId *ForkKey) []*v1.Validator

GetCanonicalValidatorSet returns the latest canonical validator set. If an overrideForkId is provided, the latest validator set for the fork is returned.

func (*Indexer) GetChainHeads

func (indexer *Indexer) GetChainHeads() []*ChainHead

GetChainHeads returns the chain heads sorted by voting percentages.

func (*Indexer) GetEpochStats

func (indexer *Indexer) GetEpochStats(epoch phase0.Epoch, overrideForkId *ForkKey) *EpochStats

GetEpochStats returns the epoch stats for the given epoch and optional fork ID override.

func (*Indexer) GetForkHeads

func (indexer *Indexer) GetForkHeads() []*ForkHead

GetForkHeads returns a slice of fork heads in the indexer.

func (*Indexer) GetOrphanedBlockByRoot

func (indexer *Indexer) GetOrphanedBlockByRoot(blockRoot phase0.Root) (*Block, error)

GetOrphanedBlockByRoot returns the orphaned block with the given block root.

func (*Indexer) GetParentForkIds added in v1.12.1

func (indexer *Indexer) GetParentForkIds(forkId ForkKey) []ForkKey

GetParentForkIds returns the parent fork ids of the given fork.

func (*Indexer) GetReadyClient

func (indexer *Indexer) GetReadyClient(preferArchive bool) *Client

GetReadyClient returns a single client that is on the finalized chain and preference for archive clients.

func (*Indexer) GetReadyClientByBlockRoot

func (indexer *Indexer) GetReadyClientByBlockRoot(blockRoot phase0.Root, preferArchive bool) *Client

GetReadyClientByBlockRoot returns a single client that is ready for requests for the chain including the block root and preference for archive clients.

func (*Indexer) GetReadyClients

func (indexer *Indexer) GetReadyClients(preferArchive bool) []*Client

GetReadyClients returns a slice of clients that are on the finalized chain and preference for archive clients.

func (*Indexer) GetReadyClientsByBlockRoot

func (indexer *Indexer) GetReadyClientsByBlockRoot(blockRoot phase0.Root, preferArchive bool) []*Client

GetReadyClientsByBlockRoot returns a slice of clients that are ready for requests for the chain including the block root and preference for archive clients.

func (*Indexer) GetReadyClientsByCheckpoint

func (indexer *Indexer) GetReadyClientsByCheckpoint(finalizedRoot phase0.Root, preferArchive bool) []*Client

GetReadyClientsByCheckpoint returns a slice of clients that are ready for processing based on the finalized root and preference for archive clients.

func (*Indexer) GetSynchronizerState added in v1.12.1

func (indexer *Indexer) GetSynchronizerState() (running bool, syncHead phase0.Epoch)

GetSynchronizerState returns the state of the synchronizer, including whether it is running and the current epoch.

func (*Indexer) IsCanonicalBlock

func (indexer *Indexer) IsCanonicalBlock(block *Block, overrideForkId *ForkKey) bool

func (*Indexer) StartIndexer

func (indexer *Indexer) StartIndexer()

StartIndexer starts the indexing process.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL