indexer

Documentation

Index

• Variables
• func HandleCollection(collection *flow.Collection, collections storage.Collections, ...) error
• type CollectionExecutedMetricImpl
  • func NewCollectionExecutedMetricImpl(log zerolog.Logger, accessMetrics module.AccessMetrics, ...) (*CollectionExecutedMetricImpl, error)
  • func (c *CollectionExecutedMetricImpl) BlockFinalized(block *flow.Block)
  • func (c *CollectionExecutedMetricImpl) CollectionExecuted(light flow.LightCollection)
  • func (c *CollectionExecutedMetricImpl) CollectionFinalized(light flow.LightCollection)
  • func (c *CollectionExecutedMetricImpl) ExecutionReceiptReceived(r *flow.ExecutionReceipt)
  • func (c *CollectionExecutedMetricImpl) UpdateLastFullBlockHeight(height uint64)
• type Indexer
  • func NewIndexer(log zerolog.Logger, initHeight uint64, registers storage.RegisterIndex, ...) (*Indexer, error)
  • func (i *Indexer) HighestIndexedHeight() (uint64, error)
  • func (i *Indexer) LowestIndexedHeight() (uint64, error)
  • func (i *Indexer) OnExecutionData(_ *execution_data.BlockExecutionDataEntity)
  • func (i *Indexer) Start(ctx irrecoverable.SignalerContext)
• type IndexerCore
  • func New(log zerolog.Logger, metrics module.ExecutionStateIndexerMetrics, ...) (*IndexerCore, error)
  • func (c *IndexerCore) IndexBlockData(data *execution_data.BlockExecutionDataEntity) error
  • func (c *IndexerCore) RegisterValue(ID flow.RegisterID, height uint64) (flow.RegisterValue, error)

Variables

var ErrIndexNotInitialized = errors.New("index not initialized")

ErrIndexNotInitialized is returned when the indexer is not initialized

This generally indicates that the index databases are still being initialized, and trying again later may succeed

Functions

func HandleCollection

func HandleCollection(
	collection *flow.Collection,
	collections storage.Collections,
	transactions storage.Transactions,
	logger zerolog.Logger,
	collectionExecutedMetric module.CollectionExecutedMetric,
) error

HandleCollection handles the response of the collection request made earlier when a block was received. No errors expected during normal operations.

Types

type CollectionExecutedMetricImpl

type CollectionExecutedMetricImpl struct {
	// contains filtered or unexported fields
}

CollectionExecutedMetricImpl tracks metrics to measure how long it takes for tx to reach each step in their lifecycle

func NewCollectionExecutedMetricImpl

func NewCollectionExecutedMetricImpl(
	log zerolog.Logger,
	accessMetrics module.AccessMetrics,
	collectionsToMarkFinalized *stdmap.Times,
	collectionsToMarkExecuted *stdmap.Times,
	blocksToMarkExecuted *stdmap.Times,
	collections storage.Collections,
	blocks storage.Blocks,
	blockTransactions *stdmap.IdentifierMap,
) (*CollectionExecutedMetricImpl, error)

func (*CollectionExecutedMetricImpl) BlockFinalized

func (c *CollectionExecutedMetricImpl) BlockFinalized(block *flow.Block)

BlockFinalized tracks finalized metric for block

func (*CollectionExecutedMetricImpl) CollectionExecuted

func (c *CollectionExecutedMetricImpl) CollectionExecuted(light flow.LightCollection)

CollectionExecuted tracks collections to mark executed

func (*CollectionExecutedMetricImpl) CollectionFinalized

func (c *CollectionExecutedMetricImpl) CollectionFinalized(light flow.LightCollection)

CollectionFinalized tracks collections to mark finalized

func (*CollectionExecutedMetricImpl) ExecutionReceiptReceived

func (c *CollectionExecutedMetricImpl) ExecutionReceiptReceived(r *flow.ExecutionReceipt)

ExecutionReceiptReceived tracks execution receipt metrics

func (*CollectionExecutedMetricImpl) UpdateLastFullBlockHeight

func (c *CollectionExecutedMetricImpl) UpdateLastFullBlockHeight(height uint64)

type Indexer

type Indexer struct {
	component.Component
	execution_data.ProcessedHeightRecorder
	// contains filtered or unexported fields
}

Indexer handles ingestion of new execution data available and uses the execution data indexer module to index the data. The processing of new available data is done by creating a jobqueue that uses the execution data reader to obtain new jobs. The worker also implements the `highestConsecutiveHeight` method which is used by the execution data reader, so it doesn't surpass the highest sealed block height when fetching the data. The execution state worker has a callback that is used by the upstream queues which download new execution data to notify new data is available and kick off indexing.

func NewIndexer

func NewIndexer(
	log zerolog.Logger,
	initHeight uint64,
	registers storage.RegisterIndex,
	indexer *IndexerCore,
	executionCache *cache.ExecutionDataCache,
	executionDataLatestHeight func() (uint64, error),
	processedHeight storage.ConsumerProgress,
) (*Indexer, error)

NewIndexer creates a new execution worker.

func (*Indexer) HighestIndexedHeight

func (i *Indexer) HighestIndexedHeight() (uint64, error)

HighestIndexedHeight returns the highest height indexed by the execution indexer.

func (*Indexer) LowestIndexedHeight

func (i *Indexer) LowestIndexedHeight() (uint64, error)

LowestIndexedHeight returns the lowest height indexed by the execution indexer.

func (*Indexer) OnExecutionData

func (i *Indexer) OnExecutionData(_ *execution_data.BlockExecutionDataEntity)

OnExecutionData is used to notify when new execution data is downloaded by the execution data requester jobqueue.

func (*Indexer) Start

func (i *Indexer) Start(ctx irrecoverable.SignalerContext)

Start the worker jobqueue to consume the available data.

type IndexerCore

type IndexerCore struct {
	// contains filtered or unexported fields
}

IndexerCore indexes the execution state.

func New

func New(
	log zerolog.Logger,
	metrics module.ExecutionStateIndexerMetrics,
	batcher bstorage.BatchBuilder,
	registers storage.RegisterIndex,
	headers storage.Headers,
	events storage.Events,
	collections storage.Collections,
	transactions storage.Transactions,
	results storage.LightTransactionResults,
	chain flow.Chain,
	derivedChainData *derived.DerivedChainData,
	collectionExecutedMetric module.CollectionExecutedMetric,
) (*IndexerCore, error)

New execution state indexer used to ingest block execution data and index it by height. The passed RegisterIndex storage must be populated to include the first and last height otherwise the indexer won't be initialized to ensure we have bootstrapped the storage first.

func (*IndexerCore) IndexBlockData

func (c *IndexerCore) IndexBlockData(data *execution_data.BlockExecutionDataEntity) error

IndexBlockData indexes all execution block data by height. This method shouldn't be used concurrently. Expected errors: - storage.ErrNotFound if the block for execution data was not found

func (*IndexerCore) RegisterValue

func (c *IndexerCore) RegisterValue(ID flow.RegisterID, height uint64) (flow.RegisterValue, error)

RegisterValue retrieves register values by the register IDs at the provided block height. Even if the register wasn't indexed at the provided height, returns the highest height the register was indexed at. If a register is not found it will return a nil value and not an error. Expected errors: - storage.ErrHeightNotIndexed if the given height was not indexed yet or lower than the first indexed height.