indexer

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 26, 2022 License: Apache-2.0, MIT Imports: 57 Imported by: 0

Documentation

Overview

Code generate by: `make tasks-gen`. DO NOT EDIT.

Index

Constants

View Source
const (
	BlockHeader                    = "block_header"
	BlockParent                    = "block_parent"
	DrandBlockEntrie               = "drand_block_entrie"
	MinerSectorDeal                = "miner_sector_deal"
	MinerSectorInfoV7              = "miner_sector_infos_v7"
	MinerSectorInfoV1_6            = "miner_sector_infos"
	MinerSectorPost                = "miner_sector_post"
	MinerPreCommitInfo             = "miner_pre_commit_info"
	MinerSectorEvent               = "miner_sector_event"
	MinerCurrentDeadlineInfo       = "miner_current_deadline_info"
	MinerFeeDebt                   = "miner_fee_debt"
	MinerLockedFund                = "miner_locked_fund"
	MinerInfo                      = "miner_info"
	MarketDealProposal             = "market_deal_proposal"
	MarketDealState                = "market_deal_state"
	Message                        = "message"
	BlockMessage                   = "block_message"
	Receipt                        = "receipt"
	MessageGasEconomy              = "message_gas_economy"
	ParsedMessage                  = "parsed_message"
	InternalMessage                = "internal_messages"
	InternalParsedMessage          = "internal_parsed_messages"
	MultisigTransaction            = "multisig_transaction"
	ChainPower                     = "chain_power"
	PowerActorClaim                = "power_actor_claim"
	ChainReward                    = "chain_reward"
	Actor                          = "actor"
	ActorState                     = "actor_state"
	IdAddress                      = "id_address"
	GasOutputs                     = "derived_gas_outputs"
	ChainEconomics                 = "chain_economics"
	ChainConsensus                 = "chain_consensus"
	MultisigApproval               = "multisig_approvals"
	VerifiedRegistryVerifier       = "verified_registry_verifier"
	VerifiedRegistryVerifiedClient = "verified_registry_verified_client"
)
View Source
const (
	ActorStatesRawTask      = "actorstatesraw"      // task that only extracts raw actor state
	ActorStatesPowerTask    = "actorstatespower"    // task that only extracts power actor states (but not the raw state)
	ActorStatesRewardTask   = "actorstatesreward"   // task that only extracts reward actor states (but not the raw state)
	ActorStatesMinerTask    = "actorstatesminer"    // task that only extracts miner actor states (but not the raw state)
	ActorStatesInitTask     = "actorstatesinit"     // task that only extracts init actor states (but not the raw state)
	ActorStatesMarketTask   = "actorstatesmarket"   // task that only extracts market actor states (but not the raw state)
	ActorStatesMultisigTask = "actorstatesmultisig" // task that only extracts multisig actor states (but not the raw state)
	ActorStatesVerifreg     = "actorstatesverifreg" // task that only extracts verified registry actor states (but not the raw state)
	BlocksTask              = "blocks"              // task that extracts block data
	MessagesTask            = "messages"            // task that extracts message data
	ChainEconomicsTask      = "chaineconomics"      // task that extracts chain economics data
	MultisigApprovalsTask   = "msapprovals"         // task that extracts multisig actor approvals
	ImplicitMessageTask     = "implicitmessage"     // task that extract implicitly executed messages: cron tick and block reward.
	ChainConsensusTask      = "consensus"
)

Variables

View Source
var TableComment = map[string]string{
	BlockHeader:                    ``,
	BlockParent:                    ``,
	DrandBlockEntrie:               `DrandBlockEntrie contains Drand randomness round numbers used in each block.`,
	MinerSectorDeal:                ``,
	MinerSectorInfoV7:              `MinerSectorInfoV7 is the default model exported from the miner actor extractor. the table is returned iff the miner actor code is greater than or equal to v7. The table receives a new name since we cannot rename the miner_sector_info table, else we will break backfill.`,
	MinerSectorInfoV1_6:            `MinerSectorInfoV1_6 is exported from the miner actor iff the actor code is less than v7. The table keeps its original name since that's a requirement to support lily backfills`,
	MinerSectorPost:                ``,
	MinerPreCommitInfo:             ``,
	MinerSectorEvent:               ``,
	MinerCurrentDeadlineInfo:       ``,
	MinerFeeDebt:                   ``,
	MinerLockedFund:                ``,
	MinerInfo:                      ``,
	MarketDealProposal:             `MarketDealProposal contains all storage deal states with latest values applied to end_epoch when updates are detected on-chain.`,
	MarketDealState:                ``,
	Message:                        ``,
	BlockMessage:                   ``,
	Receipt:                        ``,
	MessageGasEconomy:              ``,
	ParsedMessage:                  ``,
	InternalMessage:                ``,
	InternalParsedMessage:          ``,
	MultisigTransaction:            ``,
	ChainPower:                     ``,
	PowerActorClaim:                ``,
	ChainReward:                    ``,
	Actor:                          `Actor on chain that were added or updated at an epoch. Associates the actor's state root CID (head) with the chain state root CID from which it decends. Includes account ID nonce and balance at each state.`,
	ActorState:                     `ActorState that were changed at an epoch. Associates actors states as single-level trees with CIDs pointing to complete state tree with the root CID (head) for that actor’s state.`,
	IdAddress:                      `IdAddress contains a mapping of ID addresses to robust addresses from the init actor’s state.`,
	GasOutputs:                     ``,
	ChainEconomics:                 ``,
	ChainConsensus:                 ``,
	MultisigApproval:               ``,
	VerifiedRegistryVerifier:       ``,
	VerifiedRegistryVerifiedClient: ``,
}
View Source
var TableFieldComments = map[string]map[string]string{
	BlockHeader: {},
	BlockParent: {},
	DrandBlockEntrie: {
		"Block": "Block is the CID of the block.",
		"Round": "Round is the round number of randomness used.",
	},
	MinerSectorDeal: {},
	MinerSectorInfoV7: {
		"SectorKeyCID": "added in specs-actors v7, will be null for all sectors and only gets set on the first ReplicaUpdate",
	},
	MinerSectorInfoV1_6:      {},
	MinerSectorPost:          {},
	MinerPreCommitInfo:       {},
	MinerSectorEvent:         {},
	MinerCurrentDeadlineInfo: {},
	MinerFeeDebt:             {},
	MinerLockedFund:          {},
	MinerInfo:                {},
	MarketDealProposal: {
		"ClientCollateral":     "The amount of FIL (in attoFIL) the client has pledged as collateral.",
		"ClientID":             "Address of the actor proposing the deal.",
		"DealID":               "Identifier for the deal.",
		"EndEpoch":             "The epoch at which this deal with end.",
		"Height":               "Epoch at which this deal proposal was added or changed.",
		"IsVerified":           "Deal is with a verified provider.",
		"Label":                "An arbitrary client chosen label to apply to the deal.",
		"PaddedPieceSize":      "The piece size in bytes with padding.",
		"PieceCID":             "CID of a sector piece. A Piece is an object that represents a whole or part of a File.",
		"ProviderCollateral":   "The amount of FIL (in attoFIL) the provider has pledged as collateral. The Provider deal collateral is only slashed when a sector is terminated before the deal expires.",
		"ProviderID":           "Address of the actor providing the services.",
		"StartEpoch":           "The epoch at which this deal with begin. Storage deal must appear in a sealed (proven) sector no later than start_epoch, otherwise it is invalid.",
		"StateRoot":            "CID of the parent state root for this deal.",
		"StoragePricePerEpoch": "The amount of FIL (in attoFIL) that will be transferred from the client to the provider every epoch this deal is active for.",
		"UnpaddedPieceSize":    "The piece size in bytes without padding.",
	},
	MarketDealState:       {},
	Message:               {},
	BlockMessage:          {},
	Receipt:               {},
	MessageGasEconomy:     {},
	ParsedMessage:         {},
	InternalMessage:       {},
	InternalParsedMessage: {},
	MultisigTransaction: {
		"To": "Transaction State",
	},
	ChainPower:      {},
	PowerActorClaim: {},
	ChainReward:     {},
	Actor: {
		"Balance":   "Balance of Actor in attoFIL.",
		"Code":      "Human-readable identifier for the type of the actor.",
		"Head":      "CID of the root of the state tree for the actor.",
		"Height":    "Epoch when this actor was created or updated.",
		"ID":        "ID Actor address.",
		"Nonce":     "The next Actor nonce that is expected to appear on chain.",
		"StateRoot": "CID of the state root when this actor was created or changed.",
	},
	ActorState: {
		"Code":   "CID identifier for the type of the actor.",
		"Head":   "CID of the root of the state tree for the actor.",
		"Height": "Epoch when this actor was created or updated.",
		"State":  "Top level of state data as json.",
	},
	IdAddress: {
		"Address":   "Robust address",
		"Height":    "Epoch when this address mapping was created or updated.",
		"ID":        "ID address",
		"StateRoot": "StateRoot when this address mapping was created or updated.",
	},
	GasOutputs:                     {},
	ChainEconomics:                 {},
	ChainConsensus:                 {},
	MultisigApproval:               {},
	VerifiedRegistryVerifier:       {},
	VerifiedRegistryVerifiedClient: {},
}

Functions

This section is empty.

Types

type Exporter

type Exporter interface {
	ExportResult(ctx context.Context, strg model.Storage, height int64, m []*ModelResult) error
}

type Indexer

type Indexer interface {
	TipSet(ctx context.Context, ts *types.TipSet) (chan *Result, chan error, error)
}

type Manager

type Manager struct {
	// contains filtered or unexported fields
}

Manager manages the execution of an Indexer. It may be used to index TipSets both serially or in parallel.

func NewManager

func NewManager(api tasks.DataSource, strg model.Storage, name string, tasks []string, opts ...ManagerOpt) (*Manager, error)

NewManager returns a default Manager. Any provided ManagerOpt's will override Manager's default values.

func (*Manager) TipSet

func (i *Manager) TipSet(ctx context.Context, ts *types.TipSet) (bool, error)

TipSet synchronously indexes and persists `ts`. TipSet returns an error if the Manager's Indexer encounters a fatal error. TipSet returns false if one or more of the Indexer's tasks complete with a status `ERROR` or `SKIPPED`, else returns true. Upon cancellation of `ctx` TipSet will persist all incomplete tasks with status `SKIPPED` before returning.

type ManagerOpt

type ManagerOpt func(i *Manager)

func WithExporter

func WithExporter(e Exporter) ManagerOpt

WithExporter overrides the Manager's default Exporter with the provided Exporter. An Exporter is used to export the results of the Manager's Indexer.

func WithIndexer

func WithIndexer(i Indexer) ManagerOpt

WithIndexer overrides the Manager's default Indexer with the provided Indexer. An Indexer is used to collect state from a tipset.

func WithWindow

func WithWindow(w time.Duration) ManagerOpt

WithWindow overrides the Manager's default (0) window with the provided window. The value of the window controls a timeout after which the Manager aborts processing the tipset, marking any incomplete tasks as SKIPPED.

type ModelExporter

type ModelExporter struct {
	// contains filtered or unexported fields
}

func NewModelExporter

func NewModelExporter(name string) *ModelExporter

func (*ModelExporter) ExportResult

func (me *ModelExporter) ExportResult(ctx context.Context, strg model.Storage, height int64, results []*ModelResult) error

ExportResult persists []ModelResult `results` to model.Storage `strg`. An error is returned if persisting the model fails. This method will block if models at `height` are being persisted allowing the following constraints to be met: - if data with height N and SR1 is being persisted and a request to persist data with the same values is made, allow it - if data with height N and SR2 is being persisted and a request to persist data with height N and SR1 is made, block

type ModelResult

type ModelResult struct {
	Name  string
	Model model.Persistable
}

type Result

type Result struct {
	// Name of the task executed.
	Name string
	// Data extracted during task execution.
	Data model.Persistable
	// Report containing details of task execution success and duration.
	Report visormodel.ProcessingReportList
}

type TipSetIndexer

type TipSetIndexer struct {
	// contains filtered or unexported fields
}

TipSetIndexer waits for tipsets and persists their block data into a database.

func NewTipSetIndexer

func NewTipSetIndexer(node tasks.DataSource, name string, tasks []string, options ...TipSetIndexerOpt) (*TipSetIndexer, error)

NewTipSetIndexer extracts block, message and actor state data from a tipset and persists it to storage. Extraction and persistence are concurrent. Extraction of the a tipset can proceed while data from the previous extraction is being persisted. The indexer may be given a time window in which to complete data extraction. The name of the indexer is used as the reporter in the visor_processing_reports table.

func (*TipSetIndexer) TipSet

func (t *TipSetIndexer) TipSet(ctx context.Context, ts *types.TipSet) (chan *Result, chan error, error)

TipSet keeps no internal state and asynchronously indexes `ts` returning Result's as they extracted. If the TipSetIndexer encounters an error (fails to fetch ts's parent) it returns immediately and performs no work. If one of the TipSetIndexer's tasks encounters a fatal error, the error is return on the error channel.

type TipSetIndexerOpt

type TipSetIndexerOpt func(t *TipSetIndexer)

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL