coordinator

package
v1.7.0-rc1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 13, 2021 License: AGPL-3.0 Imports: 36 Imported by: 0

Documentation

Overview

Package coordinator handles all the logic related to forging batches as a coordinator in the hermez network.

The forging of batches is done with a pipeline in order to allow multiple batches being forged in parallel. The maximum number of batches that can be forged in parallel is determined by the number of available proof servers.

The Coordinator begins with the pipeline stopped. The main Coordinator goroutine keeps listening for synchronizer events sent by the node package, which allow the coordinator to determine if the configured forger address is allowed to forge at the current block or not. When the forger address becomes allowed to forge, the pipeline is started, and when it terminates being allowed to forge, the pipeline is stopped.

The Pipeline consists of two goroutines. The first one is in charge of preparing a batch internally, which involves making a selection of transactions and calculating the ZKInputs for the batch proof, and sending these ZKInputs to an idle proof server. This goroutine will keep preparing batches while there are idle proof servers, if the forging policy determines that a batch should be forged in the current state. The second goroutine is in charge of waiting for the proof server to finish computing the proof, retreiving it, prepare the arguments for the `forgeBatch` Rollup transaction, and sending the result to the TxManager. All the batch information moves between functions and goroutines via the BatchInfo struct.

Finally, the TxManager contains a single goroutine that makes forgeBatch ethereum transactions for the batches sent by the Pipeline, and keeps them in a list to check them periodically. In the periodic checks, the ethereum transaction is checked for successfulness, and it's only forgotten after a number of confirmation blocks have passed after being successfully mined. At any point if a transaction failure is detected, the TxManager can signal the Coordinator to reset the Pipeline in order to reforge the failed batches.

The Coordinator goroutine acts as a manager. The synchronizer events (which notify about new blocks and associated new state) that it receives are broadcasted to the Pipeline and the TxManager. This allows the Coordinator, Pipeline and TxManager to have a copy of the current hermez network state required to perform their duties.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BatchInfo 

type BatchInfo struct {
	PipelineNum           int
	BatchNum              common.BatchNum
	ServerProof           prover.Client
	ProofStart            time.Time
	ZKInputs              *common.ZKInputs
	Proof                 *prover.Proof
	PublicInputs          []*big.Int
	L1Batch               bool
	VerifierIdx           uint8
	L1UserTxs             []common.L1Tx
	L1CoordTxs            []common.L1Tx
	L1CoordinatorTxsAuths [][]byte
	L2Txs                 []common.L2Tx
	CoordIdxs             []common.Idx
	ForgeBatchArgs        *eth.RollupForgeBatchArgs
	Auth                  *bind.TransactOpts `json:"-"`
	EthTxs                []*types.Transaction
	EthTxsErrs            []error
	// SendTimestamp  the time of batch sent to ethereum
	SendTimestamp time.Time
	Receipt       *types.Receipt
	// Fail is true if:
	// - The receipt status is failed
	// - A previous parent batch is failed
	Fail  bool
	Debug Debug
}

BatchInfo contans the Batch information

func (*BatchInfo) DebugStore 

func (b *BatchInfo) DebugStore(storePath string) error

DebugStore is a debug function to store the BatchInfo as a json text file in storePath. The filename contains the batchNumber followed by a timestamp of batch start.

type Config 

type Config struct {
	// ForgerAddress is the address under which this coordinator is forging
	ForgerAddress ethCommon.Address
	// ConfirmBlocks is the number of confirmation blocks to wait for sent
	// ethereum transactions before forgetting about them
	ConfirmBlocks int64
	// L1BatchTimeoutPerc is the portion of the range before the L1Batch
	// timeout that will trigger a schedule to forge an L1Batch
	L1BatchTimeoutPerc float64
	// StartSlotBlocksDelay is the number of blocks of delay to wait before
	// starting the pipeline when we reach a slot in which we can forge.
	StartSlotBlocksDelay int64
	// ScheduleBatchBlocksAheadCheck is the number of blocks ahead in which
	// the forger address is checked to be allowed to forge (apart from
	// checking the next block), used to decide when to stop scheduling new
	// batches (by stopping the pipeline).
	// For example, if we are at block 10 and ScheduleBatchBlocksAheadCheck
	// is 5, even though at block 11 we canForge, the pipeline will be
	// stopped if we can't forge at block 15.
	// This value should be the expected number of blocks it takes between
	// scheduling a batch and having it mined.
	ScheduleBatchBlocksAheadCheck int64
	// SendBatchBlocksMarginCheck is the number of margin blocks ahead in
	// which the coordinator is also checked to be allowed to forge, apart
	// from the next block; used to decide when to stop sending batches to
	// the smart contract.
	// For example, if we are at block 10 and SendBatchBlocksMarginCheck is
	// 5, even though at block 11 we canForge, the batch will be discarded
	// if we can't forge at block 15.
	// This value should be the expected number of blocks it takes between
	// sending a batch and having it mined.
	SendBatchBlocksMarginCheck int64
	// EthClientAttempts is the number of attempts to do an eth client RPC
	// call before giving up
	EthClientAttempts int
	// ForgeRetryInterval is the waiting interval between calls forge a
	// batch after an error
	ForgeRetryInterval time.Duration
	// ForgeDelay is the delay after which a batch is forged if the slot is
	// already committed.  If set to 0s, the coordinator will continuously
	// forge at the maximum rate.
	ForgeDelay time.Duration
	// ForgeNoTxsDelay is the delay after which a batch is forged even if
	// there are no txs to forge if the slot is already committed.  If set
	// to 0s, the coordinator will continuously forge even if the batches
	// are empty.
	ForgeNoTxsDelay time.Duration
	// MustForgeAtSlotDeadline enables the coordinator to forge slots if
	// the empty slots reach the slot deadline.
	MustForgeAtSlotDeadline bool
	// IgnoreSlotCommitment disables forcing the coordinator to forge a
	// slot immediately when the slot is not committed. If set to false,
	// the coordinator will immediately forge a batch at the beginning of
	// a slot if it's the slot winner.
	IgnoreSlotCommitment bool
	// ForgeOncePerSlotIfTxs will make the coordinator forge at most one
	// batch per slot, only if there are included txs in that batch, or
	// pending l1UserTxs in the smart contract.  Setting this parameter
	// overrides `ForgeDelay`, `ForgeNoTxsDelay`, `MustForgeAtSlotDeadline`
	// and `IgnoreSlotCommitment`.
	ForgeOncePerSlotIfTxs bool
	// SyncRetryInterval is the waiting interval between calls to the main
	// handler of a synced block after an error
	SyncRetryInterval time.Duration
	// PurgeByExtDelInterval is the waiting interval between calls
	// to the PurgeByExternalDelete function of the l2db which deletes
	// pending txs externally marked by the column `external_delete`
	PurgeByExtDelInterval time.Duration
	// EthClientAttemptsDelay is delay between attempts do do an eth client
	// RPC call
	EthClientAttemptsDelay time.Duration
	// EthTxResendTimeout is the timeout after which a non-mined ethereum
	// transaction will be resent (reusing the nonce) with a newly
	// calculated gas price
	EthTxResendTimeout time.Duration
	// EthNoReuseNonce disables reusing nonces of pending transactions for
	// new replacement transactions
	EthNoReuseNonce bool
	// MaxGasPrice is the maximum gas price in gwei allowed for ethereum
	// transactions
	MaxGasPrice int64
	// MinGasPrice is the minimum gas price in gwei allowed for ethereum
	MinGasPrice int64
	// GasPriceIncPerc is the percentage increase of gas price set in an
	// ethereum transaction from the suggested gas price by the ehtereum
	// node
	GasPriceIncPerc int64
	// TxManagerCheckInterval is the waiting interval between receipt
	// checks of ethereum transactions in the TxManager
	TxManagerCheckInterval time.Duration
	// DebugBatchPath if set, specifies the path where batchInfo is stored
	// in JSON in every step/update of the pipeline
	DebugBatchPath string
	Purger         PurgerCfg
	// VerifierIdx is the index of the verifier contract registered in the
	// smart contract
	VerifierIdx uint8
	// ForgeBatchGasCost contains the cost of each action in the
	// ForgeBatch transaction.
	ForgeBatchGasCost config.ForgeBatchGasCost
	TxProcessorConfig txprocessor.Config
}

Config contains the Coordinator configuration

type Coordinator 

type Coordinator struct {
	// contains filtered or unexported fields
}

Coordinator implements the Coordinator type

func NewCoordinator 

func NewCoordinator(cfg Config,
	historyDB *historydb.HistoryDB,
	l2DB *l2db.L2DB,
	txSelector *txselector.TxSelector,
	batchBuilder *batchbuilder.BatchBuilder,
	serverProofs []prover.Client,
	ethClient eth.ClientInterface,
	scConsts *common.SCConsts,
	initSCVars *common.SCVariables,
	etherscanService *etherscan.Service,
) (*Coordinator, error)

NewCoordinator creates a new Coordinator

func (*Coordinator) BatchBuilder 

func (c *Coordinator) BatchBuilder() *batchbuilder.BatchBuilder

BatchBuilder returns the inner BatchBuilder

func (*Coordinator) SendMsg 

func (c *Coordinator) SendMsg(ctx context.Context, msg interface{})

SendMsg is a thread safe method to pass a message to the Coordinator

func (*Coordinator) Start 

func (c *Coordinator) Start()

Start the coordinator

func (*Coordinator) Stop 

func (c *Coordinator) Stop()

Stop the coordinator

func (*Coordinator) TxSelector 

func (c *Coordinator) TxSelector() *txselector.TxSelector

TxSelector returns the inner TxSelector

type Debug 

type Debug struct {
	// StartTimestamp of is the time of batch start
	StartTimestamp time.Time
	// SendTimestamp  the time of batch sent to ethereum
	SendTimestamp time.Time
	// Status of the Batch
	Status Status
	// StartBlockNum is the blockNum when the Batch was started
	StartBlockNum int64
	// MineBlockNum is the blockNum in which the batch was mined
	MineBlockNum int64
	// SendBlockNum is the blockNum when the batch was sent to ethereum
	SendBlockNum int64
	// ResendNum is the number of times the tx has been resent
	ResendNum int
	// LastScheduledL1BatchBlockNum is the blockNum when the last L1Batch
	// was scheduled
	LastScheduledL1BatchBlockNum int64
	// LastL1BatchBlock is the blockNum in which the last L1Batch was
	// synced
	LastL1BatchBlock int64
	// LastL1BatchBlockDelta is the number of blocks after the last L1Batch
	LastL1BatchBlockDelta int64
	// L1BatchBlockScheduleDeadline is the number of blocks after the last
	// L1Batch after which an L1Batch will be scheduled
	L1BatchBlockScheduleDeadline int64
	// StartToMineBlocksDelay is the number of blocks that happen between
	// scheduling a batch and having it mined
	StartToMineBlocksDelay int64
	// StartToSendDelay is the delay between starting a batch and sending
	// it to ethereum, in seconds
	StartToSendDelay float64
	// StartToMineDelay is the delay between starting a batch and having
	// it mined in seconds
	StartToMineDelay float64
	// SendToMineDelay is the delay between sending a batch tx and having
	// it mined in seconds
	SendToMineDelay float64
}

Debug information related to the Batch

type MsgStopPipeline 

type MsgStopPipeline struct {
	Reason string
	// FailedBatchNum indicates the first batchNum that failed in the
	// pipeline.  If FailedBatchNum is 0, it should be ignored.
	FailedBatchNum common.BatchNum
}

MsgStopPipeline indicates a signal to reset the pipeline

type MsgSyncBlock 

type MsgSyncBlock struct {
	Stats   synchronizer.Stats
	Batches []common.BatchData
	// Vars contains each Smart Contract variables if they are updated, or
	// nil if they haven't changed.
	Vars common.SCVariablesPtr
}

MsgSyncBlock indicates an update to the Synchronizer stats

type MsgSyncReorg 

type MsgSyncReorg struct {
	Stats synchronizer.Stats
	Vars  common.SCVariablesPtr
}

MsgSyncReorg indicates a reorg

type Pipeline 

type Pipeline struct {
	// contains filtered or unexported fields
}

Pipeline manages the forging of batches with parallel server proofs

func NewPipeline 

func NewPipeline(ctx context.Context,
	cfg Config,
	num int,
	historyDB *historydb.HistoryDB,
	l2DB *l2db.L2DB,
	txSelector *txselector.TxSelector,
	batchBuilder *batchbuilder.BatchBuilder,
	mutexL2DBUpdateDelete *sync.Mutex,
	purger *Purger,
	coord *Coordinator,
	txManager *TxManager,
	provers []prover.Client,
	scConsts *common.SCConsts,
) (*Pipeline, error)

NewPipeline creates a new Pipeline

func (*Pipeline) SetSyncStatsVars 

func (p *Pipeline) SetSyncStatsVars(ctx context.Context, stats *synchronizer.Stats,
	vars *common.SCVariablesPtr)

SetSyncStatsVars is a thread safe method to sets the synchronizer Stats

func (*Pipeline) Start 

func (p *Pipeline) Start(batchNum common.BatchNum,
	stats *synchronizer.Stats, vars *common.SCVariables) error

Start the forging pipeline

func (*Pipeline) Stop 

func (p *Pipeline) Stop(ctx context.Context)

Stop the forging pipeline

type ProversPool 

type ProversPool struct {
	// contains filtered or unexported fields
}

ProversPool contains the multiple prover clients

func NewProversPool 

func NewProversPool(maxServerProofs int) *ProversPool

NewProversPool creates a new pool of provers.

func (*ProversPool) Add 

func (p *ProversPool) Add(ctx context.Context, serverProof prover.Client)

Add a prover to the pool

func (*ProversPool) Get 

func (p *ProversPool) Get(ctx context.Context) (prover.Client, error)

Get returns the next available prover

type Purger 

type Purger struct {
	// contains filtered or unexported fields
}

Purger manages cleanup of transactions in the pool

func (*Purger) CanInvalidate 

func (p *Purger) CanInvalidate(blockNum, batchNum int64) bool

CanInvalidate returns true if it's a good time to invalidate according to the configuration

func (*Purger) CanPurge 

func (p *Purger) CanPurge(blockNum, batchNum int64) bool

CanPurge returns true if it's a good time to purge according to the configuration

func (*Purger) InvalidateMaybe 

func (p *Purger) InvalidateMaybe(l2DB *l2db.L2DB, stateDB *statedb.LocalStateDB,
	blockNum, batchNum int64) (bool, error)

InvalidateMaybe invalidates txs if it's a good time to do so

func (*Purger) PurgeMaybe 

func (p *Purger) PurgeMaybe(l2DB *l2db.L2DB, blockNum, batchNum int64) (bool, error)

PurgeMaybe purges txs if it's a good time to do so

type PurgerCfg 

type PurgerCfg struct {
	// PurgeBatchDelay is the delay between batches to purge outdated
	// transactions. Outdated L2Txs are those that have been forged or
	// marked as invalid for longer than the SafetyPeriod and pending L2Txs
	// that have been in the pool for longer than TTL once there are
	// MaxTxs.
	PurgeBatchDelay int64
	// InvalidateBatchDelay is the delay between batches to mark invalid
	// transactions due to nonce lower than the account nonce.
	InvalidateBatchDelay int64
	// PurgeBlockDelay is the delay between blocks to purge outdated
	// transactions. Outdated L2Txs are those that have been forged or
	// marked as invalid for longer than the SafetyPeriod and pending L2Txs
	// that have been in the pool for longer than TTL once there are
	// MaxTxs.
	PurgeBlockDelay int64
	// InvalidateBlockDelay is the delay between blocks to mark invalid
	// transactions due to nonce lower than the account nonce.
	InvalidateBlockDelay int64
}

PurgerCfg is the purger configuration

type Queue 

type Queue struct {
	// contains filtered or unexported fields
}

Queue of BatchInfos

func NewQueue 

func NewQueue() Queue

NewQueue returns a new queue

func (*Queue) At 

func (q *Queue) At(position int) *BatchInfo

At returns the BatchInfo at position (or nil if position is out of bounds)

func (*Queue) Len 

func (q *Queue) Len() int

Len is the length of the queue

func (*Queue) Next 

func (q *Queue) Next() (int, *BatchInfo)

Next returns the next BatchInfo (or nil if queue is empty)

func (*Queue) Push 

func (q *Queue) Push(batchInfo *BatchInfo)

Push adds a new BatchInfo

func (*Queue) Remove 

func (q *Queue) Remove(position int)

Remove removes the BatchInfo at position

type Status 

type Status string

Status is used to mark the status of the batch 

const (
	// StatusPending marks the Tx as Pending
	StatusPending Status = "pending"
	// StatusForged marks the batch as forged internally
	StatusForged Status = "forged"
	// StatusProof marks the batch as proof calculated
	StatusProof Status = "proof"
	// StatusSent marks the EthTx as Sent
	StatusSent Status = "sent"
	// StatusMined marks the EthTx as Mined
	StatusMined Status = "mined"
	// StatusFailed marks the EthTx as Failed
	StatusFailed Status = "failed"
)

type TxManager 

type TxManager struct {
	// contains filtered or unexported fields
}

TxManager handles everything related to ethereum transactions: It makes the call to forge, waits for transaction confirmation, and keeps checking them until a number of confirmed blocks have passed.

func NewTxManager 

func NewTxManager(ctx context.Context, cfg *Config, ethClient eth.ClientInterface, l2DB *l2db.L2DB,
	coord *Coordinator, scConsts *common.SCConsts, initSCVars *common.SCVariables, etherscanService *etherscan.Service) (
	*TxManager, error)

NewTxManager creates a new TxManager

func (*TxManager) AddBatch 

func (t *TxManager) AddBatch(ctx context.Context, batchInfo *BatchInfo)

AddBatch is a thread safe method to pass a new batch TxManager to be sent to the smart contract via the forge call

func (*TxManager) DiscardPipeline 

func (t *TxManager) DiscardPipeline(ctx context.Context, pipelineNum int)

DiscardPipeline is a thread safe method to notify about a discarded pipeline due to a reorg

func (*TxManager) NewAuth 

func (t *TxManager) NewAuth(ctx context.Context, batchInfo *BatchInfo) (*bind.TransactOpts, error)

NewAuth generates a new auth object for an ethereum transaction

func (*TxManager) Run 

func (t *TxManager) Run(ctx context.Context)

Run the TxManager

func (*TxManager) SetSyncStatsVars 

func (t *TxManager) SetSyncStatsVars(ctx context.Context, stats *synchronizer.Stats,
	vars *common.SCVariablesPtr)

SetSyncStatsVars is a thread safe method to sets the synchronizer Stats

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.