README
¶
netsync
Overview
This package implements a concurrency safe block syncing protocol. The SyncManager communicates with connected peers to perform an initial block download, keep the chain and unconfirmed transaction pool in sync, and announce new blocks connected to the dag. Currently the sync manager selects a single sync peer that it downloads all blocks from until it is up to date with the longest chain the sync peer is aware of.
netsync
package provides a 'manager' (goroutine) that is started when soterd is run. Managers communicate with other managers via message passing over channels associated with them. For example, the mining manager can pass a newly-generated block to the netsync manager, so that this soterd node can advertise the new block to its peers.
Sync flow
The sync manager works by reacting to messages it receives, and ends up with synchronization behaviour due to how the sync manager on each node will react to each other's messages. The end result of their interaction is that the node that initiated the flow has a copy of blocks it was previously missing from its sync peer.
There are multiple ways that a nodes trigger sync flow:
- When a block received from a peer is being processed and our dag height is still below our peers', a
getblocks
message is sent for blocks ranging from our maximum dag height to the tip of our peer's dag. - When a block received from a peer is being processed and its determined that it is an orphan (we don't have their parent blocks), a
getdata
message is sent to retrieve the parents of the orphan block. - When a block received from a peer is being processed and its determined that its is a parent of an orphan, a
getblocks
message is generated from the parent's height to the hash of the orphan. - When a block is generated by a miner, a node will advertise it to its peers with an
inventory
message. If peers don't have this block, they will respond with agetdata
message.
Description of messages
Here's a quick reference for the messages involved in sync flow:
Message | Response to | Description |
---|---|---|
getblocks |
None | Asks for block inventory, for a range of blocks. Range is specified as (start block height, end block hash). A special zero-hash is used to indicate "end at tips". Responses will contain hashes and height of blocks, but no further information. |
getdata |
None | Asks for the full block data |
inventory |
getblocks |
A message containing an array of inventory. Each inventory element consists of Type (block, tx, etc), Hash , Height fields. |
block |
getdata |
Contains a full block |
Sync mode
Sync mode is where a node detects that it is behind its peer(s), and needs to concentrate on downloading blocks from a peer it selects (called a sync peer) before expanding the scope of its node communication. While in sync mode, a node will drop block-related messages from non-sync nodes.
When a node is started and gets its first sync-eligible peer, SyncManager.handleNewPeerMsg()
is called, which calls SyncManager.startSync()
. The node is also now in sync mode.
startSync()
issues a getblocks
message to the peer for blocks from its tips of dag to the tips of the peer's dag.
The sync peer receives the getblocks
message, and responds with an inventory
message containing an array of inventory. Each inventory element consists of:
Type (block, transaction, etc)
Hash
Height (currently only meaningful for block type)
The node receives the inventory
message, and processes it with SyncManager.handleInvMsg()
. handleInvMsg
does three main tasks:
- Issues
getdata
messages for blocks the node doesn't have (if not in headers-first mode) - Requests more inventory with another
getblocks
message, if the maximum height of the blocks in the inventory are higher than our current dag tips. It asks for blocks from the height of the block after the maxHeight block in the inventory to the tips of the sync peer. - Looks through orphans, and issues
getdata
messages for orphan block's parents if the node doesn't have them.
The sync peer receives the getdata
message, and and replies with block
message.
The node receives the block
message, and processes it with SyncManager.handleBlockMsg()
. handleBlockMsg
does four main tasks:
- Attempts to process the block, for inclusion into the node's dag.
- If the block is an orphan (parents missing from node's dag), it's added to the orphan pool and the node issues
getdata
requests for the parents.
- If the block is an orphan (parents missing from node's dag), it's added to the orphan pool and the node issues
- If the node isn't in headers-first mode and its tips are now current with peers, the node exits sync mode.
- If the block processed is the parent of an orphan,
getblocks
message is sent for blocks between the parent's height and the child orphans' hash. - If the node is in headers-first mode,
getblocks
message for blocks after this block's height to the tips of the peer is sent.
Installation and Updating
$ go get -u github.com/totaloutput/soterd/netsync
License
Package netsync is licensed under the copyfree ISC License.
Documentation
¶
Overview ¶
Package netsync implements a concurrency safe block syncing protocol. The SyncManager communicates with connected peers to perform an initial block download, keep the chain and unconfirmed transaction pool in sync, and announce new blocks connected to the dag. Currently the sync manager selects a single sync peer that it downloads all blocks from until it is up to date with the longest chain the sync peer is aware of.
Index ¶
- func DisableLog()
- func NewRequestExpiry() *requestExpiry
- func UseLogger(logger soterlog.Logger)
- type Config
- type PeerNotifier
- type SyncManager
- func (sm *SyncManager) DonePeer(peer *peerpkg.Peer)
- func (sm *SyncManager) IsCurrent() bool
- func (sm *SyncManager) NewPeer(peer *peerpkg.Peer)
- func (sm *SyncManager) Pause() chan<- struct{}
- func (sm *SyncManager) ProcessBlock(block *soterutil.Block, flags blockdag.BehaviorFlags) (bool, error)
- func (sm *SyncManager) QueueBlock(block *soterutil.Block, peer *peerpkg.Peer, done chan struct{})
- func (sm *SyncManager) QueueHeaders(headers *wire.MsgHeaders, peer *peerpkg.Peer)
- func (sm *SyncManager) QueueInv(inv *wire.MsgInv, peer *peerpkg.Peer)
- func (sm *SyncManager) QueueTx(tx *soterutil.Tx, peer *peerpkg.Peer, done chan struct{})
- func (sm *SyncManager) Start()
- func (sm *SyncManager) Stop() error
- func (sm *SyncManager) SyncPeerID() int32
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DisableLog ¶
func DisableLog()
DisableLog disables all library log output. Logging output is disabled by default until either UseLogger or SetLogWriter are called.
func NewRequestExpiry ¶
func NewRequestExpiry() *requestExpiry
NewRequestExpiry returns a new requestExpiry type
Types ¶
type Config ¶
type Config struct { PeerNotifier PeerNotifier Chain *blockdag.BlockDAG TxMemPool *mempool.TxPool ChainParams *chaincfg.Params // NOTE(cedric): Commented out to disable checkpoint-related code (JIRA DAG-3) // // // DisableCheckpoints bool MaxPeers int FeeEstimator *mempool.FeeEstimator }
Config is a configuration struct used to initialize a new SyncManager.
type PeerNotifier ¶
type PeerNotifier interface { AnnounceNewTransactions(newTxs []*mempool.TxDesc) UpdatePeerHeights(latestBlkHash *chainhash.Hash, latestHeight int32, updateSource *peer.Peer) RelayInventory(invVect *wire.InvVect, data interface{}) TransactionConfirmed(tx *soterutil.Tx) }
PeerNotifier exposes methods to notify peers of status changes to transactions, blocks, etc. Currently server (in the main package) implements this interface.
type SyncManager ¶
type SyncManager struct {
// contains filtered or unexported fields
}
SyncManager is used to communicate block related messages with peers. The SyncManager is started as by executing Start() in a goroutine. Once started, it selects peers to sync from and starts the initial block download. Once the chain is in sync, the SyncManager handles incoming block and header notifications and relays announcements of new blocks to peers.
func New ¶
func New(config *Config) (*SyncManager, error)
New constructs a new SyncManager. Use Start to begin processing asynchronous block, tx, and inv updates.
func (*SyncManager) DonePeer ¶
func (sm *SyncManager) DonePeer(peer *peerpkg.Peer)
DonePeer informs the blockmanager that a peer has disconnected.
func (*SyncManager) IsCurrent ¶
func (sm *SyncManager) IsCurrent() bool
IsCurrent returns whether or not the sync manager believes it is synced with the connected peers.
func (*SyncManager) NewPeer ¶
func (sm *SyncManager) NewPeer(peer *peerpkg.Peer)
NewPeer informs the sync manager of a newly active peer.
func (*SyncManager) Pause ¶
func (sm *SyncManager) Pause() chan<- struct{}
Pause pauses the sync manager until the returned channel is closed.
Note that while paused, all peer and block processing is halted. The message sender should avoid pausing the sync manager for long durations.
func (*SyncManager) ProcessBlock ¶
func (sm *SyncManager) ProcessBlock(block *soterutil.Block, flags blockdag.BehaviorFlags) (bool, error)
ProcessBlock makes use of ProcessBlock on an internal instance of a block chain.
func (*SyncManager) QueueBlock ¶
func (sm *SyncManager) QueueBlock(block *soterutil.Block, peer *peerpkg.Peer, done chan struct{})
QueueBlock adds the passed block message and peer to the block handling queue. Responds to the done channel argument after the block message is processed.
func (*SyncManager) QueueHeaders ¶
func (sm *SyncManager) QueueHeaders(headers *wire.MsgHeaders, peer *peerpkg.Peer)
QueueHeaders adds the passed headers message and peer to the block handling queue.
func (*SyncManager) QueueInv ¶
func (sm *SyncManager) QueueInv(inv *wire.MsgInv, peer *peerpkg.Peer)
QueueInv adds the passed inv message and peer to the block handling queue.
func (*SyncManager) QueueTx ¶
func (sm *SyncManager) QueueTx(tx *soterutil.Tx, peer *peerpkg.Peer, done chan struct{})
QueueTx adds the passed transaction message and peer to the block handling queue. Responds to the done channel argument after the tx message is processed.
func (*SyncManager) Start ¶
func (sm *SyncManager) Start()
Start begins the core block handler which processes block and inv messages.
func (*SyncManager) Stop ¶
func (sm *SyncManager) Stop() error
Stop gracefully shuts down the sync manager by stopping all asynchronous handlers and waiting for them to finish.
func (*SyncManager) SyncPeerID ¶
func (sm *SyncManager) SyncPeerID() int32
SyncPeerID returns the ID of the current sync peer, or 0 if there is none.