netsync

README

netsync

Overview

This package implements a concurrency safe block syncing protocol. The SyncManager communicates with connected peers to perform an initial block download, keep the chain and unconfirmed transaction pool in sync, and announce new blocks connected to the dag. Currently the sync manager selects a single sync peer that it downloads all blocks from until it is up to date with the longest chain the sync peer is aware of.

netsync package provides a 'manager' (goroutine) that is started when soterd is run. Managers communicate with other managers via message passing over channels associated with them. For example, the mining manager can pass a newly-generated block to the netsync manager, so that this soterd node can advertise the new block to its peers.

Sync flow

The sync manager works by reacting to messages it receives, and ends up with synchronization behaviour due to how the sync manager on each node will react to each other's messages. The end result of their interaction is that the node that initiated the flow has a copy of blocks it was previously missing from its sync peer.

There are multiple ways that a nodes trigger sync flow:

1. When a block received from a peer is being processed and our dag height is still below our peers', a getblocks message is sent for blocks ranging from our maximum dag height to the tip of our peer's dag.
2. When a block received from a peer is being processed and its determined that it is an orphan (we don't have their parent blocks), a getdata message is sent to retrieve the parents of the orphan block.
3. When a block received from a peer is being processed and its determined that its is a parent of an orphan, a getblocks message is generated from the parent's height to the hash of the orphan.
4. When a block is generated by a miner, a node will advertise it to its peers with an inventory message. If peers don't have this block, they will respond with a getdata message.

Description of messages

Here's a quick reference for the messages involved in sync flow:

Message	Response to	Description
getblocks	None	Asks for block inventory, for a range of blocks. Range is specified as (start block height, end block hash). A special zero-hash is used to indicate "end at tips". Responses will contain hashes and height of blocks, but no further information.
getdata	None	Asks for the full block data
inventory	getblocks	A message containing an array of inventory. Each inventory element consists of Type (block, tx, etc), Hash, Height fields.
block	getdata	Contains a full block

Sync mode

Sync mode is where a node detects that it is behind its peer(s), and needs to concentrate on downloading blocks from a peer it selects (called a sync peer) before expanding the scope of its node communication. While in sync mode, a node will drop block-related messages from non-sync nodes.

When a node is started and gets its first sync-eligible peer, SyncManager.handleNewPeerMsg() is called, which calls SyncManager.startSync(). The node is also now in sync mode.

startSync() issues a getblocks message to the peer for blocks from its tips of dag to the tips of the peer's dag.

The sync peer receives the getblocks message, and responds with an inventory message containing an array of inventory. Each inventory element consists of:

Type (block, transaction, etc)
Hash
Height (currently only meaningful for block type)

The node receives the inventory message, and processes it with SyncManager.handleInvMsg(). handleInvMsg does three main tasks:

1. Issues getdata messages for blocks the node doesn't have (if not in headers-first mode)
2. Requests more inventory with another getblocks message, if the maximum height of the blocks in the inventory are higher than our current dag tips. It asks for blocks from the height of the block after the maxHeight block in the inventory to the tips of the sync peer.
3. Looks through orphans, and issues getdata messages for orphan block's parents if the node doesn't have them.

The sync peer receives the getdata message, and and replies with block message.

The node receives the block message, and processes it with SyncManager.handleBlockMsg(). handleBlockMsg does four main tasks:

1. Attempts to process the block, for inclusion into the node's dag.
   • If the block is an orphan (parents missing from node's dag), it's added to the orphan pool and the node issues getdata requests for the parents.
2. If the node isn't in headers-first mode and its tips are now current with peers, the node exits sync mode.
3. If the block processed is the parent of an orphan, getblocks message is sent for blocks between the parent's height and the child orphans' hash.
4. If the node is in headers-first mode, getblocks message for blocks after this block's height to the tips of the peer is sent.

Installation and Updating

$ go get -u github.com/totaloutput/soterd/netsync

License

Package netsync is licensed under the copyfree ISC License.

Documentation

Overview

Package netsync implements a concurrency safe block syncing protocol. The SyncManager communicates with connected peers to perform an initial block download, keep the chain and unconfirmed transaction pool in sync, and announce new blocks connected to the dag. Currently the sync manager selects a single sync peer that it downloads all blocks from until it is up to date with the longest chain the sync peer is aware of.

Index

• func DisableLog()
• func NewRequestExpiry() *requestExpiry
• func UseLogger(logger soterlog.Logger)
• type Config
• type PeerNotifier
• type SyncManager
  • func New(config *Config) (*SyncManager, error)
  • func (sm *SyncManager) DonePeer(peer *peerpkg.Peer)
  • func (sm *SyncManager) IsCurrent() bool
  • func (sm *SyncManager) NewPeer(peer *peerpkg.Peer)
  • func (sm *SyncManager) Pause() chan<- struct{}
  • func (sm *SyncManager) ProcessBlock(block *soterutil.Block, flags blockdag.BehaviorFlags) (bool, error)
  • func (sm *SyncManager) QueueBlock(block *soterutil.Block, peer *peerpkg.Peer, done chan struct{})
  • func (sm *SyncManager) QueueHeaders(headers *wire.MsgHeaders, peer *peerpkg.Peer)
  • func (sm *SyncManager) QueueInv(inv *wire.MsgInv, peer *peerpkg.Peer)
  • func (sm *SyncManager) QueueTx(tx *soterutil.Tx, peer *peerpkg.Peer, done chan struct{})
  • func (sm *SyncManager) Start()
  • func (sm *SyncManager) Stop() error
  • func (sm *SyncManager) SyncPeerID() int32

Functions

func DisableLog

func DisableLog()

DisableLog disables all library log output. Logging output is disabled by default until either UseLogger or SetLogWriter are called.

func NewRequestExpiry

func NewRequestExpiry() *requestExpiry

NewRequestExpiry returns a new requestExpiry type

func UseLogger

func UseLogger(logger soterlog.Logger)

UseLogger uses a specified Logger to output package logging info. This should be used in preference to SetLogWriter if the caller is also using soterlog.

Types

type Config

type Config struct {
	PeerNotifier PeerNotifier
	Chain        *blockdag.BlockDAG
	TxMemPool    *mempool.TxPool
	ChainParams  *chaincfg.Params

	// NOTE(cedric): Commented out to disable checkpoint-related code (JIRA DAG-3)
	//
	//
	// DisableCheckpoints bool
	MaxPeers int

	FeeEstimator *mempool.FeeEstimator
}

Config is a configuration struct used to initialize a new SyncManager.

type PeerNotifier

type PeerNotifier interface {
	AnnounceNewTransactions(newTxs []*mempool.TxDesc)

	UpdatePeerHeights(latestBlkHash *chainhash.Hash, latestHeight int32, updateSource *peer.Peer)

	RelayInventory(invVect *wire.InvVect, data interface{})

	TransactionConfirmed(tx *soterutil.Tx)
}

PeerNotifier exposes methods to notify peers of status changes to transactions, blocks, etc. Currently server (in the main package) implements this interface.

type SyncManager

type SyncManager struct {
	// contains filtered or unexported fields
}

SyncManager is used to communicate block related messages with peers. The SyncManager is started as by executing Start() in a goroutine. Once started, it selects peers to sync from and starts the initial block download. Once the chain is in sync, the SyncManager handles incoming block and header notifications and relays announcements of new blocks to peers.

func New

func New(config *Config) (*SyncManager, error)

New constructs a new SyncManager. Use Start to begin processing asynchronous block, tx, and inv updates.

func (*SyncManager) DonePeer

func (sm *SyncManager) DonePeer(peer *peerpkg.Peer)

DonePeer informs the blockmanager that a peer has disconnected.

func (*SyncManager) IsCurrent

func (sm *SyncManager) IsCurrent() bool

IsCurrent returns whether or not the sync manager believes it is synced with the connected peers.

func (*SyncManager) NewPeer

func (sm *SyncManager) NewPeer(peer *peerpkg.Peer)

NewPeer informs the sync manager of a newly active peer.

func (*SyncManager) Pause

func (sm *SyncManager) Pause() chan<- struct{}

Pause pauses the sync manager until the returned channel is closed.

Note that while paused, all peer and block processing is halted. The message sender should avoid pausing the sync manager for long durations.

func (*SyncManager) ProcessBlock

func (sm *SyncManager) ProcessBlock(block *soterutil.Block, flags blockdag.BehaviorFlags) (bool, error)

ProcessBlock makes use of ProcessBlock on an internal instance of a block chain.

func (*SyncManager) QueueBlock

func (sm *SyncManager) QueueBlock(block *soterutil.Block, peer *peerpkg.Peer, done chan struct{})

QueueBlock adds the passed block message and peer to the block handling queue. Responds to the done channel argument after the block message is processed.

func (*SyncManager) QueueHeaders

func (sm *SyncManager) QueueHeaders(headers *wire.MsgHeaders, peer *peerpkg.Peer)

QueueHeaders adds the passed headers message and peer to the block handling queue.

func (*SyncManager) QueueInv

func (sm *SyncManager) QueueInv(inv *wire.MsgInv, peer *peerpkg.Peer)

QueueInv adds the passed inv message and peer to the block handling queue.

func (*SyncManager) QueueTx

func (sm *SyncManager) QueueTx(tx *soterutil.Tx, peer *peerpkg.Peer, done chan struct{})

QueueTx adds the passed transaction message and peer to the block handling queue. Responds to the done channel argument after the tx message is processed.

func (*SyncManager) Start

func (sm *SyncManager) Start()

Start begins the core block handler which processes block and inv messages.

func (*SyncManager) Stop

func (sm *SyncManager) Stop() error

Stop gracefully shuts down the sync manager by stopping all asynchronous handlers and waiting for them to finish.

func (*SyncManager) SyncPeerID

func (sm *SyncManager) SyncPeerID() int32

SyncPeerID returns the ID of the current sync peer, or 0 if there is none.