syncer

README

Syncer

The Syncer package provides support for syncing blocks from any Rosetta Data API implementation. If you want to see an example of how to use this package, take a look at rosetta-cli.

Features

• Automatic handling of block re-orgs
• Multi-threaded block fetching (using the fetcher package)
• Implementable Handler to define your own block processing logic (ex: store processed blocks to a db or print our balance changes)

Installation

go get github.com/coinbase/rosetta-sdk-go/syncer

Future Work

• Sync multiple shards in a sharded blockchain

Documentation

Index

• Constants
• type Handler
• type Helper
• type Option
  • func WithConcurrency(concurrency uint64) Option
  • func WithPastBlocks(blocks []*types.BlockIdentifier) Option
• type Syncer
  • func New(network *types.NetworkIdentifier, helper Helper, handler Handler, ...) *Syncer
  • func (s *Syncer) Sync(ctx context.Context, startIndex int64, endIndex int64) error

Constants

const (
	// PastBlockSize is the maximum number of previously
	// processed blocks we keep in the syncer to handle
	// reorgs correctly. If there is a reorg greater than
	// PastBlockSize, it will not be handled correctly.
	//
	// TODO: make configurable
	PastBlockSize = 20

	// DefaultConcurrency is the default number of
	// blocks the syncer will try to get concurrently.
	DefaultConcurrency = 8
)

Types

type Handler

type Handler interface {
	BlockAdded(
		ctx context.Context,
		block *types.Block,
	) error

	BlockRemoved(
		ctx context.Context,
		block *types.BlockIdentifier,
	) error
}

Handler is called at various times during the sync cycle to handle different events. It is common to write logs or perform reconciliation in the sync processor.

type Helper

type Helper interface {
	NetworkStatus(context.Context, *types.NetworkIdentifier) (*types.NetworkStatusResponse, error)

	Block(
		context.Context,
		*types.NetworkIdentifier,
		*types.PartialBlockIdentifier,
	) (*types.Block, error)
}

Helper is called at various times during the sync cycle to get information about a blockchain network. It is common to implement this helper using the Fetcher package.

type Option

type Option func(s *Syncer)

Option is used to overwrite default values in Syncer construction. Any Option not provided falls back to the default value.

func WithConcurrency

func WithConcurrency(concurrency uint64) Option

WithConcurrency overrides the default block concurrency.

func WithPastBlocks

func WithPastBlocks(blocks []*types.BlockIdentifier) Option

WithPastBlocks provides the syncer with a cache of previously processed blocks to handle reorgs.

type Syncer

type Syncer struct {
	// contains filtered or unexported fields
}

Syncer coordinates blockchain syncing without relying on a storage interface. Instead, it calls a provided Handler whenever a block is added or removed. This provides the client the opportunity to define the logic used to handle each new block. In the rosetta-cli, we handle reconciliation, state storage, and logging in the handler.

func New

func New(
	network *types.NetworkIdentifier,
	helper Helper,
	handler Handler,
	cancel context.CancelFunc,
	options ...Option,
) *Syncer

New creates a new Syncer. If pastBlocks is left nil, it will be set to an empty slice.

func (*Syncer) Sync

func (s *Syncer) Sync(
	ctx context.Context,
	startIndex int64,
	endIndex int64,
) error

Sync cycles endlessly until there is an error or the requested range is synced.