exchange

Documentation

Overview

Package exchange contains the ChainExchange server and client components.

ChainExchange is the basic chain synchronization protocol of Filecoin. ChainExchange is an RPC-oriented protocol, with a single operation to request blocks for now.

A request contains a start anchor block (referred to with a CID), and a amount of blocks requested beyond the anchor (including the anchor itself).

A client can also pass options, encoded as a 64-bit bitfield. Lotus supports two options at the moment:

• include block contents
• include block messages

The response will include a status code, an optional message, and the response payload in case of success. The payload is a slice of serialized tipsets.

Index

• Constants
• Variables
• type BSTipSet
  • func (t *BSTipSet) MarshalCBOR(w io.Writer) error
  • func (t *BSTipSet) UnmarshalCBOR(r io.Reader) (err error)
• type Client
  • func NewClient(lc fx.Lifecycle, host host.Host, pmgr peermgr.MaybePeerMgr) Client
• type CompactedMessages
  • func (t *CompactedMessages) MarshalCBOR(w io.Writer) error
  • func (t *CompactedMessages) UnmarshalCBOR(r io.Reader) (err error)
• type CompactedMessagesCBOR
  • func (t *CompactedMessagesCBOR) MarshalCBOR(w io.Writer) error
  • func (t *CompactedMessagesCBOR) UnmarshalCBOR(r io.Reader) (err error)
• type Request
  • func (t *Request) MarshalCBOR(w io.Writer) error
  • func (t *Request) UnmarshalCBOR(r io.Reader) (err error)
• type Response
  • func (t *Response) MarshalCBOR(w io.Writer) error
  • func (t *Response) UnmarshalCBOR(r io.Reader) (err error)
• type Server
  • func NewServer(cs *store.ChainStore) Server

Constants

const (
	// Extracted constants from the code.
	// FIXME: Should be reviewed and confirmed.
	SuccessPeerTagValue = 25
	WriteReqDeadline    = 5 * time.Second
	ReadResDeadline     = WriteReqDeadline
	ReadResMinSpeed     = 50 << 10
	ShufflePeersPrefix  = 16
	WriteResDeadline    = 60 * time.Second
)

const (
	Headers = 1 << iota
	Messages
)

Request options. When fetching the chain segment we can fetch either block headers, messages, or both.

const (
	Ok status = 0
	// We could not fetch all blocks requested (but at least we returned
	// the `Head` requested). Not considered an error.
	Partial = 101

	// Errors
	NotFound      = 201
	GoAway        = 202
	InternalError = 203
	BadRequest    = 204
)

const (
	// ChainExchangeProtocolID is the protocol ID of the chain exchange
	// protocol.
	ChainExchangeProtocolID = "/fil/chain/xchg/0.0.1"
)

Variables

var MaxRequestLength = uint64(policy.ChainFinality)

FIXME: Bumped from original 800 to this to accommodate `syncFork()`

use of `GetBlocks()`. It seems the expectation of that API is to
fetch any amount of blocks leaving it to the internal logic here
to partition and reassemble the requests if they go above the maximum.
(Also as a consequence of this temporarily removing the `const`
 qualifier to avoid "const initializer [...] is not a constant" error.)

Types

type BSTipSet

type BSTipSet struct {
	// List of blocks belonging to a single tipset to which the
	// `CompactedMessages` are linked.
	Blocks   []*types.BlockHeader
	Messages *CompactedMessages
}

func (*BSTipSet) MarshalCBOR

func (t *BSTipSet) MarshalCBOR(w io.Writer) error

func (*BSTipSet) UnmarshalCBOR

func (t *BSTipSet) UnmarshalCBOR(r io.Reader) (err error)

type Client

type Client interface {
	// GetBlocks fetches block headers from the network, from the provided
	// tipset *backwards*, returning as many tipsets as the count parameter.
	// The ONLY case in which we return fewer than `count` tipsets is if we hit genesis.
	GetBlocks(ctx context.Context, tsk types.TipSetKey, count int) ([]*types.TipSet, error)

	// GetChainMessages fetches messages from the network, starting from the first provided tipset
	// and returning messages from as many tipsets as requested or less.
	GetChainMessages(ctx context.Context, tipsets []*types.TipSet) ([]*CompactedMessages, error)

	// GetFullTipSet fetches a full tipset from a given peer. If successful,
	// the fetched object contains block headers and all messages in full form.
	GetFullTipSet(ctx context.Context, peer peer.ID, tsk types.TipSetKey) (*store.FullTipSet, error)

	// AddPeer adds a peer to the pool of peers that the Client requests
	// data from.
	AddPeer(peer peer.ID)

	// RemovePeer removes a peer from the pool of peers that the Client
	// requests data from.
	RemovePeer(peer peer.ID)
}

Client is the requesting side of the ChainExchange protocol. It acts as a proxy for other components to request chain data from peers. It is chiefly used by the Syncer.

func NewClient

func NewClient(lc fx.Lifecycle, host host.Host, pmgr peermgr.MaybePeerMgr) Client

NewClient creates a new libp2p-based exchange.Client that uses the libp2p ChainExhange protocol as the fetching mechanism.

type CompactedMessages

type CompactedMessages struct {
	Bls         []*types.Message
	BlsIncludes [][]uint64

	Secpk         []*types.SignedMessage
	SecpkIncludes [][]uint64
}

CompactedMessages has all messages of a single tipset compacted together instead of grouped by block to save space, since there are normally many repeated messages per tipset in different blocks.

`BlsIncludes`/`SecpkIncludes` matches `Bls`/`Secpk` messages to blocks in the tipsets with the format: `BlsIncludes[BI][MI]`

• BI: block index in the tipset.
• MI: message index in `Bls` list

FIXME: The logic to decompress this structure should belong

to itself, not to the consumer.

NOTE: Max messages is: BlockMessageLimit (10k) * MaxTipsetSize (15) = 150k

func (*CompactedMessages) MarshalCBOR

func (t *CompactedMessages) MarshalCBOR(w io.Writer) error

MarshalCBOR copies into the encoding struct, then marshals.

func (*CompactedMessages) UnmarshalCBOR

func (t *CompactedMessages) UnmarshalCBOR(r io.Reader) (err error)

UnmarshalCBOR unmarshals into the "decoding" struct, then copies into the actual struct.

type CompactedMessagesCBOR

type CompactedMessagesCBOR struct {
	Bls         []*types.Message `cborgen:"maxlen=150000"`
	BlsIncludes []messageIndices

	Secpk         []*types.SignedMessage `cborgen:"maxlen=150000"`
	SecpkIncludes []messageIndices
}

CompactedMessagesCBOR is used for encoding/decoding compacted messages. This is a ustom type as we need custom limits. - Max messages is 150,000 as that's 15 times the max block size (in messages). It needs to be large enough to cover a full tipset full of full blocks.

func (*CompactedMessagesCBOR) MarshalCBOR

func (t *CompactedMessagesCBOR) MarshalCBOR(w io.Writer) error

func (*CompactedMessagesCBOR) UnmarshalCBOR

func (t *CompactedMessagesCBOR) UnmarshalCBOR(r io.Reader) (err error)

type Request

type Request struct {
	// List of ordered CIDs comprising a `TipSetKey` from where to start
	// fetching backwards.
	// FIXME: Consider using `TipSetKey` now (introduced after the creation
	//  of this protocol) instead of converting back and forth.
	Head []cid.Cid
	// Number of block sets to fetch from `Head` (inclusive, should always
	// be in the range `[1, MaxRequestLength]`).
	Length uint64
	// Request options, see `Options` type for more details. Compressed
	// in a single `uint64` to save space.
	Options uint64
}

func (*Request) MarshalCBOR

func (t *Request) MarshalCBOR(w io.Writer) error

func (*Request) UnmarshalCBOR

func (t *Request) UnmarshalCBOR(r io.Reader) (err error)

type Response

type Response struct {
	Status status
	// String that complements the error status when converting to an
	// internal error (see `statusToError()`).
	ErrorMessage string

	Chain []*BSTipSet
}

func (*Response) MarshalCBOR

func (t *Response) MarshalCBOR(w io.Writer) error

func (*Response) UnmarshalCBOR

func (t *Response) UnmarshalCBOR(r io.Reader) (err error)

type Server

type Server interface {
	// HandleStream is the protocol handler to be registered on a libp2p
	// protocol router.
	//
	// In the current version of the protocol, streams are single-use. The
	// server will read a single Request, and will respond with a single
	// Response. It will dispose of the stream straight after.
	HandleStream(stream network.Stream)
}

Server is the responder side of the ChainExchange protocol. It accepts requests from clients and services them by returning the requested chain data.

func NewServer

func NewServer(cs *store.ChainStore) Server

NewServer creates a new libp2p-based exchange.Server. It services requests for the libp2p ChainExchange protocol.