eds

Documentation

Index

• Variables
• func ODSReader(carReader io.Reader) (io.Reader, error)
• func ReadEDS(ctx context.Context, r io.Reader, root share.DataHash) (*rsmt2d.ExtendedDataSquare, error)
• func WriteEDS(ctx context.Context, eds *rsmt2d.ExtendedDataSquare, w io.Writer) error
• type PubSub
  • func NewPubSub(ctx context.Context, h host.Host, suffix string) (*PubSub, error)
  • func (s *PubSub) AddValidator(validate Validator) error
  • func (s *PubSub) Broadcast(ctx context.Context, data share.DataHash) error
  • func (s *PubSub) Start(context.Context) error
  • func (s *PubSub) Stop(context.Context) error
  • func (s *PubSub) Subscribe() (*Subscription, error)
• type Retriever
  • func NewRetriever(bServ blockservice.BlockService) *Retriever
  • func (r *Retriever) Retrieve(ctx context.Context, dah *da.DataAvailabilityHeader) (*rsmt2d.ExtendedDataSquare, error)
• type Store
  • func NewStore(basepath string, ds datastore.Batching) (*Store, error)
  • func (s *Store) Blockstore() bstore.Blockstore
  • func (s *Store) CARBlockstore(ctx context.Context, dataHash []byte) (dagstore.ReadBlockstore, error)
  • func (s *Store) Get(ctx context.Context, root share.DataHash) (*rsmt2d.ExtendedDataSquare, error)
  • func (s *Store) GetCAR(ctx context.Context, root share.DataHash) (io.ReadCloser, error)
  • func (s *Store) Has(ctx context.Context, root share.DataHash) (bool, error)
  • func (s *Store) Put(ctx context.Context, root share.DataHash, square *rsmt2d.ExtendedDataSquare) error
  • func (s *Store) Remove(ctx context.Context, root share.DataHash) error
  • func (s *Store) Start(context.Context) error
  • func (s *Store) Stop(context.Context) error
• type Subscription
  • func (subs *Subscription) Cancel()
  • func (subs *Subscription) Next(ctx context.Context) (share.DataHash, error)
• type Validator

Variables

var ErrEmptySquare = errors.New("share: importing empty data")

var RetrieveQuadrantTimeout = blockTime / numQuadrants * 2

RetrieveQuadrantTimeout defines how much time Retriever waits before starting to retrieve another quadrant.

NOTE: - The whole data square must be retrieved in less than block time. - We have 4 quadrants from two sources(rows, cols) which equals to 8 in total.

Functions

func ODSReader

func ODSReader(carReader io.Reader) (io.Reader, error)

ODSReader reads CARv1 encoded data from io.ReadCloser and limits the reader to the CAR header and first quadrant (ODS)

func ReadEDS

func ReadEDS(ctx context.Context, r io.Reader, root share.DataHash) (*rsmt2d.ExtendedDataSquare, error)

ReadEDS reads the first EDS quadrant (1/4) from an io.Reader CAR file. Only the first quadrant will be read, which represents the original data. The returned EDS is guaranteed to be full and valid against the DataRoot, otherwise ReadEDS errors.

func WriteEDS

func WriteEDS(ctx context.Context, eds *rsmt2d.ExtendedDataSquare, w io.Writer) error

WriteEDS writes the entire EDS into the given io.Writer as CARv1 file. This includes all shares in quadrant order, followed by all inner nodes of the NMT tree. Order: [ Carv1Header | Q1 | Q2 | Q3 | Q4 | inner nodes ] For more information about the header: https://ipld.io/specs/transport/car/carv1/#header

Types

type PubSub

type PubSub struct {
	// contains filtered or unexported fields
}

PubSub manages receiving and propagating the EDS from/to the network over "eds-sub" subscription.

func NewPubSub

func NewPubSub(ctx context.Context, h host.Host, suffix string) (*PubSub, error)

NewPubSub creates a libp2p.PubSub wrapper.

func (*PubSub) AddValidator

func (s *PubSub) AddValidator(validate Validator) error

AddValidator registers given Validator for EDS notifications (DataHash). Any amount of Validators can be registered.

func (*PubSub) Broadcast

func (s *PubSub) Broadcast(ctx context.Context, data share.DataHash) error

Broadcast sends the EDS notification (DataHash) to every connected peer.

func (*PubSub) Start

func (s *PubSub) Start(context.Context) error

Start creates an instances of FloodSub and joins specified topic.

func (*PubSub) Stop

func (s *PubSub) Stop(context.Context) error

Stop completely stops the PubSub: * Unregisters all the added Validators * Closes the `ShrEx/Sub` topic

func (*PubSub) Subscribe

func (s *PubSub) Subscribe() (*Subscription, error)

Subscribe provides a new Subscription for EDS notifications.

type Retriever

type Retriever struct {
	// contains filtered or unexported fields
}

Retriever retrieves rsmt2d.ExtendedDataSquares from the IPLD network. Instead of requesting data 'share by share' it requests data by quadrants minimizing bandwidth usage in the happy cases.

 ---- ----
| 0  | 1  |
 ---- ----
| 2  | 3  |
 ---- ----

Retriever randomly picks one of the data square quadrants and tries to request them one by one until it is able to reconstruct the whole square.

func NewRetriever

func NewRetriever(bServ blockservice.BlockService) *Retriever

NewRetriever creates a new instance of the Retriever over IPLD BlockService and rmst2d.Codec

func (*Retriever) Retrieve

func (r *Retriever) Retrieve(ctx context.Context, dah *da.DataAvailabilityHeader) (*rsmt2d.ExtendedDataSquare, error)

Retrieve retrieves all the data committed to DataAvailabilityHeader.

If not available locally, it aims to request from the network only one quadrant (1/4) of the data square and reconstructs the other three quadrants (3/4). If the requested quadrant is not available within RetrieveQuadrantTimeout, it starts requesting another quadrant until either the data is reconstructed, context is canceled or ErrByzantine is generated.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store maintains (via DAGStore) a top-level index enabling granular and efficient random access to every share and/or Merkle proof over every registered CARv1 file. The EDSStore provides a custom blockstore interface implementation to achieve access. The main use-case is randomized sampling over the whole chain of EDS block data and getting data by namespace.

func NewStore

func NewStore(basepath string, ds datastore.Batching) (*Store, error)

NewStore creates a new EDS Store under the given basepath and datastore.

func (*Store) Blockstore

func (s *Store) Blockstore() bstore.Blockstore

Blockstore returns an IPFS blockstore providing access to individual shares/nodes of all EDS registered on the Store. NOTE: The blockstore does not store whole Celestia Blocks but IPFS blocks. We represent `shares` and NMT Merkle proofs as IPFS blocks and IPLD nodes so Bitswap can access those.

func (*Store) CARBlockstore

func (s *Store) CARBlockstore(ctx context.Context, dataHash []byte) (dagstore.ReadBlockstore, error)

CARBlockstore returns the IPFS blockstore that provides access to the IPLD blocks stored in an individual CAR file.

func (*Store) Get

func (s *Store) Get(ctx context.Context, root share.DataHash) (*rsmt2d.ExtendedDataSquare, error)

Get reads EDS out of Store by given DataRoot.

It reads only one quadrant(1/4) of the EDS and verifies the integrity of the stored data by recomputing it.

func (*Store) GetCAR

func (s *Store) GetCAR(ctx context.Context, root share.DataHash) (io.ReadCloser, error)

GetCAR takes a DataRoot and returns a buffered reader to the respective EDS serialized as a CARv1 file. The Reader strictly reads the CAR header and first quadrant (1/4) of the EDS, omitting all the NMT Merkle proofs. Integrity of the store data is not verified.

Caller must Close returned reader after reading.

func (*Store) Has

func (s *Store) Has(ctx context.Context, root share.DataHash) (bool, error)

Has checks if EDS exists by the given share.Root hash.

func (*Store) Put

func (s *Store) Put(ctx context.Context, root share.DataHash, square *rsmt2d.ExtendedDataSquare) error

Put stores the given data square with DataRoot's hash as a key.

The square is verified on the Exchange level, and Put only stores the square, trusting it. The resulting file stores all the shares and NMT Merkle Proofs of the EDS. Additionally, the file gets indexed s.t. store.Blockstore can access them.

func (*Store) Remove

func (s *Store) Remove(ctx context.Context, root share.DataHash) error

Remove removes EDS from Store by the given share.Root hash and cleans up all the indexing.

func (*Store) Start

func (s *Store) Start(context.Context) error

func (*Store) Stop

func (s *Store) Stop(context.Context) error

Stop stops the underlying DAGStore.

type Subscription

type Subscription struct {
	// contains filtered or unexported fields
}

Subscription is a wrapper over pubsub.Subscription that handles receiving an EDS DataHash from other peers.

func (*Subscription) Cancel

func (subs *Subscription) Cancel()

Cancel stops the subscription.

func (*Subscription) Next

func (subs *Subscription) Next(ctx context.Context) (share.DataHash, error)

Next blocks the caller until any new EDS DataHash notification arrives. Returns only notifications which successfully pass validation.

type Validator

type Validator func(context.Context, peer.ID, share.DataHash) pubsub.ValidationResult

Validator is an injectable func and governs EDS notification or DataHash validity. It receives the notification and sender peer and expects the validation result. Validator is allowed to be blocking for an indefinite time or until the context is canceled.