eds

Documentation

Index

• Variables
• func ODSReader(carReader io.Reader) (io.Reader, error)
• func ReadEDS(ctx context.Context, r io.Reader, root share.DataHash) (*rsmt2d.ExtendedDataSquare, error)
• func WriteEDS(ctx context.Context, eds *rsmt2d.ExtendedDataSquare, w io.Writer) error
• type Retriever
  • func NewRetriever(bServ blockservice.BlockService) *Retriever
  • func (r *Retriever) Retrieve(ctx context.Context, dah *da.DataAvailabilityHeader) (*rsmt2d.ExtendedDataSquare, error)
• type Store
  • func NewStore(basepath string, ds datastore.Batching) (*Store, error)
  • func (s *Store) Blockstore() bstore.Blockstore
  • func (s *Store) CARBlockstore(ctx context.Context, root share.DataHash) (dagstore.ReadBlockstore, error)
  • func (s *Store) Get(ctx context.Context, root share.DataHash) (*rsmt2d.ExtendedDataSquare, error)
  • func (s *Store) GetCAR(ctx context.Context, root share.DataHash) (io.ReadCloser, error)
  • func (s *Store) GetDAH(ctx context.Context, root share.DataHash) (*share.Root, error)
  • func (s *Store) Has(ctx context.Context, root share.DataHash) (bool, error)
  • func (s *Store) Put(ctx context.Context, root share.DataHash, square *rsmt2d.ExtendedDataSquare) error
  • func (s *Store) Remove(ctx context.Context, root share.DataHash) error
  • func (s *Store) Start(context.Context) error
  • func (s *Store) Stop(context.Context) error

Variables

var ErrEmptySquare = errors.New("share: importing empty data")

var RetrieveQuadrantTimeout = blockTime / numQuadrants * 2

RetrieveQuadrantTimeout defines how much time Retriever waits before starting to retrieve another quadrant.

NOTE: - The whole data square must be retrieved in less than block time. - We have 4 quadrants from two sources(rows, cols) which equals to 8 in total.

Functions

func ODSReader

func ODSReader(carReader io.Reader) (io.Reader, error)

ODSReader reads CARv1 encoded data from io.ReadCloser and limits the reader to the CAR header and first quadrant (ODS)

func ReadEDS

func ReadEDS(ctx context.Context, r io.Reader, root share.DataHash) (*rsmt2d.ExtendedDataSquare, error)

ReadEDS reads the first EDS quadrant (1/4) from an io.Reader CAR file. Only the first quadrant will be read, which represents the original data. The returned EDS is guaranteed to be full and valid against the DataRoot, otherwise ReadEDS errors.

func WriteEDS

func WriteEDS(ctx context.Context, eds *rsmt2d.ExtendedDataSquare, w io.Writer) error

WriteEDS writes the entire EDS into the given io.Writer as CARv1 file. This includes all shares in quadrant order, followed by all inner nodes of the NMT tree. Order: [ Carv1Header | Q1 | Q2 | Q3 | Q4 | inner nodes ] For more information about the header: https://ipld.io/specs/transport/car/carv1/#header

Types

type Retriever

type Retriever struct {
	// contains filtered or unexported fields
}

Retriever retrieves rsmt2d.ExtendedDataSquares from the IPLD network. Instead of requesting data 'share by share' it requests data by quadrants minimizing bandwidth usage in the happy cases.

 ---- ----
| 0  | 1  |
 ---- ----
| 2  | 3  |
 ---- ----

Retriever randomly picks one of the data square quadrants and tries to request them one by one until it is able to reconstruct the whole square.

func NewRetriever

func NewRetriever(bServ blockservice.BlockService) *Retriever

NewRetriever creates a new instance of the Retriever over IPLD BlockService and rmst2d.Codec

func (*Retriever) Retrieve

func (r *Retriever) Retrieve(ctx context.Context, dah *da.DataAvailabilityHeader) (*rsmt2d.ExtendedDataSquare, error)

Retrieve retrieves all the data committed to DataAvailabilityHeader.

If not available locally, it aims to request from the network only one quadrant (1/4) of the data square and reconstructs the other three quadrants (3/4). If the requested quadrant is not available within RetrieveQuadrantTimeout, it starts requesting another quadrant until either the data is reconstructed, context is canceled or ErrByzantine is generated.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store maintains (via DAGStore) a top-level index enabling granular and efficient random access to every share and/or Merkle proof over every registered CARv1 file. The EDSStore provides a custom blockstore interface implementation to achieve access. The main use-case is randomized sampling over the whole chain of EDS block data and getting data by namespace.

func NewStore

func NewStore(basepath string, ds datastore.Batching) (*Store, error)

NewStore creates a new EDS Store under the given basepath and datastore.

func (*Store) Blockstore

func (s *Store) Blockstore() bstore.Blockstore

Blockstore returns an IPFS blockstore providing access to individual shares/nodes of all EDS registered on the Store. NOTE: The blockstore does not store whole Celestia Blocks but IPFS blocks. We represent `shares` and NMT Merkle proofs as IPFS blocks and IPLD nodes so Bitswap can access those.

func (*Store) CARBlockstore

func (s *Store) CARBlockstore(
	ctx context.Context,
	root share.DataHash,
) (dagstore.ReadBlockstore, error)

CARBlockstore returns an IPFS Blockstore providing access to individual shares/nodes of a specific EDS identified by DataHash and registered on the Store. NOTE: The Blockstore does not store whole Celestia Blocks but IPFS blocks. We represent `shares` and NMT Merkle proofs as IPFS blocks and IPLD nodes so Bitswap can access those.

func (*Store) Get

func (s *Store) Get(ctx context.Context, root share.DataHash) (*rsmt2d.ExtendedDataSquare, error)

Get reads EDS out of Store by given DataRoot.

It reads only one quadrant(1/4) of the EDS and verifies the integrity of the stored data by recomputing it.

func (*Store) GetCAR

func (s *Store) GetCAR(ctx context.Context, root share.DataHash) (io.ReadCloser, error)

GetCAR takes a DataRoot and returns a buffered reader to the respective EDS serialized as a CARv1 file. The Reader strictly reads the CAR header and first quadrant (1/4) of the EDS, omitting all the NMT Merkle proofs. Integrity of the store data is not verified.

Caller must Close returned reader after reading.

func (*Store) GetDAH

func (s *Store) GetDAH(ctx context.Context, root share.DataHash) (*share.Root, error)

GetDAH returns the DataAvailabilityHeader for the EDS identified by DataHash.

func (*Store) Has

func (s *Store) Has(ctx context.Context, root share.DataHash) (bool, error)

Has checks if EDS exists by the given share.Root hash.

func (*Store) Put

func (s *Store) Put(ctx context.Context, root share.DataHash, square *rsmt2d.ExtendedDataSquare) error

Put stores the given data square with DataRoot's hash as a key.

The square is verified on the Exchange level, and Put only stores the square, trusting it. The resulting file stores all the shares and NMT Merkle Proofs of the EDS. Additionally, the file gets indexed s.t. store.Blockstore can access them.

func (*Store) Remove

func (s *Store) Remove(ctx context.Context, root share.DataHash) error

Remove removes EDS from Store by the given share.Root hash and cleans up all the indexing.

func (*Store) Start

func (s *Store) Start(context.Context) error

func (*Store) Stop

func (s *Store) Stop(context.Context) error

Stop stops the underlying DAGStore.