eds

package
v0.15.0-alpha.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 7, 2024 License: Apache-2.0 Imports: 48 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrEmptySquare = errors.New("share: importing empty data")
View Source 
var ErrNotFound = errors.New("eds not found in store")
View Source 
var ErrNotFoundInIndex = errors.New("does not exist in index")

ErrNotFoundInIndex is returned instead of ErrNotFound if the multihash doesn't exist in the index

View Source 
var RetrieveQuadrantTimeout = blockTime / numQuadrants * 2

RetrieveQuadrantTimeout defines how much time Retriever waits before starting to retrieve another quadrant.

NOTE: - The whole data square must be retrieved in less than block time. - We have 4 quadrants from two sources(rows, cols) which equals to 8 in total.

Functions

func CollectSharesByNamespace added in v0.11.0 

func CollectSharesByNamespace(
	ctx context.Context,
	bg blockservice.BlockGetter,
	root *share.Root,
	namespace share.Namespace,
) (shares share.NamespacedShares, err error)

CollectSharesByNamespace collects NamespaceShares within the given namespace from share.Root.

func ODSReader added in v0.5.0 

func ODSReader(carReader io.Reader) (io.Reader, error)

ODSReader reads CARv1 encoded data from io.ReadCloser and limits the reader to the CAR header and first quadrant (ODS)

func ReadEDS added in v0.5.0 

func ReadEDS(ctx context.Context, r io.Reader, root share.DataHash) (eds *rsmt2d.ExtendedDataSquare, err error)

ReadEDS reads the first EDS quadrant (1/4) from an io.Reader CAR file. Only the first quadrant will be read, which represents the original data. The returned EDS is guaranteed to be full and valid against the DataRoot, otherwise ReadEDS errors.

func RetrieveNamespaceFromStore added in v0.11.0 

func RetrieveNamespaceFromStore(
	ctx context.Context,
	store *Store,
	dah *share.Root,
	namespace share.Namespace,
) (shares share.NamespacedShares, err error)

RetrieveNamespaceFromStore gets all EDS shares in the given namespace from the EDS store through the corresponding CAR-level blockstore. It is extracted from the store getter to make it available for reuse in the shrexnd server.

func WriteEDS added in v0.5.0 

func WriteEDS(ctx context.Context, eds *rsmt2d.ExtendedDataSquare, w io.Writer) (err error)

WriteEDS writes the entire EDS into the given io.Writer as CARv1 file. This includes all shares in quadrant order, followed by all inner nodes of the NMT tree. Order: [ Carv1Header | Q1 | Q2 | Q3 | Q4 | inner nodes ] For more information about the header: https://ipld.io/specs/transport/car/carv1/#header

Types

type BlockGetter added in v0.6.2 

type BlockGetter struct {
	// contains filtered or unexported fields
}

BlockGetter is an adapter for dagstore.ReadBlockstore to implement blockservice.BlockGetter interface.

func NewBlockGetter added in v0.6.2 

func NewBlockGetter(store dagstore.ReadBlockstore) *BlockGetter

NewBlockGetter creates new blockservice.BlockGetter adapter from dagstore.ReadBlockstore

func (*BlockGetter) GetBlock added in v0.6.2 

func (bg *BlockGetter) GetBlock(ctx context.Context, cid cid.Cid) (blocks.Block, error)

GetBlock gets the requested block by the given CID.

func (*BlockGetter) GetBlocks added in v0.6.2 

func (bg *BlockGetter) GetBlocks(ctx context.Context, cids []cid.Cid) <-chan blocks.Block

GetBlocks does a batch request for the given cids, returning blocks as they are found, in no particular order.

It implements blockservice.BlockGetter interface, that requires: It may not be able to find all requested blocks (or the context may be canceled). In that case, it will close the channel early. It is up to the consumer to detect this situation and keep track which blocks it has received and which it hasn't.

type BlockstoreCloser added in v0.11.0 

type BlockstoreCloser struct {
	dagstore.ReadBlockstore
	io.Closer
}

BlockstoreCloser represents a blockstore that can also be closed. It combines the functionality of a dagstore.ReadBlockstore with that of an io.Closer.

type Parameters added in v0.11.0 

type Parameters struct {
	// GC performs DAG store garbage collection by reclaiming transient files of
	// shards that are currently available but inactive, or errored.
	// We don't use transient files right now, so GC is turned off by default.
	GCInterval time.Duration

	// RecentBlocksCacheSize is the size of the cache for recent blocks.
	RecentBlocksCacheSize int

	// BlockstoreCacheSize is the size of the cache for blockstore requested accessors.
	BlockstoreCacheSize int
}

func DefaultParameters added in v0.11.0 

func DefaultParameters() *Parameters

DefaultParameters returns the default configuration values for the EDS store parameters.

func (*Parameters) Validate added in v0.11.0 

func (p *Parameters) Validate() error

type Retriever 

type Retriever struct {
	// contains filtered or unexported fields
}

Retriever retrieves rsmt2d.ExtendedDataSquares from the IPLD network. Instead of requesting data 'share by share' it requests data by quadrants minimizing bandwidth usage in the happy cases. 

 ---- ----
| 0  | 1  |
 ---- ----
| 2  | 3  |
 ---- ----

Retriever randomly picks one of the data square quadrants and tries to request them one by one until it is able to reconstruct the whole square.

func NewRetriever 

func NewRetriever(bServ blockservice.BlockService) *Retriever

NewRetriever creates a new instance of the Retriever over IPLD BlockService and rmst2d.Codec

func (*Retriever) Retrieve 

func (r *Retriever) Retrieve(ctx context.Context, dah *da.DataAvailabilityHeader) (*rsmt2d.ExtendedDataSquare, error)

Retrieve retrieves all the data committed to DataAvailabilityHeader.

If not available locally, it aims to request from the network only one quadrant (1/4) of the data square and reconstructs the other three quadrants (3/4). If the requested quadrant is not available within RetrieveQuadrantTimeout, it starts requesting another quadrant until either the data is reconstructed, context is canceled or ErrByzantine is generated.

type Store added in v0.5.0 

type Store struct {
	// contains filtered or unexported fields
}

Store maintains (via DAGStore) a top-level index enabling granular and efficient random access to every share and/or Merkle proof over every registered CARv1 file. The EDSStore provides a custom blockstore interface implementation to achieve access. The main use-case is randomized sampling over the whole chain of EDS block data and getting data by namespace.

func NewStore added in v0.5.0 

func NewStore(params *Parameters, basePath string, ds datastore.Batching) (*Store, error)

NewStore creates a new EDS Store under the given basepath and datastore.

func (*Store) Blockstore added in v0.5.0 

func (s *Store) Blockstore() bstore.Blockstore

Blockstore returns an IPFS blockstore providing access to individual shares/nodes of all EDS registered on the Store. NOTE: The blockstore does not store whole Celestia Blocks but IPFS blocks. We represent `shares` and NMT Merkle proofs as IPFS blocks and IPLD nodes so Bitswap can access those.

func (*Store) CARBlockstore added in v0.5.0 

func (s *Store) CARBlockstore(
	ctx context.Context,
	root share.DataHash,
) (*BlockstoreCloser, error)

CARBlockstore returns an IPFS Blockstore providing access to individual shares/nodes of a specific EDS identified by DataHash and registered on the Store. NOTE: The Blockstore does not store whole Celestia Blocks but IPFS blocks. We represent `shares` and NMT Merkle proofs as IPFS blocks and IPLD nodes so Bitswap can access those.

func (*Store) Get added in v0.5.0 

func (s *Store) Get(ctx context.Context, root share.DataHash) (*rsmt2d.ExtendedDataSquare, error)

Get reads EDS out of Store by given DataRoot.

It reads only one quadrant(1/4) of the EDS and verifies the integrity of the stored data by recomputing it.

func (*Store) GetCAR added in v0.5.0 

func (s *Store) GetCAR(ctx context.Context, root share.DataHash) (io.ReadCloser, error)

GetCAR takes a DataRoot and returns a buffered reader to the respective EDS serialized as a CARv1 file. The Reader strictly reads the CAR header and first quadrant (1/4) of the EDS, omitting all the NMT Merkle proofs. Integrity of the store data is not verified.

The shard is cached in the Store, so subsequent calls to GetCAR with the same root will use the same reader. The cache is responsible for closing the underlying reader.

func (*Store) GetDAH added in v0.6.2 

func (s *Store) GetDAH(ctx context.Context, root share.DataHash) (*share.Root, error)

GetDAH returns the DataAvailabilityHeader for the EDS identified by DataHash.

func (*Store) Has added in v0.5.0 

func (s *Store) Has(ctx context.Context, root share.DataHash) (has bool, err error)

Has checks if EDS exists by the given share.Root hash.

func (*Store) List added in v0.11.0 

func (s *Store) List() ([]share.DataHash, error)

List lists all the registered EDSes.

func (*Store) Put added in v0.5.0 

func (s *Store) Put(ctx context.Context, root share.DataHash, square *rsmt2d.ExtendedDataSquare) error

Put stores the given data square with DataRoot's hash as a key.

The square is verified on the Exchange level, and Put only stores the square, trusting it. The resulting file stores all the shares and NMT Merkle Proofs of the EDS. Additionally, the file gets indexed s.t. store.Blockstore can access them.

func (*Store) Remove added in v0.5.0 

func (s *Store) Remove(ctx context.Context, root share.DataHash) error

Remove removes EDS from Store by the given share.Root hash and cleans up all the indexing.

func (*Store) Start added in v0.5.0 

func (s *Store) Start(ctx context.Context) error

func (*Store) Stop added in v0.5.0 

func (s *Store) Stop(context.Context) error

Stop stops the underlying DAGStore.

func (*Store) WithMetrics added in v0.11.0 

func (s *Store) WithMetrics() error

Source Files

View all Source files

Directories

Path Synopsis
pb

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.