share

package
v0.4.6-dirty Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 27, 2022 License: Apache-2.0 Imports: 42 Imported by: 0

Documentation

Overview

Package share contains logic related to the retrieval and random sampling of shares of block data.

Though this package contains several useful methods for getting specific shares and/or sampling them at random, a particularly useful method is GetSharesByNamespace which retrieves all shares of block data of the given namespace.ID from the block associated with the given DataAvailabilityHeader (DAH, but referred to as Root within this package).

This package also contains both implementations of the Availability interface: lightAvailability which samples for 16 shares of block data (enough to verify the block's availability on the network) and fullAvailability which samples for as many shares as necessary to fully reconstruct the block data.

TODO(@Wondertan): Instead of doing sampling over the coordinates do a random walk over NMT trees.

Index

Constants

View Source 
const AvailabilityTimeout = 20 * time.Minute

AvailabilityTimeout specifies timeout for DA validation during which data have to be found on the network, otherwise ErrNotAvailable is fired. TODO: https://github.com/celestiaorg/celestia-node/issues/10

Variables

View Source 
var DefaultSampleAmount = 16

DefaultSampleAmount sets the default amount of samples to be sampled from the network by lightAvailability.

View Source 
var (
	// DefaultWriteBatchSize defines the size of the batched header write.
	// Headers are written in batches not to thrash the underlying Datastore with writes.
	// TODO(@Wondertan, @renaynay): Those values must be configurable and proper defaults should be set for specific node
	//  type. (#709)
	DefaultWriteBatchSize = 2048
)
View Source 
var ErrNotAvailable = errors.New("da: data not available")

ErrNotAvailable is returned whenever DA sampling fails.

View Source 
var GetData = ipld.ShareData

GetData extracts data out of a Share.

View Source 
var GetID = ipld.ShareID

GetID extracts namespace ID out of a Share.

Functions

func EnsureEmptySquareExists added in v0.3.0 

func EnsureEmptySquareExists(ctx context.Context, bServ blockservice.BlockService) error

EnsureEmptySquareExists checks if the given DAG contains an empty block data square. If it does not, it stores an empty block. This optimization exists to prevent redundant storing of empty block data so that it is only stored once and returned upon request for a block with an empty data square. Ref: header/header.go#L56

func NewTestDAGNet added in v0.3.0 

func NewTestDAGNet(ctx context.Context, t *testing.T) *dagNet

NewTestDAGNet creates a new testing swarm utility to spawn different nodes and test how they interact and/or exchange data.

func RandFullLocalServiceWithSquare added in v0.3.0 

func RandFullLocalServiceWithSquare(t *testing.T, n int) (*Service, *Root)

RandFullLocalServiceWithSquare is the same as RandFullServiceWithSquare, except the Availability is wrapped with CacheAvailability.

func RandFullServiceWithSquare added in v0.2.0 

func RandFullServiceWithSquare(t *testing.T, n int) (*Service, *Root)

RandFullServiceWithSquare provides a share.Service filled with 'n' NMT trees of 'n' random shares, essentially storing a whole square.

func RandLightLocalServiceWithSquare added in v0.3.0 

func RandLightLocalServiceWithSquare(t *testing.T, n int) (*Service, *Root)

RandLightLocalServiceWithSquare is the same as RandLightServiceWithSquare, except the Availability is wrapped with CacheAvailability.

func RandLightServiceWithSquare added in v0.2.0 

func RandLightServiceWithSquare(t *testing.T, n int) (*Service, *Root)

RandLightServiceWithSquare provides a share.Service filled with 'n' NMT trees of 'n' random shares, essentially storing a whole square.

Types

type Availability 

type Availability interface {
	// SharesAvailable subjectively validates if Shares committed to the given Root are available on the Network.
	SharesAvailable(context.Context, *Root) error
	// ProbabilityOfAvailability calculates the probability of the data square
	// being available based on the number of samples collected.
	// TODO(@Wondertan): Merge with SharesAvailable method, eventually
	ProbabilityOfAvailability() float64
}

Availability defines interface for validation of Shares' availability.

func NewTestBrokenAvailability added in v0.3.0 

func NewTestBrokenAvailability() Availability

NewTestBrokenAvailability returns an instance of Availability that allows for testing error cases during sampling.

If the Root field is empty, it will return ErrNotAvailable on every call to SharesAvailable. Otherwise, it will only return ErrNotAvailable if the given Root hash matches the stored Root hash.

func NewTestSuccessfulAvailability added in v0.3.0 

func NewTestSuccessfulAvailability() Availability

NewTestSuccessfulAvailability returns an Availability that always returns successfully when SharesAvailable is called.

type CacheAvailability added in v0.3.0 

type CacheAvailability struct {
	// contains filtered or unexported fields
}

CacheAvailability wraps a given Availability (whether it's light or full) and stores the results of a successful sampling routine over a given Root's hash to disk.

func NewCacheAvailability added in v0.3.0 

func NewCacheAvailability(avail Availability, ds datastore.Batching) *CacheAvailability

NewCacheAvailability wraps the given Availability with an additional datastore for sampling result caching.

func (*CacheAvailability) Close added in v0.3.0 

func (ca *CacheAvailability) Close(ctx context.Context) error

Close flushes all queued writes to disk.

func (*CacheAvailability) ProbabilityOfAvailability added in v0.3.0 

func (ca *CacheAvailability) ProbabilityOfAvailability() float64

func (*CacheAvailability) SharesAvailable added in v0.3.0 

func (ca *CacheAvailability) SharesAvailable(ctx context.Context, root *Root) error

SharesAvailable will store, upon success, the hash of the given Root to disk.

type Discovery 

type Discovery struct {
	// contains filtered or unexported fields
}

discovery combines advertise and discover services and allows to store discovered nodes.

func NewDiscovery added in v0.3.0 

func NewDiscovery(
	h host.Host,
	d core.Discovery,
	peersLimit uint,
	discInterval,
	advertiseInterval time.Duration,
) *Discovery

NewDiscovery constructs a new discovery.

type FullAvailability added in v0.3.0 

type FullAvailability struct {
	// contains filtered or unexported fields
}

FullAvailability implements Availability using the full data square recovery technique. It is considered "full" because it is required to download enough shares to fully reconstruct the data square.

func NewFullAvailability added in v0.2.0 

func NewFullAvailability(bServ blockservice.BlockService, disc *Discovery) *FullAvailability

NewFullAvailability creates a new full Availability.

func TestFullAvailability added in v0.3.0 

func TestFullAvailability(bServ blockservice.BlockService) *FullAvailability

func (*FullAvailability) ProbabilityOfAvailability added in v0.3.0 

func (fa *FullAvailability) ProbabilityOfAvailability() float64

func (*FullAvailability) SharesAvailable added in v0.3.0 

func (fa *FullAvailability) SharesAvailable(ctx context.Context, root *Root) error

SharesAvailable reconstructs the data committed to the given Root by requesting enough Shares from the network.

func (*FullAvailability) Start added in v0.3.0 

func (fa *FullAvailability) Start(context.Context) error

func (*FullAvailability) Stop added in v0.3.0 

func (fa *FullAvailability) Stop(context.Context) error

type LightAvailability added in v0.3.0 

type LightAvailability struct {
	// contains filtered or unexported fields
}

LightAvailability implements Availability using Data Availability Sampling technique. It is light because it does not require the downloading of all the data to verify its availability. It is assumed that there are a lot of lightAvailability instances on the network doing sampling over the same Root to collectively verify its availability.

func NewLightAvailability 

func NewLightAvailability(
	bserv blockservice.BlockService,
	disc *Discovery,
) *LightAvailability

NewLightAvailability creates a new light Availability.

func TestLightAvailability added in v0.3.0 

func TestLightAvailability(bServ blockservice.BlockService) *LightAvailability

func (*LightAvailability) ProbabilityOfAvailability added in v0.3.0 

func (la *LightAvailability) ProbabilityOfAvailability() float64

ProbabilityOfAvailability calculates the probability that the data square is available based on the amount of samples collected (DefaultSampleAmount).

Formula: 1 - (0.75 ** amount of samples)

func (*LightAvailability) SharesAvailable added in v0.3.0 

func (la *LightAvailability) SharesAvailable(ctx context.Context, dah *Root) error

SharesAvailable randomly samples DefaultSamples amount of Shares committed to the given Root. This way SharesAvailable subjectively verifies that Shares are available.

func (*LightAvailability) Start added in v0.3.0 

func (la *LightAvailability) Start(context.Context) error

func (*LightAvailability) Stop added in v0.3.0 

func (la *LightAvailability) Stop(context.Context) error

type Root 

type Root = da.DataAvailabilityHeader

Root represents root commitment to multiple Shares. In practice, it is a commitment to all the Data in a square.

func FillBS added in v0.3.0 

func FillBS(t *testing.T, bServ blockservice.BlockService, shares []Share) *Root

FillBS fills the given BlockService with the given shares.

func RandFillBS added in v0.3.0 

func RandFillBS(t *testing.T, n int, bServ blockservice.BlockService) *Root

RandFillBS fills the given BlockService with a random block of a given size.

type Sample 

type Sample struct {
	Row, Col int
}

Sample is a point in 2D space over square.

func SampleSquare 

func SampleSquare(squareWidth int, num int) ([]Sample, error)

SampleSquare randomly picks *num* unique points from the given *width* square and returns them as samples.

type Service 

type Service struct {
	Availability
	// contains filtered or unexported fields
}

Service provides access to any data square or block share on the network.

All Get methods provided on Service follow the following flow:

  1. Check local storage for the requested Share.
  2. If exists * Load from disk * Return
  3. If not * Find provider on the network * Fetch the Share from the provider * Store the Share * Return

TODO(@Wondertan): Simple thread safety for Start and Stop would not hurt.

func NewService 

func NewService(bServ blockservice.BlockService, avail Availability) *Service

NewService creates new basic share.Service.

func RandLightService added in v0.3.0 

func RandLightService() (*Service, blockservice.BlockService)

RandLightService provides an unfilled share.Service with corresponding blockservice.BlockService than can be filled by the test.

func (*Service) GetShare 

func (s *Service) GetShare(ctx context.Context, dah *Root, row, col int) (Share, error)

func (*Service) GetShares 

func (s *Service) GetShares(ctx context.Context, root *Root) ([][]Share, error)

func (*Service) GetSharesByNamespace 

func (s *Service) GetSharesByNamespace(ctx context.Context, root *Root, nID namespace.ID) ([]Share, error)

GetSharesByNamespace iterates over a square's row roots and accumulates the found shares in the given namespace.ID.

func (*Service) Start 

func (s *Service) Start(context.Context) error

func (*Service) Stop 

func (s *Service) Stop(context.Context) error

type Share 

type Share = ipld.Share

Share is a fixed-size data chunk associated with a namespace ID, whose data will be erasure-coded and committed to in Namespace Merkle trees.

func RandShares 

func RandShares(t *testing.T, n int) []Share

RandShares provides 'n' randomized shares prefixed with random namespaces.

type TestBrokenAvailability added in v0.3.0 

type TestBrokenAvailability struct {
	Root *Root
}

func (*TestBrokenAvailability) ProbabilityOfAvailability added in v0.3.0 

func (b *TestBrokenAvailability) ProbabilityOfAvailability() float64

func (*TestBrokenAvailability) SharesAvailable added in v0.3.0 

func (b *TestBrokenAvailability) SharesAvailable(_ context.Context, root *Root) error

type TestSuccessfulAvailability added in v0.3.0 

type TestSuccessfulAvailability struct {
}

func (*TestSuccessfulAvailability) ProbabilityOfAvailability added in v0.3.0 

func (tsa *TestSuccessfulAvailability) ProbabilityOfAvailability() float64

func (*TestSuccessfulAvailability) SharesAvailable added in v0.3.0 

func (tsa *TestSuccessfulAvailability) SharesAvailable(context.Context, *Root) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.