storageimpl

package
v0.0.0-...-75360cb Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 11, 2023 License: Apache-2.0, MIT Imports: 52 Imported by: 0

Documentation

Overview

Package storageimpl provides the primary implementation of storage market top level interfaces

This package provides a production implementation of `StorageClient` and `StorageProvider`.

Index

Constants

View Source 
const DefaultMaxTraversalLinks = 2 << 29

DefaultMaxTraversalLinks is the maximum number of links to traverse during CommP calculation before returning an error

View Source 
const DefaultPollingInterval = 30 * time.Second

DefaultPollingInterval is the frequency with which we query the provider for a status update

Variables

View Source 
var ClientFSMParameterSpec = fsm.Parameters{
	Environment:     &clientDealEnvironment{},
	StateType:       storagemarket.ClientDeal{},
	StateKeyField:   "State",
	Events:          clientstates.ClientEvents,
	StateEntryFuncs: clientstates.ClientStateEntryFuncs,
	FinalityStates:  clientstates.ClientFinalityStates,
}

ClientFSMParameterSpec is a valid set of parameters for a client deal FSM - used in doc generation

View Source 
var ProviderFSMParameterSpec = fsm.Parameters{
	Environment:     &providerDealEnvironment{},
	StateType:       storagemarket.MinerDeal{},
	StateKeyField:   "State",
	Events:          providerstates.ProviderEvents,
	StateEntryFuncs: providerstates.ProviderStateEntryFuncs,
	FinalityStates:  providerstates.ProviderFinalityStates,
}

ProviderFSMParameterSpec is a valid set of parameters for a provider FSM - used in doc generation

Functions

func NewProvider 

func NewProvider(net network.StorageMarketNetwork,
	ds datastore.Batching,
	fs filestore.FileStore,
	dagStore stores.DAGStoreWrapper,
	indexer provider.Interface,
	pieceStore piecestore.PieceStore,
	dataTransfer datatransfer.Manager,
	spn storagemarket.StorageProviderNode,
	minerAddress address.Address,
	storedAsk StoredAsk,
	meshCreator MeshCreator,
	options ...StorageProviderOption,
) (storagemarket.StorageProvider, error)

NewProvider returns a new storage provider

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is the production implementation of the StorageClient interface

func NewClient 

func NewClient(
	net network.StorageMarketNetwork,
	dataTransfer datatransfer.Manager,
	discovery *discoveryimpl.Local,
	ds datastore.Batching,
	scn storagemarket.StorageClientNode,
	bstores storagemarket.BlockstoreAccessor,
	options ...StorageClientOption,
) (*Client, error)

NewClient creates a new storage client

func (*Client) AddPaymentEscrow 

func (c *Client) AddPaymentEscrow(ctx context.Context, addr address.Address, amount abi.TokenAmount) error

AddPaymentEscrow adds funds for storage deals

func (*Client) Configure 

func (c *Client) Configure(options ...StorageClientOption)

Configure applies the given list of StorageClientOptions after a StorageClient is initialized

func (*Client) GetAsk 

func (c *Client) GetAsk(ctx context.Context, info storagemarket.StorageProviderInfo) (*storagemarket.StorageAsk, error)

GetAsk queries a provider for its current storage ask

The client creates a new `StorageAskStream` for the chosen peer ID, and calls WriteAskRequest on it, which constructs a message and writes it to the Ask stream. When it receives a response, it verifies the signature and returns the validated StorageAsk if successful

func (*Client) GetLocalDeal 

func (c *Client) GetLocalDeal(ctx context.Context, cid cid.Cid) (storagemarket.ClientDeal, error)

GetLocalDeal lists deals that are in progress or rejected

func (*Client) GetPaymentEscrow 

func (c *Client) GetPaymentEscrow(ctx context.Context, addr address.Address) (storagemarket.Balance, error)

GetPaymentEscrow returns the current funds available for deal payment

func (*Client) GetProviderDealState 

func (c *Client) GetProviderDealState(ctx context.Context, proposalCid cid.Cid) (*storagemarket.ProviderDealState, error)

GetProviderDealState queries a provider for the current state of a client's deal

func (*Client) ListLocalDeals 

func (c *Client) ListLocalDeals(ctx context.Context) ([]storagemarket.ClientDeal, error)

ListLocalDeals lists deals initiated by this storage client

func (*Client) ListProviders 

func (c *Client) ListProviders(ctx context.Context) (<-chan storagemarket.StorageProviderInfo, error)

ListProviders queries chain state and returns active storage providers

func (*Client) OnReady 

func (c *Client) OnReady(ready shared.ReadyFunc)

OnReady registers a listener for when the client has finished starting up

func (*Client) PollingInterval 

func (c *Client) PollingInterval() time.Duration

PollingInterval is a getter for the polling interval option

func (*Client) ProposeStorageDeal 

func (c *Client) ProposeStorageDeal(ctx context.Context, params storagemarket.ProposeStorageDealParams) (*storagemarket.ProposeStorageDealResult, error)

ProposeStorageDeal initiates the retrieval deal flow, which involves multiple requests and responses.

This function is called after using ListProviders and QueryAs are used to identify an appropriate provider to store data. The parameters passed to ProposeStorageDeal should matched those returned by the miner from QueryAsk to ensure the greatest likelihood the provider will accept the deal.

When called, the client takes the following actions:

1. Calculates the PieceCID for this deal from the given PayloadCID. (by writing the payload to a CAR file then calculating a merkle root for the resulting data)

2. Constructs a `DealProposal` (spec-actors type) with deal terms

3. Signs the `DealProposal` to make a ClientDealProposal

4. Gets the CID for the ClientDealProposal

5. Construct a ClientDeal to track the state of this deal.

6. Tells its statemachine to begin tracking the deal state by the CID of the ClientDealProposal

7. Triggers a `ClientEventOpen` event on its statemachine.

8. Records the Provider as a possible peer for retrieving this data in the future

From then on, the statemachine controls the deal flow in the client. Other components may listen for events in this flow by calling `SubscribeToEvents` on the Client. The Client also provides access to the node and network and other functionality through its implementation of the Client FSM's ClientDealEnvironment.

Documentation of the client state machine can be found at https://godoc.org/github.com/filecoin-project/go-fil-markets/storagemarket/impl/clientstates

func (*Client) Start 

func (c *Client) Start(ctx context.Context) error

Start initializes deal processing on a StorageClient, runs migrations and restarts in progress deals

func (*Client) Stop 

func (c *Client) Stop() error

Stop ends deal processing on a StorageClient

func (*Client) SubscribeToEvents 

func (c *Client) SubscribeToEvents(subscriber storagemarket.ClientSubscriber) shared.Unsubscribe

SubscribeToEvents allows another component to listen for events on the StorageClient in order to track deals as they progress through the deal flow

type DealDeciderFunc 

type DealDeciderFunc func(context.Context, storagemarket.MinerDeal) (bool, string, error)

DealDeciderFunc is a function which evaluates an incoming deal to decide if it its accepted It returns: - boolean = true if deal accepted, false if rejected - string = reason deal was not excepted, if rejected - error = if an error occurred trying to decide

type MeshCreator 

type MeshCreator interface {
	Connect(context.Context) error
}

type MetadataFunc 

type MetadataFunc func(storagemarket.MinerDeal) metadata.Metadata

type Provider 

type Provider struct {
	// contains filtered or unexported fields
}

Provider is the production implementation of the StorageProvider interface

func (*Provider) AddStorageCollateral 

func (p *Provider) AddStorageCollateral(ctx context.Context, amount abi.TokenAmount) error

AddStorageCollateral adds storage collateral

func (*Provider) AnnounceAllDealsToIndexer 

func (p *Provider) AnnounceAllDealsToIndexer(ctx context.Context) error

func (*Provider) AnnounceDealToIndexer 

func (p *Provider) AnnounceDealToIndexer(ctx context.Context, proposalCid cid.Cid) error

AnnounceDealToIndexer informs indexer nodes that a new deal was received, so they can download its index

func (*Provider) AwaitReady 

func (p *Provider) AwaitReady() error

func (*Provider) Configure 

func (p *Provider) Configure(options ...StorageProviderOption)

Configure applies the given list of StorageProviderOptions after a StorageProvider is initialized

func (*Provider) GetAsk 

func (p *Provider) GetAsk() *storagemarket.SignedStorageAsk

GetAsk returns the storage miner's ask, or nil if one does not exist.

func (*Provider) GetLocalDeal 

func (p *Provider) GetLocalDeal(propCid cid.Cid) (storagemarket.MinerDeal, error)

func (*Provider) GetStorageCollateral 

func (p *Provider) GetStorageCollateral(ctx context.Context) (storagemarket.Balance, error)

GetStorageCollateral returns the current collateral balance

func (*Provider) HandleAskStream 

func (p *Provider) HandleAskStream(s network.StorageAskStream)

HandleAskStream is called by the network implementation whenever a new message is received on the ask protocol

A Provider handling a `AskRequest` does the following:

1. Reads the current signed storage ask from storage

2. Wraps the signed ask in an AskResponse and writes it on the StorageAskStream

The connection is kept open only as long as the request-response exchange.

func (*Provider) HandleDealStatusStream 

func (p *Provider) HandleDealStatusStream(s network.DealStatusStream)

HandleDealStatusStream is called by the network implementation whenever a new message is received on the deal status protocol

A Provider handling a `DealStatuRequest` does the following:

1. Lots the deal state from the Provider FSM

2. Verifies the signature on the DealStatusRequest matches the Client for this deal

3. Constructs a ProviderDealState from the deal state

4. Signs the ProviderDealState with its private key

5. Writes a DealStatusResponse with the ProviderDealState and signature onto the DealStatusStream

The connection is kept open only as long as the request-response exchange.

func (*Provider) HandleDealStream 

func (p *Provider) HandleDealStream(s network.StorageDealStream)

HandleDealStream is called by the network implementation whenever a new message is received on the deal protocol

It initiates the provider side of the deal flow.

When a provider receives a DealProposal of the deal protocol, it takes the following steps:

1. Calculates the CID for the received ClientDealProposal.

2. Constructs a MinerDeal to track the state of this deal.

3. Tells its statemachine to begin tracking this deal state by CID of the received ClientDealProposal

4. Tracks the received deal stream by the CID of the ClientDealProposal

4. Triggers a `ProviderEventOpen` event on its statemachine.

From then on, the statemachine controls the deal flow in the client. Other components may listen for events in this flow by calling `SubscribeToEvents` on the Provider. The Provider handles loading the next block to send to the client.

Documentation of the client state machine can be found at https://godoc.org/github.com/filecoin-project/go-fil-markets/storagemarket/impl/providerstates

func (*Provider) ImportDataForDeal 

func (p *Provider) ImportDataForDeal(ctx context.Context, propCid cid.Cid, data io.Reader) error

ImportDataForDeal manually imports data for an offline storage deal It will verify that the data in the passed io.Reader matches the expected piece cid for the given deal or it will error

func (*Provider) ListLocalDeals 

func (p *Provider) ListLocalDeals() ([]storagemarket.MinerDeal, error)

ListLocalDeals lists deals processed by this storage provider

func (*Provider) ListLocalDealsPage 

func (p *Provider) ListLocalDealsPage(startPropCid *cid.Cid, offset int, limit int) ([]storagemarket.MinerDeal, error)

func (*Provider) LocalDealCount 

func (p *Provider) LocalDealCount() (int, error)

func (*Provider) OnReady 

func (p *Provider) OnReady(ready shared.ReadyFunc)

OnReady registers a listener for when the provider has finished starting up

func (*Provider) RetryDealPublishing 

func (p *Provider) RetryDealPublishing(propcid cid.Cid) error

func (*Provider) SetAsk 

func (p *Provider) SetAsk(price abi.TokenAmount, verifiedPrice abi.TokenAmount, duration abi.ChainEpoch, options ...storagemarket.StorageAskOption) error

SetAsk configures the storage miner's ask with the provided price, duration, and options. Any previously-existing ask is replaced.

func (*Provider) Start 

func (p *Provider) Start(ctx context.Context) error

Start initializes deal processing on a StorageProvider and restarts in progress deals. It also registers the provider with a StorageMarketNetwork so it can receive incoming messages on the storage market's libp2p protocols

func (*Provider) Stop 

func (p *Provider) Stop() error

Stop terminates processing of deals on a StorageProvider

func (*Provider) SubscribeToEvents 

func (p *Provider) SubscribeToEvents(subscriber storagemarket.ProviderSubscriber) shared.Unsubscribe

SubscribeToEvents allows another component to listen for events on the StorageProvider in order to track deals as they progress through the deal flow

type StorageClientOption 

type StorageClientOption func(c *Client)

StorageClientOption allows custom configuration of a storage client

func DealPollingInterval 

func DealPollingInterval(t time.Duration) StorageClientOption

DealPollingInterval sets the interval at which this client will query the Provider for deal state while waiting for deal acceptance 

func MaxTraversalLinks(m uint64) StorageClientOption

MaxTraversalLinks sets the maximum number of links in a DAG to traverse when calculating CommP, sets a budget that limits the depth and density of a DAG that can be traversed

type StorageProviderOption 

type StorageProviderOption func(p *Provider)

StorageProviderOption allows custom configuration of a storage provider

func AwaitTransferRestartTimeout 

func AwaitTransferRestartTimeout(waitTime time.Duration) StorageProviderOption

AwaitTransferRestartTimeout sets the maximum amount of time a provider will wait for a client to restart a data transfer when the node starts up before failing the deal

func CustomDealDecisionLogic 

func CustomDealDecisionLogic(decider DealDeciderFunc) StorageProviderOption

CustomDealDecisionLogic allows a provider to call custom decision logic when validating incoming deal proposals

func CustomMetadataGenerator 

func CustomMetadataGenerator(metadataFunc MetadataFunc) StorageProviderOption

type StoredAsk 

type StoredAsk interface {
	GetAsk() *storagemarket.SignedStorageAsk
	SetAsk(price abi.TokenAmount, verifiedPrice abi.TokenAmount, duration abi.ChainEpoch, options ...storagemarket.StorageAskOption) error
}

StoredAsk is an interface which provides access to a StorageAsk

Source Files

View all Source files

Directories

Path Synopsis
Package blockrecorder provides utilits to record locations of CIDs to a temporary metadata file, since writing a CAR happens BEFORE we actually hand off for sealing.
 Package blockrecorder provides utilits to record locations of CIDs to a temporary metadata file, since writing a CAR happens BEFORE we actually hand off for sealing.
Package clientstates contains state machine logic relating to the `StorageMarket`.
 Package clientstates contains state machine logic relating to the `StorageMarket`.
Package clientutils provides utility functions for the storage client & client FSM
 Package clientutils provides utility functions for the storage client & client FSM
Package connmanager tracks open connections maping storage proposal CID -> StorageDealStream
 Package connmanager tracks open connections maping storage proposal CID -> StorageDealStream
Package dtutils provides event listeners for the client and provider to listen for events on the data transfer module and dispatch FSM events based on them
 Package dtutils provides event listeners for the client and provider to listen for events on the data transfer module and dispatch FSM events based on them
Package providerstates contains state machine logic relating to the `StorageProvider`.
 Package providerstates contains state machine logic relating to the `StorageProvider`.
Package providerutils provides utility functions for the storage provider & provider FSM
 Package providerutils provides utility functions for the storage provider & provider FSM
Package requestvalidation implements a request validator for the data transfer module to validate data transfer requests for storage deals
 Package requestvalidation implements a request validator for the data transfer module to validate data transfer requests for storage deals

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.