storagemarket

package
v0.0.0-...-f311442 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 7, 2023 License: Apache-2.0, MIT Imports: 22 Imported by: 0

README

storagemarket

The storagemarket module is intended for Filecoin node implementations written in Go. It implements functionality to allow execution of storage market deals, and for Providers to set their storage price on the Filecoin network. The node implementation must provide access to chain operations, and persistent data storage.

Table of Contents

Background reading

Please see the Filecoin Storage Market Specification.

Installation

The build process for storagemarket requires Go >= v1.13.

To install:

go get github.com/filecoin-project/go-fil-markets/storagemarket

Operation

The storagemarket package provides high level APIs to execute data storage deals between a storage client and a storage provider (a.k.a. storage miner) on the Filecoin network. The Filecoin node must implement the StorageCommon, StorageProviderNode, and StorageClientNode interfaces in order to construct and use the module.

Deals are expected to survive a node restart; deals and related information are expected to be stored on disk.

storagemarket communicates its deal operations and requested data via go-data-transfer using go-graphsync.

Implementation

General Steps
  1. Decide if your node can be configured as a Storage Provider, a Storage Client or both.
  2. Determine how and where your retrieval calls to StorageProvider and StorageClient functions will be made.
  3. Implement the required interfaces as described in this section.
  4. Construct a StorageClient and/or StorageProvider in your node's startup. Call the StorageProvider's Start function it in the appropriate place, and its Stop function in the appropriate place.
  5. Expose desired storagemarket functionality to whatever internal modules desired, such as command line interface, JSON RPC, or HTTP API.

Implement the StorageCommon, StorageProviderNode, and StorageClientNode interfaces in storagemarket/types.go, described below:

StorageCommon

StorageCommon is an interface common to both StorageProviderNode and StorageClientNode. Its functions are:

AddFunds
func AddFunds(ctx context.Context, addr address.Address, amount abi.TokenAmount) (cid.Cid, error)

Send amount to addr by posting a message on chain. Return the message CID.

ReserveFunds
func ReserveFunds(ctx context.Context, addr, wallet address.Address, amount abi.TokenAmount) (cid.Cid, error)

Add amount to the total reserved funds for addr. If total available balance for addr in StorageMarketActor is not greater than total reserved, wallet should send any needed balance to addr by posting a message on chain. Returns the message CID.

ReserveFunds
func ReleaseFunds(ctx context.Context, addr address.Address, amount abi.TokenAmount) (cid.Cid, error)

Release amount funds from reserved total for addr. No withdrawal is performed for addr in the storage market actor but the funds released become available for future withdrawal (if new total reserved < total available in SMA)

GetBalance
func GetBalance(ctx context.Context, addr address.Address, tok shared.TipSetToken) (Balance, error)

Retrieve the Balance of FIL in addr. A Balance consists of Locked and Available abi.TokenAmounts

VerifySignature
func VerifySignature(ctx context.Context, signature crypto.Signature, signer address.Address, 
                plaintext []byte, tok shared.TipSetToken) (bool, error)

Verify that signature is valid, cryptographically and otherwise, for the given signer, plaintext, and tok.

WaitForMessage
func WaitForMessage(ctx context.Context, mcid cid.Cid, 
               onCompletion func(exitcode.ExitCode, []byte, error) error) error

Wait for message CID mcid to appear on chain, and call onCompletion when it does so.

SignBytes
func SignBytes(ctx context.Context, signer address.Address, b []byte) (*crypto.Signature, error)

Cryptographically sign bytes b using the private key referenced by address signer.

GetMinerWorkerAddress
func GetMinerWorkerAddress(ctx context.Context, addr address.Address, tok shared.TipSetToken,
                     ) (address.Address, error)

Get the miner worker address for the given miner owner, as of tok.

StorageProviderNode

StorageProviderNode is the interface for dependencies for a StorageProvider. It contains:

GetChainHead
func GetChainHead(ctx context.Context) (shared.TipSetToken, abi.ChainEpoch, error)

Get the current chain head. Return its TipSetToken and its abi.ChainEpoch.

PublishDeals
func PublishDeals(ctx context.Context, deal MinerDeal) (cid.Cid, error)

Post the deal to chain, returning the posted message CID.

OnDealComplete
func OnDealComplete(ctx context.Context, deal MinerDeal, pieceSize abi.UnpaddedPieceSize, 
               pieceReader io.Reader) error

The function to be called when MinerDeal deal has reached the storagemarket.StorageDealCompleted state. A MinerDeal contains more information than a StorageDeal, including paths, addresses, and CIDs pertinent to the deal. See storagemarket/types.go

OnDealSectorCommitted
func OnDealSectorCommitted(ctx context.Context, provider address.Address, dealID abi.DealID, 
                      cb DealSectorCommittedCallback) error

Register the function to be called once provider has committed sector(s) for dealID.

LocatePieceForDealWithinSector
func LocatePieceForDealWithinSector(ctx context.Context, dealID abi.DealID, tok shared.TipSetToken,
                              ) (sectorID uint64, offset uint64, length uint64, err error)

Find the piece associated with dealID as of tok and return the sector id, plus the offset and length of the data within the sector.

OnDealExpiredOrSlashed
func OnDealExpiredOrSlashed(
    ctx context.Context,
    dealID abi.DealID,
    onDealExpired DealExpiredCallback,
    onDealSlashed DealSlashedCallback) error

Register callbacks to be called when a deal expires or is slashed.

StorageClientNode

StorageClientNode implements dependencies for a StorageClient. It contains:

StorageCommon

StorageClientNode implements StorageCommon, described above.

GetChainHead
func GetChainHead(ctx context.Context) (shared.TipSetToken, abi.ChainEpoch, error)

Get the current chain head. Return its TipSetToken and its abi.ChainEpoch.

ListStorageProviders
func ListStorageProviders(ctx context.Context, tok shared.TipSetToken
                         ) ([]*StorageProviderInfo, error)

Return a slice of StorageProviderInfo, for all known storage providers.

ValidatePublishedDeal
func ValidatePublishedDeal(ctx context.Context, deal ClientDeal) (abi.DealID, error)

Query the chain for deal and inspect the message parameters to make sure they match the expected deal. Return the deal ID.

SignProposal
func SignProposal(ctx context.Context, signer address.Address, proposal market.DealProposal
                 ) (*market.ClientDealProposal, error)

Cryptographically sign proposal using the private key of signer and return a ClientDealProposal (includes signature data).

GetDefaultWalletAddress
func GetDefaultWalletAddress(ctx context.Context) (address.Address, error)

Get the Client's default wallet address, which will be used to add Storage Market funds (collateral and payment).

OnDealSectorCommitted
func OnDealSectorCommitted(ctx context.Context, provider address.Address, dealID abi.DealID, 
                          cb DealSectorCommittedCallback) error

Register a callback to be called once the Deal's sector(s) are committed.

OnDealExpiredOrSlashed
func OnDealExpiredOrSlashed(
    ctx context.Context,
    dealID abi.DealID,
    onDealExpired DealExpiredCallback,
    onDealSlashed DealSlashedCallback) error

Register callbacks to be called when a deal expires or is slashed.

GetMinerInfo
func GetMinerInfo(ctx context.Context, maddr address.Address, tok shared.TipSetToken,
                         ) (*StorageProviderInfo, error)

Returns StorageProviderInfo for a specific provider at the given address

Construction

StorageClient

To construct a new StorageClient:

func NewClient(
	net network.StorageMarketNetwork,
	bs blockstore.Blockstore,
	dataTransfer datatransfer.Manager,
	discovery *discovery.Local,
	ds datastore.Batching,
	scn storagemarket.StorageClientNode,
) (*Client, error)

Parameters

  • net network.StorageMarketNetwork is a network abstraction for the storage market. To create it, use:

    package network
func NewFromLibp2pHost(h host.Host) StorageMarketNetwork

  • bs blockstore.Blockstore is an IPFS blockstore for storing and retrieving data for deals. See github.com/ipfs/go-ipfs-blockstore.

  • dataTransfer datatransfer.Manager is an interface from github.com/filecoin-project/go-data-transfer There is more than one implementation, but one way to create a new datatransfer.Manager is:

    package graphsyncimpl

func NewGraphSyncDataTransfer(host host.Host, gs graphsync.GraphExchange, storedCounter *storedcounter.StoredCounter) datatransfer.Manager

    Also:

     package datatransfer

 // NewDAGServiceDataTransfer returns a data transfer manager based on
 // an IPLD DAGService
 func NewDAGServiceDataTransfer(dag ipldformat.DAGService) datatransfer.Manager

    Please see the go-data-transfer repo for more information.

  • discovery *discovery.Local implements the PeerResolver interface. To initialize a new discovery.Local:

    func NewLocal(ds datastore.Batching) *Local

  • ds datastore.Batching is a datastore for the deal's state machine. It is typically the node's own datastore that implements the IPFS datastore.Batching interface. See github.com/ipfs/go-datastore.

  • scn storagemarket.StorageClientNode is the implementation of the StorageClientNode API that was written for your node.

StorageProvider

To construct a new StorageProvider:

func NewProvider(net network.StorageMarketNetwork, 
                ds datastore.Batching, 
                bs blockstore.Blockstore, 
                fs filestore.FileStore, 
                pieceStore piecestore.PieceStore, 
                dataTransfer datatransfer.Manager, 
                spn storagemarket.StorageProviderNode, 
                minerAddress address.Address, 
                rt abi.RegisteredProof, 
                storedAsk StoredAsk, 
                options ...StorageProviderOption,
) (storagemarket.StorageProvider, error) {

Parameters

  • net network.StorageMarketNetwork is the same interface as for StorageClientNode
  • ds datastore.Batching is the same interface as for StorageClientNode
  • bs blockstore.Blockstore is the same interface as for StorageClientNode
  • fs filestore.FileStore is an instance of the filestore.FileStore struct from the go-fil-markets repo.
  • pieceStore piecestore.PieceStore is the database of deals and pieces associated with them. See this repo's piecestore module.
  • dataTransfer is the same interface as for StorageClientNode
  • spn storagemarket.StorageProviderNode is the implementation of the StorageProviderNode API that was written for your node.
  • minerAddress address.Address is the miner owner address.
  • rt abi.RegisteredProof is an int64 indicating the type of proof to use when generating a piece commitment (CommP). see github.com/filecoin-project/go-state-types/abi/sector.go for the list and meaning of accepted values.
  • storedAsk StoredAsk is an interface for getting and adding storage Asks. It is implemented in storagemarket. To create a StoredAsk: 
    package storedask
func NewStoredAsk(ds datastore.Batching, dsKey datastore.Key, spn storagemarket.StorageProviderNode, 
                  actor address.Address) (*StoredAsk, error)
  • options ...StorageProviderOption options is a variable length parameter to provide functions that change the StorageProvider default configuration. See provider.go for the available options.

Technical Documentation

  • GoDoc contains an architectural overview and robust API documentation

  • Storage Client FSM diagram:

Diagram of StorageClientFSM

  • Storage Provider FSM diagram:

Diagram of StorageClientFSM

Documentation

Overview

Package storagemarket implements the Filecoin storage protocol.

An overview of the storage protocol can be found in the Filecoin specification:

https://filecoin-project.github.io/specs/#systems__filecoin_markets__storage_market

The following architectural components provide a brief overview of the design of the storagemarket module:

Public Interfaces And Node Dependencies

A core goal of this module is to isolate the negotiation of deals from the actual chain operations performed by the node to put the deal on chain. The module primarily orchestrates the storage deal flow, rather than performing specific chain operations which are delegated to the node.

As such, for both the client and the provider in the storage market, the module defines a top level public interface which it provides an implementation for, and a node interface that must be implemented by the Filecoin node itself, and provided as a dependency. These node interfaces provide a universal way to talk to potentially multiple different Filecoin node implementations, and can be implemented using HTTP or some other interprocess communication to talk to a node implementation running in a different process.

The top level interfaces this package implements are StorageClient & StorageProvider. The dependencies the Filecoin node is expected to implement are StorageClientNode & StorageProviderNode. Further documentation of exactly what those dependencies should do can be found in the readme.

Finite State Machines and Resumability

Making deals in Filecoin is a highly asynchronous process. For a large piece of data, it's possible that the entire process of proposing a deal, transferring data, publishing the deal, putting the data in a sector and sealing it could take hours or even days. Not surprisingly, many things can go wrong along the way. To manage the process of orchestrating deals, we use finite state machines that update deal state when discrete events occur. State updates always persist state to disk. This means we have a permanent record of exactly what's going on with deals at any time, and we can ideally survive our Filecoin processes shutting down and restarting.

The following diagrams visualize the statemachine flows for the client and the provider:

Client FSM - https://raw.githubusercontent.com/brossetti1/go-fil-markets/master/docs/storageclient.mmd.svg

Provider FSM - https://raw.githubusercontent.com/brossetti1/go-fil-markets/master/docs/storageprovider.mmd.svg

Identifying Providers For A Deal

The StorageClient provides two functions to locate a provider with whom to make a deal:

`ListProviders` returns a list of storage providers on the Filecoin network. This list is assembled by querying the chain state for active storage miners.

`QueryAsk` queries a single provider for more specific details about the kinds of deals they accept, as expressed through a `StorageAsk`.

Deal Flow

The primary mechanism for initiating storage deals is the `ProposeStorageDeal` method on the StorageClient.

When `ProposeStorageDeal` is called, it constructs and signs a DealProposal, initiates tracking of deal state and hands the deal to the Client FSM, returning the CID of the DealProposal which constitutes the identifier for that deal.

After some preparation steps, the FSM will send the deal proposal to the StorageProvider, which receives the deal in `HandleDealStream`. `HandleDealStream` initiates tracking of deal state on the Provider side and hands the deal to the Provider FSM, which handles the rest of deal flow.

From this point forward, deal negotiation is completely asynchronous and runs in the FSMs.

A user of the modules can monitor deal progress through `SubscribeToEvents` methods on StorageClient and StorageProvider, or by simply calling `ListLocalDeals` to get all deal statuses.

The FSMs implement every step in deal negotiation up to deal publishing. However, adding the deal to a sector and sealing it is handled outside this module. When a deal is published, the StorageProvider calls `OnDealComplete` on the StorageProviderNode interface (the node itself likely delegates management of sectors and sealing to an implementation of the Storage Mining subsystem of the Filecoin spec). At this point, the markets implementations essentially shift to being monitors of deal progression: they wait to see and record when the deal becomes active and later expired or slashed.

When a deal becomes active on chain, the provider records the location of where it's stored in a sector in the PieceStore, so that it's available for retrieval.

Major Dependencies

Other libraries in go-fil-markets:

https://github.com/brossetti1/go-fil-markets/tree/master/filestore - used to store pieces and other temporary data before it's transferred to either a sector or the PieceStore.

https://github.com/brossetti1/go-fil-markets/tree/master/pieceio - used to convert back and forth between raw payload data and pieces that fit in sector. Also provides utilities for generating CommP.

https://github.com/brossetti1/go-fil-markets/tree/master/piecestore - used to write information about where data lives in sectors so that it can later be retrieved.

https://github.com/brossetti1/go-fil-markets/tree/master/shared - types and utility functions shared with retrievalmarket package.

Other Filecoin Repos:

https://github.com/filecoin-project/go-data-transfer - for transferring data, via go-graphsync

https://github.com/filecoin-project/go-statemachine - a finite state machine that tracks deal state

https://github.com/filecoin-project/go-storedcounter - for generating and persisting unique deal IDs

https://github.com/filecoin-project/specs-actors - the Filecoin actors

IPFS Project Repos:

https://github.com/ipfs/go-graphsync - used by go-data-transfer

https://github.com/ipfs/go-datastore - for persisting statemachine state for deals

https://github.com/ipfs/go-ipfs-blockstore - for storing and retrieving block data for deals

Other Repos:

https://github.com/libp2p/go-libp2p) the network over which retrieval deal data is exchanged.

https://github.com/hannahhoward/go-pubsub - for pub/sub notifications external to the statemachine

Root Package

This top level package defines top level enumerations and interfaces. The primary implementation lives in the `impl` directory

Index

Constants

View Source 
const (
	// StorageDealUnknown means the current status of a deal is undefined
	StorageDealUnknown = StorageDealStatus(iota)

	// StorageDealProposalNotFound is a status returned in responses when the deal itself cannot
	// be located
	StorageDealProposalNotFound

	// StorageDealProposalRejected is returned by a StorageProvider when it chooses not to accept
	// a DealProposal
	StorageDealProposalRejected

	// StorageDealProposalAccepted indicates an intent to accept a storage deal proposal
	StorageDealProposalAccepted

	// StorageDealStaged means a deal has been published and data is ready to be put into a sector
	StorageDealStaged

	// StorageDealSealing means a deal is in a sector that is being sealed
	StorageDealSealing

	// StorageDealFinalizing means a deal is in a sealed sector and we're doing final
	// housekeeping before marking it active
	StorageDealFinalizing

	// StorageDealActive means a deal is in a sealed sector and the miner is proving the data
	// for the deal
	StorageDealActive

	// StorageDealExpired means a deal has passed its final epoch and is expired
	StorageDealExpired

	// StorageDealSlashed means the deal was in a sector that got slashed from failing to prove
	StorageDealSlashed

	// StorageDealRejecting means the Provider has rejected the deal, and will send a rejection response
	StorageDealRejecting

	// StorageDealFailing means something has gone wrong in a deal. Once data is cleaned up the deal will finalize on
	// StorageDealError
	StorageDealFailing

	// StorageDealFundsReserved means we've deposited funds as necessary to create a deal, ready to move forward
	StorageDealFundsReserved

	// StorageDealCheckForAcceptance means the client is waiting for a provider to seal and publish a deal
	StorageDealCheckForAcceptance

	// StorageDealValidating means the provider is validating that deal parameters are good for a proposal
	StorageDealValidating

	// StorageDealAcceptWait means the provider is running any custom decision logic to decide whether or not to accept the deal
	StorageDealAcceptWait

	// StorageDealStartDataTransfer means data transfer is beginning
	StorageDealStartDataTransfer

	// StorageDealTransferring means data is being sent from the client to the provider via the data transfer module
	StorageDealTransferring

	// StorageDealWaitingForData indicates either a manual transfer
	// or that the provider has not received a data transfer request from the client
	StorageDealWaitingForData

	// StorageDealVerifyData means data has been transferred and we are attempting to verify it against the PieceCID
	StorageDealVerifyData

	// StorageDealReserveProviderFunds means that provider is making sure it has adequate funds for the deal in the StorageMarketActor
	StorageDealReserveProviderFunds

	// StorageDealReserveClientFunds means that client is making sure it has adequate funds for the deal in the StorageMarketActor
	StorageDealReserveClientFunds

	// StorageDealProviderFunding means that the provider has deposited funds in the StorageMarketActor and it is waiting
	// to see the funds appear in its balance
	StorageDealProviderFunding

	// StorageDealClientFunding means that the client has deposited funds in the StorageMarketActor and it is waiting
	// to see the funds appear in its balance
	StorageDealClientFunding

	// StorageDealPublish means the deal is ready to be published on chain
	StorageDealPublish

	// StorageDealPublishing means the deal has been published but we are waiting for it to appear on chain
	StorageDealPublishing

	// StorageDealError means the deal has failed due to an error, and no further updates will occur
	StorageDealError

	// StorageDealProviderTransferAwaitRestart means the provider has restarted while data
	// was being transferred from client to provider, and will wait for the client to
	// resume the transfer
	StorageDealProviderTransferAwaitRestart

	// StorageDealClientTransferRestart means a storage deal data transfer from client to provider will be restarted
	// by the client
	StorageDealClientTransferRestart

	// StorageDealAwaitingPreCommit means a deal is ready and must be pre-committed
	StorageDealAwaitingPreCommit

	// StorageDealTransferQueued means the data transfer request has been queued and will be executed soon.
	StorageDealTransferQueued
)
View Source 
const (
	// TTGraphsync means data for a deal will be transferred by graphsync
	TTGraphsync = "graphsync"

	// TTManual means data for a deal will be transferred manually and imported
	// on the provider
	TTManual = "manual"
)
View Source 
const AskProtocolID = "/fil/storage/ask/1.1.0"
View Source 
const DealProtocolID101 = "/fil/storage/mk/1.0.1"

The ID for the libp2p protocol for proposing storage deals.

View Source 
const DealProtocolID110 = "/fil/storage/mk/1.1.0"
View Source 
const DealProtocolID111 = "/fil/storage/mk/1.1.1"
View Source 
const DealStatusProtocolID = "/fil/storage/status/1.1.0"
View Source 
const OldAskProtocolID = "/fil/storage/ask/1.0.1"

AskProtocolID is the ID for the libp2p protocol for querying miners for their current StorageAsk.

View Source 
const OldDealStatusProtocolID = "/fil/storage/status/1.0.1"

DealStatusProtocolID is the ID for the libp2p protocol for querying miners for the current status of a deal.

Variables

View Source 
var ClientEvents = map[ClientEvent]string{
	ClientEventOpen:                       "ClientEventOpen",
	ClientEventReserveFundsFailed:         "ClientEventReserveFundsFailed",
	ClientEventFundingInitiated:           "ClientEventFundingInitiated",
	ClientEventFundsReserved:              "ClientEventFundsReserved",
	ClientEventFundsReleased:              "ClientEventFundsReleased",
	ClientEventFundingComplete:            "ClientEventFundingComplete",
	ClientEventWriteProposalFailed:        "ClientEventWriteProposalFailed",
	ClientEventInitiateDataTransfer:       "ClientEventInitiateDataTransfer",
	ClientEventDataTransferInitiated:      "ClientEventDataTransferInitiated",
	ClientEventDataTransferComplete:       "ClientEventDataTransferComplete",
	ClientEventWaitForDealState:           "ClientEventWaitForDealState",
	ClientEventDataTransferFailed:         "ClientEventDataTransferFailed",
	ClientEventReadResponseFailed:         "ClientEventReadResponseFailed",
	ClientEventResponseVerificationFailed: "ClientEventResponseVerificationFailed",
	ClientEventResponseDealDidNotMatch:    "ClientEventResponseDealDidNotMatch",
	ClientEventUnexpectedDealState:        "ClientEventUnexpectedDealState",
	ClientEventStreamCloseError:           "ClientEventStreamCloseError",
	ClientEventDealRejected:               "ClientEventDealRejected",
	ClientEventDealAccepted:               "ClientEventDealAccepted",
	ClientEventDealPublishFailed:          "ClientEventDealPublishFailed",
	ClientEventDealPublished:              "ClientEventDealPublished",
	ClientEventDealActivationFailed:       "ClientEventDealActivationFailed",
	ClientEventDealActivated:              "ClientEventDealActivated",
	ClientEventDealCompletionFailed:       "ClientEventDealCompletionFailed",
	ClientEventDealExpired:                "ClientEventDealExpired",
	ClientEventDealSlashed:                "ClientEventDealSlashed",
	ClientEventFailed:                     "ClientEventFailed",
	ClientEventRestart:                    "ClientEventRestart",
	ClientEventDataTransferRestarted:      "ClientEventDataTransferRestarted",
	ClientEventDataTransferRestartFailed:  "ClientEventDataTransferRestartFailed",
	ClientEventDataTransferStalled:        "ClientEventDataTransferStalled",
	ClientEventDataTransferCancelled:      "ClientEventDataTransferCancelled",
	ClientEventDataTransferQueued:         "ClientEventDataTransferQueued",
}

ClientEvents maps client event codes to string names

View Source 
var DealStates = map[StorageDealStatus]string{
	StorageDealUnknown:                      "StorageDealUnknown",
	StorageDealProposalNotFound:             "StorageDealProposalNotFound",
	StorageDealProposalRejected:             "StorageDealProposalRejected",
	StorageDealProposalAccepted:             "StorageDealProposalAccepted",
	StorageDealAcceptWait:                   "StorageDealAcceptWait",
	StorageDealStartDataTransfer:            "StorageDealStartDataTransfer",
	StorageDealStaged:                       "StorageDealStaged",
	StorageDealAwaitingPreCommit:            "StorageDealAwaitingPreCommit",
	StorageDealSealing:                      "StorageDealSealing",
	StorageDealActive:                       "StorageDealActive",
	StorageDealExpired:                      "StorageDealExpired",
	StorageDealSlashed:                      "StorageDealSlashed",
	StorageDealRejecting:                    "StorageDealRejecting",
	StorageDealFailing:                      "StorageDealFailing",
	StorageDealFundsReserved:                "StorageDealFundsReserved",
	StorageDealCheckForAcceptance:           "StorageDealCheckForAcceptance",
	StorageDealValidating:                   "StorageDealValidating",
	StorageDealTransferring:                 "StorageDealTransferring",
	StorageDealWaitingForData:               "StorageDealWaitingForData",
	StorageDealVerifyData:                   "StorageDealVerifyData",
	StorageDealReserveProviderFunds:         "StorageDealReserveProviderFunds",
	StorageDealReserveClientFunds:           "StorageDealReserveClientFunds",
	StorageDealProviderFunding:              "StorageDealProviderFunding",
	StorageDealClientFunding:                "StorageDealClientFunding",
	StorageDealPublish:                      "StorageDealPublish",
	StorageDealPublishing:                   "StorageDealPublishing",
	StorageDealError:                        "StorageDealError",
	StorageDealFinalizing:                   "StorageDealFinalizing",
	StorageDealClientTransferRestart:        "StorageDealClientTransferRestart",
	StorageDealProviderTransferAwaitRestart: "StorageDealProviderTransferAwaitRestart",
	StorageDealTransferQueued:               "StorageDealTransferQueued",
}

DealStates maps StorageDealStatus codes to string names

View Source 
var DealStatesDescriptions = map[StorageDealStatus]string{
	StorageDealUnknown:                      "Unknown",
	StorageDealProposalNotFound:             "Proposal not found",
	StorageDealProposalRejected:             "Proposal rejected",
	StorageDealProposalAccepted:             "Proposal accepted",
	StorageDealAcceptWait:                   "AcceptWait",
	StorageDealStartDataTransfer:            "Starting data transfer",
	StorageDealStaged:                       "Staged",
	StorageDealAwaitingPreCommit:            "Awaiting a PreCommit message on chain",
	StorageDealSealing:                      "Sealing",
	StorageDealActive:                       "Active",
	StorageDealExpired:                      "Expired",
	StorageDealSlashed:                      "Slashed",
	StorageDealRejecting:                    "Rejecting",
	StorageDealFailing:                      "Failing",
	StorageDealFundsReserved:                "FundsReserved",
	StorageDealCheckForAcceptance:           "Checking for deal acceptance",
	StorageDealValidating:                   "Validating",
	StorageDealTransferring:                 "Transferring",
	StorageDealWaitingForData:               "Waiting for data",
	StorageDealVerifyData:                   "Verifying data",
	StorageDealReserveProviderFunds:         "Reserving provider funds",
	StorageDealReserveClientFunds:           "Reserving client funds",
	StorageDealProviderFunding:              "Provider funding",
	StorageDealClientFunding:                "Client funding",
	StorageDealPublish:                      "Publish",
	StorageDealPublishing:                   "Publishing",
	StorageDealError:                        "Error",
	StorageDealFinalizing:                   "Finalizing",
	StorageDealClientTransferRestart:        "Client transfer restart",
	StorageDealProviderTransferAwaitRestart: "ProviderTransferAwaitRestart",
}

DealStatesDescriptions maps StorageDealStatus codes to string description for better UX

View Source 
var DealStatesDurations = map[StorageDealStatus]string{
	StorageDealUnknown:                      "",
	StorageDealProposalNotFound:             "",
	StorageDealProposalRejected:             "",
	StorageDealProposalAccepted:             "a few minutes",
	StorageDealAcceptWait:                   "a few minutes",
	StorageDealStartDataTransfer:            "a few minutes",
	StorageDealStaged:                       "a few minutes",
	StorageDealAwaitingPreCommit:            "a few minutes",
	StorageDealSealing:                      "a few hours",
	StorageDealActive:                       "",
	StorageDealExpired:                      "",
	StorageDealSlashed:                      "",
	StorageDealRejecting:                    "",
	StorageDealFailing:                      "",
	StorageDealFundsReserved:                "a few minutes",
	StorageDealCheckForAcceptance:           "a few minutes",
	StorageDealValidating:                   "a few minutes",
	StorageDealTransferring:                 "a few minutes",
	StorageDealWaitingForData:               "a few minutes",
	StorageDealVerifyData:                   "a few minutes",
	StorageDealReserveProviderFunds:         "a few minutes",
	StorageDealReserveClientFunds:           "a few minutes",
	StorageDealProviderFunding:              "a few minutes",
	StorageDealClientFunding:                "a few minutes",
	StorageDealPublish:                      "a few minutes",
	StorageDealPublishing:                   "a few minutes",
	StorageDealError:                        "",
	StorageDealFinalizing:                   "a few minutes",
	StorageDealClientTransferRestart:        "depending on data size, anywhere between a few minutes to a few hours",
	StorageDealProviderTransferAwaitRestart: "a few minutes",
}
View Source 
var ProviderEvents = map[ProviderEvent]string{
	ProviderEventOpen:                        "ProviderEventOpen",
	ProviderEventNodeErrored:                 "ProviderEventNodeErrored",
	ProviderEventDealRejected:                "ProviderEventDealRejected",
	ProviderEventRejectionSent:               "ProviderEventRejectionSent",
	ProviderEventDealAccepted:                "ProviderEventDealAccepted",
	ProviderEventDealDeciding:                "ProviderEventDealDeciding",
	ProviderEventInsufficientFunds:           "ProviderEventInsufficientFunds",
	ProviderEventFundsReserved:               "ProviderEventFundsReserved",
	ProviderEventFundsReleased:               "ProviderEventFundsReleased",
	ProviderEventFundingInitiated:            "ProviderEventFundingInitiated",
	ProviderEventFunded:                      "ProviderEventFunded",
	ProviderEventDataTransferFailed:          "ProviderEventDataTransferFailed",
	ProviderEventDataRequested:               "ProviderEventDataRequested",
	ProviderEventDataTransferInitiated:       "ProviderEventDataTransferInitiated",
	ProviderEventDataTransferCompleted:       "ProviderEventDataTransferCompleted",
	ProviderEventManualDataReceived:          "ProviderEventManualDataReceived",
	ProviderEventDataVerificationFailed:      "ProviderEventDataVerificationFailed",
	ProviderEventVerifiedData:                "ProviderEventVerifiedData",
	ProviderEventSendResponseFailed:          "ProviderEventSendResponseFailed",
	ProviderEventDealPublishInitiated:        "ProviderEventDealPublishInitiated",
	ProviderEventDealPublished:               "ProviderEventDealPublished",
	ProviderEventDealPublishError:            "ProviderEventDealPublishError",
	ProviderEventFileStoreErrored:            "ProviderEventFileStoreErrored",
	ProviderEventDealHandoffFailed:           "ProviderEventDealHandoffFailed",
	ProviderEventDealHandedOff:               "ProviderEventDealHandedOff",
	ProviderEventDealActivationFailed:        "ProviderEventDealActivationFailed",
	ProviderEventDealActivated:               "ProviderEventDealActivated",
	ProviderEventPieceStoreErrored:           "ProviderEventPieceStoreErrored",
	ProviderEventFinalized:                   "ProviderEventCleanupFinished",
	ProviderEventDealCompletionFailed:        "ProviderEventDealCompletionFailed",
	ProviderEventMultistoreErrored:           "ProviderEventMultistoreErrored",
	ProviderEventDealExpired:                 "ProviderEventDealExpired",
	ProviderEventDealSlashed:                 "ProviderEventDealSlashed",
	ProviderEventFailed:                      "ProviderEventFailed",
	ProviderEventTrackFundsFailed:            "ProviderEventTrackFundsFailed",
	ProviderEventRestart:                     "ProviderEventRestart",
	ProviderEventDataTransferRestarted:       "ProviderEventDataTransferRestarted",
	ProviderEventDataTransferRestartFailed:   "ProviderEventDataTransferRestartFailed",
	ProviderEventDataTransferStalled:         "ProviderEventDataTransferStalled",
	ProviderEventDataTransferCancelled:       "ProviderEventDataTransferCancelled",
	ProviderEventDealPrecommitFailed:         "ProviderEventDealPrecommitFailed",
	ProviderEventDealPrecommitted:            "ProviderEventDealPrecommitted",
	ProviderEventAwaitTransferRestartTimeout: "ProviderEventAwaitTransferRestartTimeout",
}

ProviderEvents maps provider event codes to string names

View Source 
var SignedStorageAskUndefined = SignedStorageAsk{}

SignedStorageAskUndefined represents the empty value for SignedStorageAsk

View Source 
var StorageAskUndefined = StorageAsk{}

StorageAskUndefined represents an empty value for StorageAsk

Functions

This section is empty.

Types

type Balance 

type Balance struct {
	Locked    abi.TokenAmount
	Available abi.TokenAmount
}

Balance represents a current balance of funds in the StorageMarketActor.

func (*Balance) MarshalCBOR 

func (t *Balance) MarshalCBOR(w io.Writer) error

func (*Balance) UnmarshalCBOR 

func (t *Balance) UnmarshalCBOR(r io.Reader) (err error)

type BlockstoreAccessor 

type BlockstoreAccessor interface {
	Get(PayloadCID) (bstore.Blockstore, error)
	Done(PayloadCID) error
}

BlockstoreAccessor is used by the storage market client to get a blockstore when needed, concretely to send the payload to the provider. This abstraction allows the caller to provider any blockstore implementation: a CARv2 file, an IPFS blockstore, or something else.

They key is a payload CID because this is the unique top-level key of a client-side data import.

type ClientDeal 

type ClientDeal struct {
	market.ClientDealProposal
	ProposalCid       cid.Cid
	AddFundsCid       *cid.Cid
	State             StorageDealStatus
	Miner             peer.ID
	MinerWorker       address.Address
	DealID            abi.DealID
	DataRef           *DataRef
	Message           string
	DealStages        *DealStages
	PublishMessage    *cid.Cid
	SlashEpoch        abi.ChainEpoch
	PollRetryCount    uint64
	PollErrorCount    uint64
	FastRetrieval     bool
	FundsReserved     abi.TokenAmount
	CreationTime      cbg.CborTime
	TransferChannelID *datatransfer.ChannelID
	SectorNumber      abi.SectorNumber
}

ClientDeal is the local state tracked for a deal by a StorageClient

func (*ClientDeal) AddLog 

func (d *ClientDeal) AddLog(msg string, a ...interface{})

AddLog adds a log inside the DealStages object of the deal. EXPERIMENTAL; subject to change.

func (*ClientDeal) MarshalCBOR 

func (t *ClientDeal) MarshalCBOR(w io.Writer) error

func (*ClientDeal) UnmarshalCBOR 

func (t *ClientDeal) UnmarshalCBOR(r io.Reader) (err error)

type ClientDealProposal 

type ClientDealProposal = market.ClientDealProposal

type ClientEvent 

type ClientEvent uint64

ClientEvent is an event that happens in the client's deal state machine 

const (
	// ClientEventOpen indicates a new deal was started
	ClientEventOpen ClientEvent = iota

	// ClientEventReserveFundsFailed happens when attempting to reserve funds for a deal fails
	ClientEventReserveFundsFailed

	// ClientEventFundingInitiated happens when a client has sent a message adding funds to its balance
	ClientEventFundingInitiated

	// ClientEventFundsReserved happens when a client reserves funds for a deal (updating our tracked funds)
	ClientEventFundsReserved

	// ClientEventFundsReleased happens when a client released funds for a deal (updating our tracked funds)
	ClientEventFundsReleased

	// ClientEventFundingComplete happens when a client successfully reserves funds for a deal
	ClientEventFundingComplete

	// ClientEventWriteProposalFailed indicates an attempt to send a deal proposal to a provider failed
	ClientEventWriteProposalFailed

	// ClientEventInitiateDataTransfer happens when a a client is ready to transfer data to a provider
	ClientEventInitiateDataTransfer

	// ClientEventDataTransferInitiated happens when piece data transfer has started
	ClientEventDataTransferInitiated

	// ClientEventDataTransferRestarted happens when a data transfer from client to provider is restarted by the client
	ClientEventDataTransferRestarted

	// ClientEventDataTransferComplete happens when piece data transfer has been completed
	ClientEventDataTransferComplete

	// ClientEventWaitForDealState happens when the client needs to continue waiting for an actionable deal state
	ClientEventWaitForDealState

	// ClientEventDataTransferFailed happens the client can't initiate a push data transfer to the provider
	ClientEventDataTransferFailed

	// ClientEventDataTransferRestartFailed happens when the client can't restart an existing data transfer
	ClientEventDataTransferRestartFailed

	// ClientEventReadResponseFailed means a network error occurred reading a deal response
	ClientEventReadResponseFailed

	// ClientEventResponseVerificationFailed means a response was not verified
	ClientEventResponseVerificationFailed

	// ClientEventResponseDealDidNotMatch means a response was sent for the wrong deal
	ClientEventResponseDealDidNotMatch

	// ClientEventUnexpectedDealState means a response was sent but the state wasn't what we expected
	ClientEventUnexpectedDealState

	// ClientEventStreamCloseError happens when an attempt to close a deals stream fails
	ClientEventStreamCloseError

	// ClientEventDealRejected happens when the provider does not accept a deal
	ClientEventDealRejected

	// ClientEventDealAccepted happens when a client receives a response accepting a deal from a provider
	ClientEventDealAccepted

	// ClientEventDealPublishFailed happens when a client cannot verify a deal was published
	ClientEventDealPublishFailed

	// ClientEventDealPublished happens when a deal is successfully published
	ClientEventDealPublished

	// ClientEventDealPrecommitFailed happens when an error occurs waiting for deal pre-commit
	ClientEventDealPrecommitFailed

	// ClientEventDealPrecommitted happens when a deal is successfully pre-commited
	ClientEventDealPrecommitted

	// ClientEventDealActivationFailed happens when a client cannot verify a deal was activated
	ClientEventDealActivationFailed

	// ClientEventDealActivated happens when a deal is successfully activated
	ClientEventDealActivated

	// ClientEventDealCompletionFailed happens when a client cannot verify a deal expired or was slashed
	ClientEventDealCompletionFailed

	// ClientEventDealExpired happens when a deal expires
	ClientEventDealExpired

	// ClientEventDealSlashed happens when a deal is slashed
	ClientEventDealSlashed

	// ClientEventFailed happens when a deal terminates in failure
	ClientEventFailed

	// ClientEventRestart is used to resume the deal after a state machine shutdown
	ClientEventRestart

	// ClientEventDataTransferStalled happens when the clients data transfer experiences a disconnect
	ClientEventDataTransferStalled

	// ClientEventDataTransferCancelled happens when a data transfer is cancelled
	ClientEventDataTransferCancelled

	// ClientEventDataTransferQueued happens when we queue the provider's request to transfer data to it
	// in response to the push request we send to the provider.
	ClientEventDataTransferQueued
)

func (ClientEvent) String 

func (e ClientEvent) String() string

type ClientSubscriber 

type ClientSubscriber func(event ClientEvent, deal ClientDeal)

ClientSubscriber is a callback that is run when events are emitted on a StorageClient

type DataRef 

type DataRef struct {
	TransferType string
	Root         cid.Cid

	PieceCid     *cid.Cid              // Optional for non-manual transfer, will be recomputed from the data if not given
	PieceSize    abi.UnpaddedPieceSize // Optional for non-manual transfer, will be recomputed from the data if not given
	RawBlockSize uint64                // Optional: used as the denominator when calculating transfer %
}

DataRef is a reference for how data will be transferred for a given storage deal

func (*DataRef) MarshalCBOR 

func (t *DataRef) MarshalCBOR(w io.Writer) error

func (*DataRef) UnmarshalCBOR 

func (t *DataRef) UnmarshalCBOR(r io.Reader) (err error)

type DealExpiredCallback 

type DealExpiredCallback func(err error)

DealExpiredCallback is a callback that runs when a deal expires

type DealSectorCommittedCallback 

type DealSectorCommittedCallback func(err error)

DealSectorCommittedCallback is a callback that runs when a sector is committed

type DealSectorPreCommittedCallback 

type DealSectorPreCommittedCallback func(sectorNumber abi.SectorNumber, isActive bool, err error)

DealSectorPreCommittedCallback is a callback that runs when a sector is pre-committed sectorNumber: the number of the sector that the deal is in isActive: the deal is already active

type DealSlashedCallback 

type DealSlashedCallback func(slashEpoch abi.ChainEpoch, err error)

DealSlashedCallback is a callback that runs when a deal gets slashed

type DealStage 

type DealStage struct {
	// Human-readable fields.
	// TODO: these _will_ need to be converted to canonical representations, so
	//  they are machine readable.
	Name             string
	Description      string
	ExpectedDuration string

	// Timestamps.
	// TODO: may be worth adding an exit timestamp. It _could_ be inferred from
	//  the start of the next stage, or from the timestamp of the last log line
	//  if this is a terminal stage. But that's non-determistic and it relies on
	//  assumptions.
	CreatedTime cbg.CborTime
	UpdatedTime cbg.CborTime

	// Logs contains a detailed timeline of events that occurred inside
	// this stage.
	Logs []*Log
}

DealStages captures data about the execution of a deal stage. EXPERIMENTAL; subject to change.

func (*DealStage) MarshalCBOR 

func (t *DealStage) MarshalCBOR(w io.Writer) error

func (*DealStage) UnmarshalCBOR 

func (t *DealStage) UnmarshalCBOR(r io.Reader) (err error)

type DealStages 

type DealStages struct {
	// Stages contains an entry for every stage that the deal has gone through.
	// Each stage then contains logs.
	Stages []*DealStage
}

DealStages captures a timeline of the progress of a deal, grouped by stages. EXPERIMENTAL; subject to change.

func NewDealStages 

func NewDealStages() *DealStages

NewDealStages creates a new DealStages object ready to be used. EXPERIMENTAL; subject to change.

func (*DealStages) AddStageLog 

func (ds *DealStages) AddStageLog(stage, description, expectedDuration, msg string)

AddStageLog adds a log to the specified stage, creating the stage if it doesn't exist yet. EXPERIMENTAL; subject to change.

func (*DealStages) GetStage 

func (ds *DealStages) GetStage(stage string) *DealStage

GetStage returns the DealStage object for a named stage, or nil if not found.

TODO: the input should be a strongly-typed enum instead of a free-form string. TODO: drop Get from GetStage to make this code more idiomatic. Return a second ok boolean to make it even more idiomatic. EXPERIMENTAL; subject to change.

func (*DealStages) MarshalCBOR 

func (t *DealStages) MarshalCBOR(w io.Writer) error

func (*DealStages) UnmarshalCBOR 

func (t *DealStages) UnmarshalCBOR(r io.Reader) (err error)

type Log 

type Log struct {
	// Log is a human readable message.
	//
	// TODO: this _may_ need to be converted to a canonical data model so it
	//  is machine-readable.
	Log string

	UpdatedTime cbg.CborTime
}

Log represents a point-in-time event that occurred inside a deal stage. EXPERIMENTAL; subject to change.

func (*Log) MarshalCBOR 

func (t *Log) MarshalCBOR(w io.Writer) error

func (*Log) UnmarshalCBOR 

func (t *Log) UnmarshalCBOR(r io.Reader) (err error)

type MinerDeal 

type MinerDeal struct {
	ClientDealProposal
	ProposalCid           cid.Cid
	AddFundsCid           *cid.Cid
	PublishCid            *cid.Cid
	Miner                 peer.ID
	Client                peer.ID
	State                 StorageDealStatus
	PiecePath             filestore.Path
	MetadataPath          filestore.Path
	SlashEpoch            abi.ChainEpoch
	FastRetrieval         bool
	Message               string
	FundsReserved         abi.TokenAmount
	Ref                   *DataRef
	AvailableForRetrieval bool

	DealID       abi.DealID
	CreationTime cbg.CborTime

	TransferChannelId *datatransfer.ChannelID
	SectorNumber      abi.SectorNumber

	InboundCAR string
}

MinerDeal is the local state tracked for a deal by a StorageProvider

func (*MinerDeal) MarshalCBOR 

func (t *MinerDeal) MarshalCBOR(w io.Writer) error

func (*MinerDeal) UnmarshalCBOR 

func (t *MinerDeal) UnmarshalCBOR(r io.Reader) (err error)

type PackingResult 

type PackingResult struct {
	SectorNumber abi.SectorNumber
	Offset       abi.PaddedPieceSize
	Size         abi.PaddedPieceSize
}

PackingResult returns information about how a deal was put into a sector

type PayloadCID 

type PayloadCID = cid.Cid

type ProposeStorageDealParams 

type ProposeStorageDealParams struct {
	Addr          address.Address
	Info          *StorageProviderInfo
	Data          *DataRef
	StartEpoch    abi.ChainEpoch
	EndEpoch      abi.ChainEpoch
	Price         abi.TokenAmount
	Collateral    abi.TokenAmount
	Rt            abi.RegisteredSealProof
	FastRetrieval bool
	VerifiedDeal  bool
}

ProposeStorageDealParams describes the parameters for proposing a storage deal

type ProposeStorageDealResult 

type ProposeStorageDealResult struct {
	ProposalCid cid.Cid
}

ProposeStorageDealResult returns the result for a proposing a deal

type ProviderDealState 

type ProviderDealState struct {
	State         StorageDealStatus
	Message       string
	Proposal      *market.DealProposal
	ProposalCid   *cid.Cid
	AddFundsCid   *cid.Cid
	PublishCid    *cid.Cid
	DealID        abi.DealID
	FastRetrieval bool
}

ProviderDealState represents a Provider's current state of a deal

func (*ProviderDealState) MarshalCBOR 

func (t *ProviderDealState) MarshalCBOR(w io.Writer) error

func (*ProviderDealState) UnmarshalCBOR 

func (t *ProviderDealState) UnmarshalCBOR(r io.Reader) (err error)

type ProviderEvent 

type ProviderEvent uint64

ProviderEvent is an event that happens in the provider's deal state machine 

const (
	// ProviderEventOpen indicates a new deal proposal has been received
	ProviderEventOpen ProviderEvent = iota

	// ProviderEventNodeErrored indicates an error happened talking to the node implementation
	ProviderEventNodeErrored

	// ProviderEventDealDeciding happens when a deal is being decided on by the miner
	ProviderEventDealDeciding

	// ProviderEventDealRejected happens when a deal proposal is rejected for not meeting criteria
	ProviderEventDealRejected

	// ProviderEventRejectionSent happens after a deal proposal rejection has been sent to the client
	ProviderEventRejectionSent

	// ProviderEventDealAccepted happens when a deal is accepted based on provider criteria
	ProviderEventDealAccepted

	// ProviderEventInsufficientFunds indicates not enough funds available for a deal
	ProviderEventInsufficientFunds

	// ProviderEventFundsReserved indicates we've reserved funds for a deal, adding to our overall total
	ProviderEventFundsReserved

	// ProviderEventFundsReleased indicates we've released funds for a deal
	ProviderEventFundsReleased

	// ProviderEventFundingInitiated indicates provider collateral funding has been initiated
	ProviderEventFundingInitiated

	// ProviderEventFunded indicates provider collateral has appeared in the storage market balance
	ProviderEventFunded

	// ProviderEventDataTransferFailed happens when an error occurs transferring data
	ProviderEventDataTransferFailed

	// ProviderEventDataRequested happens when a provider requests data from a client
	ProviderEventDataRequested

	// ProviderEventDataTransferInitiated happens when a data transfer starts
	ProviderEventDataTransferInitiated

	// ProviderEventDataTransferRestarted happens when a data transfer restarts
	ProviderEventDataTransferRestarted

	// ProviderEventDataTransferCompleted happens when a data transfer is successful
	ProviderEventDataTransferCompleted

	// ProviderEventManualDataReceived happens when data is received manually for an offline deal
	ProviderEventManualDataReceived

	// ProviderEventDataVerificationFailed happens when an error occurs validating deal data
	ProviderEventDataVerificationFailed

	// ProviderEventVerifiedData happens when received data is verified as matching the pieceCID in a deal proposal
	ProviderEventVerifiedData

	// ProviderEventSendResponseFailed happens when a response cannot be sent to a deal
	ProviderEventSendResponseFailed

	// ProviderEventDealPublishInitiated happens when a provider has sent a PublishStorageDeals message to the chain
	ProviderEventDealPublishInitiated

	// ProviderEventDealPublished happens when a deal is successfully published
	ProviderEventDealPublished

	// ProviderEventDealPublishError happens when PublishStorageDeals returns a non-ok exit code
	ProviderEventDealPublishError

	// ProviderEventFileStoreErrored happens when an error occurs accessing the filestore
	ProviderEventFileStoreErrored

	// ProviderEventDealHandoffFailed happens when an error occurs handing off a deal with OnDealComplete
	ProviderEventDealHandoffFailed

	// ProviderEventDealHandedOff happens when a deal is successfully handed off to the node for processing in a sector
	ProviderEventDealHandedOff

	// ProviderEventDealPrecommitFailed happens when an error occurs waiting for deal pre-commit
	ProviderEventDealPrecommitFailed

	// ProviderEventDealPrecommitted happens when a deal is successfully pre-commited
	ProviderEventDealPrecommitted

	// ProviderEventDealActivationFailed happens when an error occurs activating a deal
	ProviderEventDealActivationFailed

	// ProviderEventDealActivated happens when a deal is successfully activated and commited to a sector
	ProviderEventDealActivated

	// ProviderEventPieceStoreErrored happens when an attempt to save data in the piece store errors
	ProviderEventPieceStoreErrored

	// ProviderEventFinalized happens when final housekeeping is complete and a deal is active
	ProviderEventFinalized

	// ProviderEventDealCompletionFailed happens when a miner cannot verify a deal expired or was slashed
	ProviderEventDealCompletionFailed

	// ProviderEventMultistoreErrored indicates an error happened with a store for a deal
	ProviderEventMultistoreErrored

	// ProviderEventDealExpired happens when a deal expires
	ProviderEventDealExpired

	// ProviderEventDealSlashed happens when a deal is slashed
	ProviderEventDealSlashed

	// ProviderEventFailed indicates a deal has failed and should no longer be processed
	ProviderEventFailed

	// ProviderEventTrackFundsFailed indicates a failure trying to locally track funds needed for deals
	ProviderEventTrackFundsFailed

	// ProviderEventRestart is used to resume the deal after a state machine shutdown
	ProviderEventRestart

	// ProviderEventDataTransferRestartFailed means a data transfer that was restarted by the provider failed
	// Deprecated: this event is no longer used
	ProviderEventDataTransferRestartFailed

	// ProviderEventDataTransferStalled happens when the providers data transfer experiences a disconnect
	ProviderEventDataTransferStalled

	// ProviderEventDataTransferCancelled happens when a data transfer is cancelled
	ProviderEventDataTransferCancelled

	// ProviderEventAwaitTransferRestartTimeout is dispatched after a certain amount of time a provider has been
	// waiting for a data transfer to restart. If transfer hasn't restarted, the provider will fail the deal
	ProviderEventAwaitTransferRestartTimeout
)

func (ProviderEvent) String 

func (e ProviderEvent) String() string

type ProviderSubscriber 

type ProviderSubscriber func(event ProviderEvent, deal MinerDeal)

ProviderSubscriber is a callback that is run when events are emitted on a StorageProvider

type PublishDealsWaitResult 

type PublishDealsWaitResult struct {
	DealID   abi.DealID
	FinalCid cid.Cid
}

PublishDealsWaitResult is the result of a call to wait for publish deals to appear on chain

type SignedStorageAsk 

type SignedStorageAsk struct {
	Ask       *StorageAsk
	Signature *crypto.Signature
}

SignedStorageAsk is an ask signed by the miner's private key

func (*SignedStorageAsk) MarshalCBOR 

func (t *SignedStorageAsk) MarshalCBOR(w io.Writer) error

func (*SignedStorageAsk) UnmarshalCBOR 

func (t *SignedStorageAsk) UnmarshalCBOR(r io.Reader) (err error)

type StorageAsk 

type StorageAsk struct {
	// Price per GiB / Epoch
	Price         abi.TokenAmount
	VerifiedPrice abi.TokenAmount

	MinPieceSize abi.PaddedPieceSize
	MaxPieceSize abi.PaddedPieceSize
	Miner        address.Address
	Timestamp    abi.ChainEpoch
	Expiry       abi.ChainEpoch
	SeqNo        uint64
}

StorageAsk defines the parameters by which a miner will choose to accept or reject a deal. Note: making a storage deal proposal which matches the miner's ask is a precondition, but not sufficient to ensure the deal is accepted (the storage provider may run its own decision logic).

func (*StorageAsk) MarshalCBOR 

func (t *StorageAsk) MarshalCBOR(w io.Writer) error

func (*StorageAsk) UnmarshalCBOR 

func (t *StorageAsk) UnmarshalCBOR(r io.Reader) (err error)

type StorageAskOption 

type StorageAskOption func(*StorageAsk)

StorageAskOption allows custom configuration of a storage ask

func MaxPieceSize 

func MaxPieceSize(maxPieceSize abi.PaddedPieceSize) StorageAskOption

MaxPieceSize configures maxiumum piece size of a StorageAsk

func MinPieceSize 

func MinPieceSize(minPieceSize abi.PaddedPieceSize) StorageAskOption

MinPieceSize configures a minimum piece size of a StorageAsk

type StorageClient 

type StorageClient interface {

	// Start initializes deal processing on a StorageClient and restarts
	// in progress deals
	Start(ctx context.Context) error

	// OnReady registers a listener for when the client comes on line
	OnReady(shared.ReadyFunc)

	// Stop ends deal processing on a StorageClient
	Stop() error

	// ListProviders queries chain state and returns active storage providers
	ListProviders(ctx context.Context) (<-chan StorageProviderInfo, error)

	// ListLocalDeals lists deals initiated by this storage client
	ListLocalDeals(ctx context.Context) ([]ClientDeal, error)

	// GetLocalDeal lists deals that are in progress or rejected
	GetLocalDeal(ctx context.Context, cid cid.Cid) (ClientDeal, error)

	// GetAsk returns the current ask for a storage provider
	GetAsk(ctx context.Context, info StorageProviderInfo) (*StorageAsk, error)

	// GetProviderDealState queries a provider for the current state of a client's deal
	GetProviderDealState(ctx context.Context, proposalCid cid.Cid) (*ProviderDealState, error)

	// ProposeStorageDeal initiates deal negotiation with a Storage Provider
	ProposeStorageDeal(ctx context.Context, params ProposeStorageDealParams) (*ProposeStorageDealResult, error)

	// GetPaymentEscrow returns the current funds available for deal payment
	GetPaymentEscrow(ctx context.Context, addr address.Address) (Balance, error)

	// AddStorageCollateral adds storage collateral
	AddPaymentEscrow(ctx context.Context, addr address.Address, amount abi.TokenAmount) error

	// SubscribeToEvents listens for events that happen related to storage deals on a provider
	SubscribeToEvents(subscriber ClientSubscriber) shared.Unsubscribe
}

StorageClient is a client interface for making storage deals with a StorageProvider

type StorageClientNode 

type StorageClientNode interface {
	StorageCommon

	// GetStorageProviders returns information about known miners
	ListStorageProviders(ctx context.Context, tok shared.TipSetToken) ([]*StorageProviderInfo, error)

	// ValidatePublishedDeal verifies a deal is published on chain and returns the dealID
	ValidatePublishedDeal(ctx context.Context, deal ClientDeal) (abi.DealID, error)

	// SignProposal signs a DealProposal
	SignProposal(ctx context.Context, signer address.Address, proposal market.DealProposal) (*market.ClientDealProposal, error)

	// GetDefaultWalletAddress returns the address for this client
	GetDefaultWalletAddress(ctx context.Context) (address.Address, error)

	// GetMinerInfo returns info for a single miner with the given address
	GetMinerInfo(ctx context.Context, maddr address.Address, tok shared.TipSetToken) (*StorageProviderInfo, error)
}

StorageClientNode are node dependencies for a StorageClient

type StorageCommon 

type StorageCommon interface {

	// GetChainHead returns a tipset token for the current chain head
	GetChainHead(ctx context.Context) (shared.TipSetToken, abi.ChainEpoch, error)

	// Adds funds with the StorageMinerActor for a storage participant.  Used by both providers and clients.
	AddFunds(ctx context.Context, addr address.Address, amount abi.TokenAmount) (cid.Cid, error)

	// ReserveFunds reserves the given amount of funds is ensures it is available for the deal
	ReserveFunds(ctx context.Context, wallet, addr address.Address, amt abi.TokenAmount) (cid.Cid, error)

	// ReleaseFunds releases funds reserved with ReserveFunds
	ReleaseFunds(ctx context.Context, addr address.Address, amt abi.TokenAmount) error

	// GetBalance returns locked/unlocked for a storage participant.  Used by both providers and clients.
	GetBalance(ctx context.Context, addr address.Address, tok shared.TipSetToken) (Balance, error)

	// VerifySignature verifies a given set of data was signed properly by a given address's private key
	VerifySignature(ctx context.Context, signature crypto.Signature, signer address.Address, plaintext []byte, tok shared.TipSetToken) (bool, error)

	// WaitForMessage waits until a message appears on chain. If it is already on chain, the callback is called immediately
	WaitForMessage(ctx context.Context, mcid cid.Cid, onCompletion func(exitcode.ExitCode, []byte, cid.Cid, error) error) error

	// SignsBytes signs the given data with the given address's private key
	SignBytes(ctx context.Context, signer address.Address, b []byte) (*crypto.Signature, error)

	// DealProviderCollateralBounds returns the min and max collateral a storage provider can issue.
	DealProviderCollateralBounds(ctx context.Context, size abi.PaddedPieceSize, isVerified bool) (abi.TokenAmount, abi.TokenAmount, error)

	// OnDealSectorPreCommitted waits for a deal's sector to be pre-committed
	OnDealSectorPreCommitted(ctx context.Context, provider address.Address, dealID abi.DealID, proposal market.DealProposal, publishCid *cid.Cid, cb DealSectorPreCommittedCallback) error

	// OnDealSectorCommitted waits for a deal's sector to be sealed and proved, indicating the deal is active
	OnDealSectorCommitted(ctx context.Context, provider address.Address, dealID abi.DealID, sectorNumber abi.SectorNumber, proposal market.DealProposal, publishCid *cid.Cid, cb DealSectorCommittedCallback) error

	// OnDealExpiredOrSlashed registers callbacks to be called when the deal expires or is slashed
	OnDealExpiredOrSlashed(ctx context.Context, dealID abi.DealID, onDealExpired DealExpiredCallback, onDealSlashed DealSlashedCallback) error
}

StorageCommon are common interfaces provided by a filecoin Node to both StorageClient and StorageProvider

type StorageDealStatus 

type StorageDealStatus = uint64

StorageDealStatus is the local status of a StorageDeal. Note: this status has meaning in the context of this module only - it is not recorded on chain

type StorageProvider 

type StorageProvider interface {

	// Start initializes deal processing on a StorageProvider and restarts in progress deals.
	// It also registers the provider with a StorageMarketNetwork so it can receive incoming
	// messages on the storage market's libp2p protocols
	Start(ctx context.Context) error

	// OnReady registers a listener for when the provider comes on line
	OnReady(shared.ReadyFunc)

	// Stop terminates processing of deals on a StorageProvider
	Stop() error

	// SetAsk configures the storage miner's ask with the provided prices (for unverified and verified deals),
	// duration, and options. Any previously-existing ask is replaced.
	SetAsk(price abi.TokenAmount, verifiedPrice abi.TokenAmount, duration abi.ChainEpoch, options ...StorageAskOption) error

	// GetAsk returns the storage miner's ask, or nil if one does not exist.
	GetAsk() *SignedStorageAsk

	// GetLocalDeal gets a deal by signed proposal cid
	GetLocalDeal(cid cid.Cid) (MinerDeal, error)

	// LocalDealCount gets the number of local deals
	LocalDealCount() (int, error)

	// ListLocalDeals lists deals processed by this storage provider
	ListLocalDeals() ([]MinerDeal, error)

	// ListLocalDealsPage lists deals by creation time descending, starting
	// at the deal with the given signed proposal cid, skipping offset deals
	// and returning up to limit deals
	ListLocalDealsPage(startPropCid *cid.Cid, offset int, limit int) ([]MinerDeal, error)

	// AddStorageCollateral adds storage collateral
	AddStorageCollateral(ctx context.Context, amount abi.TokenAmount) error

	// GetStorageCollateral returns the current collateral balance
	GetStorageCollateral(ctx context.Context) (Balance, error)

	// ImportDataForDeal manually imports data for an offline storage deal
	ImportDataForDeal(ctx context.Context, propCid cid.Cid, data io.Reader) error

	// SubscribeToEvents listens for events that happen related to storage deals on a provider
	SubscribeToEvents(subscriber ProviderSubscriber) shared.Unsubscribe

	RetryDealPublishing(propCid cid.Cid) error

	AnnounceDealToIndexer(ctx context.Context, proposalCid cid.Cid) error

	AnnounceAllDealsToIndexer(ctx context.Context) error
}

StorageProvider provides an interface to the storage market for a single storage miner.

type StorageProviderInfo 

type StorageProviderInfo struct {
	Address    address.Address // actor address
	Owner      address.Address
	Worker     address.Address // signs messages
	SectorSize uint64
	PeerID     peer.ID
	Addrs      []ma.Multiaddr
}

StorageProviderInfo describes on chain information about a StorageProvider (use QueryAsk to determine more specific deal parameters)

type StorageProviderNode 

type StorageProviderNode interface {
	StorageCommon

	// PublishDeals publishes a deal on chain, returns the message cid, but does not wait for message to appear
	PublishDeals(ctx context.Context, deal MinerDeal) (cid.Cid, error)

	// WaitForPublishDeals waits for a deal publish message to land on chain.
	WaitForPublishDeals(ctx context.Context, mcid cid.Cid, proposal market.DealProposal) (*PublishDealsWaitResult, error)

	// OnDealComplete is called when a deal is complete and on chain, and data has been transferred and is ready to be added to a sector
	OnDealComplete(ctx context.Context, deal MinerDeal, pieceSize abi.UnpaddedPieceSize, pieceReader shared.ReadSeekStarter) (*PackingResult, error)

	// GetMinerWorkerAddress returns the worker address associated with a miner
	GetMinerWorkerAddress(ctx context.Context, addr address.Address, tok shared.TipSetToken) (address.Address, error)

	// GetDataCap gets the current data cap for addr
	GetDataCap(ctx context.Context, addr address.Address, tok shared.TipSetToken) (*verifreg.DataCap, error)

	// GetProofType gets the current seal proof type for the given miner.
	GetProofType(ctx context.Context, addr address.Address, tok shared.TipSetToken) (abi.RegisteredSealProof, error)
}

StorageProviderNode are node dependencies for a StorageProvider

Source Files

View all Source files

Directories

Path Synopsis
Package storageimpl provides the primary implementation of storage market top level interfaces
 Package storageimpl provides the primary implementation of storage market top level interfaces
blockrecorder
Package blockrecorder provides utilits to record locations of CIDs to a temporary metadata file, since writing a CAR happens BEFORE we actually hand off for sealing.
 Package blockrecorder provides utilits to record locations of CIDs to a temporary metadata file, since writing a CAR happens BEFORE we actually hand off for sealing.
clientstates
Package clientstates contains state machine logic relating to the `StorageMarket`.
 Package clientstates contains state machine logic relating to the `StorageMarket`.
clientutils
Package clientutils provides utility functions for the storage client & client FSM
 Package clientutils provides utility functions for the storage client & client FSM
connmanager
Package connmanager tracks open connections maping storage proposal CID -> StorageDealStream
 Package connmanager tracks open connections maping storage proposal CID -> StorageDealStream
dtutils
Package dtutils provides event listeners for the client and provider to listen for events on the data transfer module and dispatch FSM events based on them
 Package dtutils provides event listeners for the client and provider to listen for events on the data transfer module and dispatch FSM events based on them
providerstates
Package providerstates contains state machine logic relating to the `StorageProvider`.
 Package providerstates contains state machine logic relating to the `StorageProvider`.
providerutils
Package providerutils provides utility functions for the storage provider & provider FSM
 Package providerutils provides utility functions for the storage provider & provider FSM
requestvalidation
Package requestvalidation implements a request validator for the data transfer module to validate data transfer requests for storage deals
 Package requestvalidation implements a request validator for the data transfer module to validate data transfer requests for storage deals
storedask
Package network providers an abstraction over a libp2p host for managing storage markets's Libp2p protocols:
 Package network providers an abstraction over a libp2p host for managing storage markets's Libp2p protocols:
dependencies
Package testnodes contains stubbed implementations of the StorageProviderNode and StorageClientNode interface to simulate communications with a filecoin node
 Package testnodes contains stubbed implementations of the StorageProviderNode and StorageClientNode interface to simulate communications with a filecoin node

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.