retrievalmarket

package
v0.0.0-...-f311442 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 7, 2023 License: Apache-2.0, MIT Imports: 30 Imported by: 0

README

retrievalmarket

The retrievalmarket module is intended for Filecoin node implementations written in Go. It implements functionality to allow execution of retrieval market deals on the Filecoin network. The node implementation must provide access to chain operations, and persistent data storage.

Table of Contents

Background reading

Please see the Filecoin Retrieval Market Specification.

Installation

The build process for retrievalmarket requires Go >= v1.13.

To install:

go get github.com/filecoin-project/go-fil-markets/retrievalmarket

Operation

The retrievalmarket package provides high level APIs to execute data retrieval deals between a retrieval client and a retrieval provider (a.k.a. retrieval miner) on the Filecoin netwwork. The node must implement the PeerResolver, RetrievalProviderNode, and RetrievalClientNode interfaces in order to construct and use the module.

Deals are expected to survive a node restart; deals and related information are expected to be stored on disk.

retrievalmarket communicates its deal operations and requested data via go-data-transfer using go-graphsync.

Once required Node APIs are implemented and the retrievalmarket APIs are exposed to your desired consumers (such as a command-line or web interface), a retrieval from the client side could proceed roughly like so:

  1. Your node has a record of data with payloadCIDs and their respective pieceCIDs. Someone, possibly you, wants to retrieve data referenced by paylaodCID.
  2. It calls PeerResolver.GetPeers to obtain a list of retrieval providers storing data referenced by payloadCID.
  3. It obtains retrieval deal terms by calling each retrieval miners' Query function.
  4. The node selects the best terms for a retrieval deal and initiates a deal by calling the retrieval client's Retrieve function with the selected retrieval miner and piece info.
  5. The deal then proceeds automatically until all the data is returned and full payment in the form of vouchers is made to the retrieval provider, or the deal errors.
  6. Once the deal is complete and the final payment voucher is posted to chain, your client account balance will be adjusted according to the terms of the deal.

A retrieval from the provider side is more automated; the RetrievalProvider would be listening for retrieval Query and Retrieve requests, and respond accordingly.

  1. Your node stores a record of what it has stored locally, or possibly a record of peers with data.
  2. Your node receives a Query for payloadCID and responds automatically with the terms you the node operator have set for retrieval deals.
  3. Your node receives a DealProposal for retrieval, and automatically validates and accepts or rejects it. If accepted, the deal proceeds and your node begins sending data in pieces, stopping every so often to request another voucher for a greater value.
  4. Once the deal is complete and your node has received a voucher sufficient to cover the entire data transfer, you the node operator may then redeem the voucher and collect FIL.
Collecting FIL for a deal is the node's responsibility

To collect your FIL, your node must send on-chain messages directly to the payment channel actor to send all the vouchers, Settle, and Collect on the deal. This will finalize the client and provider balances for the retrieval deal on the Filecoin blockchain. Implementation and timing of these calls is the node's responsibility and is not a part of retrievalmarket. For more information about how to interact with the payment channel actor, see the github.com/filecoin-project/specs-actors repo.

Implementation

General Steps
  1. Decide if your node can be configured as a Retrieval Provider, a Retrieval Client or both.
  2. Determine how and where your retrieval calls to RetrievalProvider and RetrievalClient functions will be made.
  3. Implement the required interfaces as described in this section.
  4. Construct a RetrievalClient in your node's startup, if your node will be a client.
  5. Construct a RetrievalProvider in your node's startup, if your node will be a provider. If setting up a RetrievalProvider, call its Start function it in the appropriate place, and its Stop function in the appropriate place.
  6. Expose desired retrievalmarket functionality to whatever internal modules desired, such as command line interface, JSON RPC, or HTTP API.

Implement the PeerResolver, RetrievalProviderNode, and RetrievalClientNode interfaces in retrievalmarket/types.go, described below:

PeerResolver

PeerResolver is an interface for looking up providers that may have a piece of identifiable data. Its functions are:

GetPeers
func GetPeers(payloadCID cid.Cid) ([]RetrievalPeer, error)

Return a slice of RetrievalPeers that store the data referenced by payloadCID.

RetrievalClientNode

RetrievalClientNode contains the node dependencies for a RetrievalClient. Its functions are:

AllocateLane
func AllocateLane(paymentChannel address.Address) (uint64, error)

Create a lane within paymentChannel so that calls to CreatePaymentVoucher will automatically make vouchers only for the difference in total. Note that payment channel Actors have a lane limit.

CreatePaymentVoucher
func CreatePaymentVoucher(ctx context.Context, paymentChannel address.Address, 
                         amount abi.TokenAmount, lane uint64, tok shared.TipSetToken
                         ) (*paych.SignedVoucher, error)

Create a new payment voucher for paymentChannel with amount, for lane lane, given chain state at tok.

GetChainHead
func GetChainHead(ctx context.Context) (shared.TipSetToken, abi.ChainEpoch, error)

Get the current chain head. Return its TipSetToken and its abi.ChainEpoch.

GetOrCreatePaymentChannel
func GetOrCreatePaymentChannel(ctx context.Context, clientAddress, minerAddress address.Address, 
                          amount abi.TokenAmount, tok shared.TipSetToken
                         ) (address.Address, cid.Cid, error)

If there is a current payment channel for deals between clientAddress and minerAddress, add amount to the channel, then return the payment channel address and cid.Undef.

If there isn't, construct a new payment channel actor with amount funds by posting the corresponding message on chain, then return address.Undef and the posted message cid.Cid. For more information about how to construct a payment channel actor, see github.com/filecoin-project/specs-actors

WaitForPaymentChannelAddFunds
func WaitForPaymentChannelAddFunds(messageCID cid.Cid) error

Wait for message with CID messageCID on chain that funds have been sent to a payment channel.

WaitForPaymentChannelCreation
func WaitForPaymentChannelCreation(messageCID cid.Cid) (address.Address, error)

Wait for a message on chain with CID messageCID that a payment channel has been created.

RetrievalProviderNode

RetrievalProviderNode contains the node dependencies for a RetrievalProvider. Its functions are:

GetChainHead
func GetChainHead(ctx context.Context) (shared.TipSetToken, abi.ChainEpoch, error)

Get the current chain head. Return its TipSetToken and its abi.ChainEpoch.

GetMinerWorkerAddress
func GetMinerWorkerAddress(ctx context.Context, addr address.Address, tok shared.TipSetToken,
                     ) (address.Address, error)

Get the miner worker address for the given miner owner, as of tok.

UnsealSector
func UnsealSector(ctx context.Context, sectorID uint64, offset uint64, length uint64,
             ) (io.ReadCloser, error)

Unseal length data contained in sectorID, starting at offset. Return an io.ReadCloser for accessing the data.

SavePaymentVoucher
func SavePaymentVoucher(ctx context.Context, paymentChannel address.Address, 
                   voucher *paych.SignedVoucher, proof []byte, expectedAmount abi.TokenAmount, 
                   tok shared.TipSetToken) (abi.TokenAmount, error)

Save the provided paych.SignedVoucher for paymentChannel. The RetrievalProviderNode implementation should validate the SignedVoucher using the provided proof, expectedAmount, based on the chain state referenced by tok. The value of the voucher should be equal or greater than the largest previous voucher by expectedAmount. It returns the actual difference.

Construction

RetrievalClient
package retrievalimpl
func NewClient(
	netwk network.RetrievalMarketNetwork,
	bs blockstore.Blockstore,
	node retrievalmarket.RetrievalClientNode,
	resolver retrievalmarket.PeerResolver,
	ds datastore.Batching,
	storedCounter *storedcounter.StoredCounter,
) (retrievalmarket.RetrievalClient, error)
Parameters

  • netwk rmnet.RetrievalMarketNetwork RetrievalMarketNetwork is an interface for creating and handling deal streams. To create it:

    package network

func NewFromLibp2pHost(h host.Host) RetrievalMarketNetwork

    where h host.Host is your node's libp2p Host. See github.com/libp2p/go-libp2p-core/host.

  • bs blockstore.Blockstore is an IPFS blockstore for storing and retrieving data for deals. See github.com/ipfs/go-ipfs-blockstore.

  • node retrievalmarket.RetrievalClientNode is the RetrievalClientNode interface you have implemented.

  • resolver retrievalmarket.PeerResolver is the PeerResolver interface you have implemented.

  • ds datastore.Batching is a datastore for the deal's state machine. It is typically the node's own datastore that implements the IPFS datastore.Batching interface. See github.com/ipfs/go-datastore.

  • storedCounter *storedcounter.StoredCounter is a file-based stored counter used to generate new dealIDs. See github.com/filecoin-project/go-storedcounter.

RetrievalProvider
package retrievalimpl

func NewProvider(minerAddress address.Address, 
                node retrievalmarket.RetrievalProviderNode, 
                netwk network.RetrievalMarketNetwork, 
                pieceStore piecestore.PieceStore, 
                bs blockstore.Blockstore, 
                ds datastore.Batching,
                ) (retrievalmarket.RetrievalProvider, error)
Parameters
  • minerAddress address.Address is the address of the retrieval miner owner.
  • node retrievalmarket.RetrievalProviderNode is the RetrievalProviderNode API you have implemented.
  • netwk rmnet.RetrievalMarketNetwork is the same interface for creating and handling deal streams as for constructing a RetrievalClient.
  • pieceStore piecestore.PieceStore is the database of deals and pieces associated with them. See this repo's piecestore module.
  • bs blockstore.Blockstore is the same interface as for constructing a RetrievalClient.
  • ds datastore.Batching is the same batching datastore interface as for constructing a RetrievalClient.

Technical Documentation

  • GoDoc contains an architectural overview and robust API documentation

  • Retrieval Client FSM diagram:

Diagram of RetrievalClientFSM

  • Retrieval Provider FSM diagram:

Diagram of RetrievalClientFSM

Documentation

Overview

Package retrievalmarket implements the Filecoin retrieval protocol.

An overview of the retrieval protocol can be found in the Filecoin specification:

https://filecoin-project.github.io/specs/#systems__filecoin_markets__retrieval_market

The following architectural components provide a brief overview of the design of the retrieval market module:

Public Interfaces And Node Dependencies

While retrieval deals primarily happen off-chain, there are some chain operations that must be performed by a Filecoin node implementation. The module is intended to separate the primarily off-chain retrieval deal flow from the on-chain operations related primarily to payment channels, the mechanism for getting paid for retrieval deals.

As such for both the client and the provider in the retrieval market, the module defines a top level public interface which it provides an implementation for, and a node interface that must be implemented by the Filecoin node itself, and provided as a dependency. These node interfaces provide a universal way to talk to potentially multiple different Filecoin node implementations, and can be implemented as using HTTP or other interprocess communication to talk to a node implementation running in a different process.

The top level interfaces this package implements are RetrievalClient & RetrievalProvider. The dependencies the Filecoin node is expected to implement are RetrievalClientNode & RetrievalProviderNode. Further documentation of exactly what those dependencies should do can be found in the readme.

Finite State Machines

While retrieval deals in general should be fairly fast, making a retrieval deal is still an asynchronous process. As documented in the Filecoin spec, the basic architecture of the Filecoin retrieval protocol is incremental payments. Because neither client nor provider trust each other, we bootstrap trust by essentially paying in small increments as we receive data. The client only sends payment when it verifies data and the provider only sends more data when it receives payment. Not surprisingly, many things can go wrong along the way. To manage this back and forth asynchronous process, we use finite state machines that update deal state when discrete events occur. State updates always persist state to disk. This means we have a permanent record of exactly what's going on with deals at any time, and we can ideally survive our Filecoin processes shutting down and restarting.

The following diagrams visualize the statemachine flows for the client and the provider:

Client FSM - https://raw.githubusercontent.com/brossetti1/go-fil-markets/master/docs/retrievalclient.mmd.svg

Provider FSM - https://raw.githubusercontent.com/brossetti1/go-fil-markets/master/docs/retrievalprovider.mmd.svg

Identifying Retrieval Providers

The RetrievalClient provides two functions to locate a provider from which to retrieve data.

`FindProviders` returns a list of retrieval peers who may have the data your looking for. FindProviders delegates its work to an implementation of the PeerResolver interface.

`Query` queries a specific retrieval provider to find out definitively if they have the requested data and if so, the parameters they will accept for a retrieval deal.

Deal Flow

The primary mechanism for initiating storage deals is the `Retrieve` method on the RetrievalClient.

When `Retrieve` is called, it allocates a new DealID from its stored counter, constructs a DealProposal, initiates tracking of deal state and hands the deal to the Client FSM, and returns the DealID which constitutes the identifier for that deal.

From this point forward, deal negotiation is completely asynchronous and runs in the FSMs.

The FSM opens a data transfer to the provider containing the deal proposal. The provider receives the deal proposal in its request validator. The request validator initiates tracking of deal state on the Provider side and hands the deal to the Provider FSM, which handles the rest of deal flow.

A user of the modules can monitor deal progress through `SubscribeToEvents` methods on RetrievalClient and RetrievalProvider, or by simply calling `ListDeals` to get all deal statuses.

The FSMs implement every remaining step in deal negotiation. Importantly, the RetrievalProvider delegates unsealing sectors back to the node via the `UnsealSector` method (the node itself likely delegates management of sectors and sealing to an implementation of the Storage Mining subsystem of the Filecoin spec). Sectors are unsealed on an as needed basis using the `PieceStore` to locate sectors that contain data related to the deal.

Major Dependencies

Other libraries in go-fil-markets:

https://github.com/brossetti1/go-fil-markets/tree/master/piecestore - used to locate data for deals in sectors https://github.com/brossetti1/go-fil-markets/tree/master/shared - types and utility functions shared with storagemarket package

Other Filecoin Repos:

https://github.com/filecoin-project/go-data-transfer - for transferring data, via go-graphsync https://github.com/filecoin-project/go-statemachine - a finite state machine that tracks deal state https://github.com/filecoin-project/go-storedcounter - for generating and persisting unique deal IDs https://github.com/filecoin-project/specs-actors - the Filecoin actors

IPFS Project Repos:

https://github.com/ipfs/go-graphsync - used by go-data-transfer https://github.com/ipfs/go-datastore - for persisting statemachine state for deals https://github.com/ipfs/go-ipfs-blockstore - for storing and retrieving block data for deals

Other Repos:

https://github.com/libp2p/go-libp2p) the network over which retrieval deal data is exchanged. https://github.com/hannahhoward/go-pubsub - for pub/sub notifications external to the statemachine

Root package

This top level package defines top level enumerations and interfaces. The primary implementation lives in the `impl` directory

Index

Constants

View Source 
const DealPaymentType = datatransfer.TypeIdentifier("RetrievalDealPayment/1")

DealPaymentType is the DealPayment voucher type

View Source 
const DealProposalType = datatransfer.TypeIdentifier("RetrievalDealProposal/1")

DealProposalType is the DealProposal voucher type

View Source 
const DealResponseType = datatransfer.TypeIdentifier("RetrievalDealResponse/1")

DealResponseType is the DealResponse usable as a voucher type

View Source 
const QueryProtocolID = protocol.ID("/fil/retrieval/qry/1.0.0")

QueryProtocolID is the protocol for querying information about retrieval deal parameters

Variables

View Source 
var (
	// ErrNotFound means a piece was not found during retrieval
	ErrNotFound = errors.New("not found")

	// ErrVerification means a retrieval contained a block response that did not verify
	ErrVerification = errors.New("Error when verify data")
)
View Source 
var AddressBindnodeOption = bindnode.TypedBytesConverter(&address.Address{}, addressFromBytes, addressToBytes)

AddressBindnodeOption converts a filecoin Address type to and from a Bytes field in a schema

View Source 
var BigIntBindnodeOption = bindnode.TypedBytesConverter(&big.Int{}, bigIntFromBytes, bigIntToBytes)

BigIntBindnodeOption converts a big.Int type to and from a Bytes field in a schema

View Source 
var BindnodeRegistry = bindnoderegistry.NewRegistry()
View Source 
var CborGenCompatibleNodeBindnodeOption = bindnode.TypedAnyConverter(&CborGenCompatibleNode{}, cborGenCompatibleNodeFromAny, cborGenCompatibleNodeToAny)

CborGenCompatibleNodeBindnodeOption converts a CborGenCompatibleNode type to and from an Any field in a schema

View Source 
var ClientEvents = map[ClientEvent]string{
	ClientEventOpen:                          "ClientEventOpen",
	ClientEventPaymentChannelErrored:         "ClientEventPaymentChannelErrored",
	ClientEventDealProposed:                  "ClientEventDealProposed",
	ClientEventAllocateLaneErrored:           "ClientEventAllocateLaneErrored",
	ClientEventPaymentChannelCreateInitiated: "ClientEventPaymentChannelCreateInitiated",
	ClientEventPaymentChannelReady:           "ClientEventPaymentChannelReady",
	ClientEventPaymentChannelAddingFunds:     "ClientEventPaymentChannelAddingFunds",
	ClientEventPaymentChannelAddFundsErrored: "ClientEventPaymentChannelAddFundsErrored",
	ClientEventWriteDealProposalErrored:      "ClientEventWriteDealProposalErrored",
	ClientEventDealRejected:                  "ClientEventDealRejected",
	ClientEventDealNotFound:                  "ClientEventDealNotFound",
	ClientEventDealAccepted:                  "ClientEventDealAccepted",
	ClientEventProviderCancelled:             "ClientEventProviderCancelled",
	ClientEventUnknownResponseReceived:       "ClientEventUnknownResponseReceived",
	ClientEventLastPaymentRequested:          "ClientEventLastPaymentRequested",
	ClientEventAllBlocksReceived:             "ClientEventAllBlocksReceived",
	ClientEventPaymentRequested:              "ClientEventPaymentRequested",
	ClientEventUnsealPaymentRequested:        "ClientEventUnsealPaymentRequested",
	ClientEventBlocksReceived:                "ClientEventBlocksReceived",
	ClientEventSendFunds:                     "ClientEventSendFunds",
	ClientEventFundsExpended:                 "ClientEventFundsExpended",
	ClientEventBadPaymentRequested:           "ClientEventBadPaymentRequested",
	ClientEventCreateVoucherFailed:           "ClientEventCreateVoucherFailed",
	ClientEventWriteDealPaymentErrored:       "ClientEventWriteDealPaymentErrored",
	ClientEventPaymentSent:                   "ClientEventPaymentSent",
	ClientEventDataTransferError:             "ClientEventDataTransferError",
	ClientEventComplete:                      "ClientEventComplete",
	ClientEventCancelComplete:                "ClientEventCancelComplete",
	ClientEventEarlyTermination:              "ClientEventEarlyTermination",
	ClientEventCompleteVerified:              "ClientEventCompleteVerified",
	ClientEventLaneAllocated:                 "ClientEventLaneAllocated",
	ClientEventVoucherShortfall:              "ClientEventVoucherShortfall",
	ClientEventRecheckFunds:                  "ClientEventRecheckFunds",
	ClientEventCancel:                        "ClientEventCancel",
	ClientEventWaitForLastBlocks:             "ClientEventWaitForLastBlocks",
	ClientEventPaymentChannelSkip:            "ClientEventPaymentChannelSkip",
	ClientEventPaymentNotSent:                "ClientEventPaymentNotSent",
	ClientEventBlockstoreFinalized:           "ClientEventBlockstoreFinalized",
	ClientEventFinalizeBlockstoreErrored:     "ClientEventFinalizeBlockstoreErrored",
}

ClientEvents is a human readable map of client event name -> event description

View Source 
var DealPaymentUndefined = DealPayment{}

DealPaymentUndefined is an undefined deal payment

View Source 
var DealProposalUndefined = DealProposal{}

DealProposalUndefined is an undefined deal proposal

View Source 
var DealResponseUndefined = DealResponse{}

DealResponseUndefined is an undefined deal response

View Source 
var DealStatuses = map[DealStatus]string{
	DealStatusNew:                              "DealStatusNew",
	DealStatusUnsealing:                        "DealStatusUnsealing",
	DealStatusUnsealed:                         "DealStatusUnsealed",
	DealStatusWaitForAcceptance:                "DealStatusWaitForAcceptance",
	DealStatusPaymentChannelCreating:           "DealStatusPaymentChannelCreating",
	DealStatusPaymentChannelAddingFunds:        "DealStatusPaymentChannelAddingFunds",
	DealStatusAccepted:                         "DealStatusAccepted",
	DealStatusFundsNeededUnseal:                "DealStatusFundsNeededUnseal",
	DealStatusFailing:                          "DealStatusFailing",
	DealStatusRejected:                         "DealStatusRejected",
	DealStatusFundsNeeded:                      "DealStatusFundsNeeded",
	DealStatusSendFunds:                        "DealStatusSendFunds",
	DealStatusSendFundsLastPayment:             "DealStatusSendFundsLastPayment",
	DealStatusOngoing:                          "DealStatusOngoing",
	DealStatusFundsNeededLastPayment:           "DealStatusFundsNeededLastPayment",
	DealStatusCompleted:                        "DealStatusCompleted",
	DealStatusDealNotFound:                     "DealStatusDealNotFound",
	DealStatusErrored:                          "DealStatusErrored",
	DealStatusBlocksComplete:                   "DealStatusBlocksComplete",
	DealStatusFinalizing:                       "DealStatusFinalizing",
	DealStatusCompleting:                       "DealStatusCompleting",
	DealStatusCheckComplete:                    "DealStatusCheckComplete",
	DealStatusCheckFunds:                       "DealStatusCheckFunds",
	DealStatusInsufficientFunds:                "DealStatusInsufficientFunds",
	DealStatusPaymentChannelAllocatingLane:     "DealStatusPaymentChannelAllocatingLane",
	DealStatusCancelling:                       "DealStatusCancelling",
	DealStatusCancelled:                        "DealStatusCancelled",
	DealStatusRetryLegacy:                      "DealStatusRetryLegacy",
	DealStatusWaitForAcceptanceLegacy:          "DealStatusWaitForAcceptanceLegacy",
	DealStatusClientWaitingForLastBlocks:       "DealStatusWaitingForLastBlocks",
	DealStatusPaymentChannelAddingInitialFunds: "DealStatusPaymentChannelAddingInitialFunds",
	DealStatusErroring:                         "DealStatusErroring",
	DealStatusRejecting:                        "DealStatusRejecting",
	DealStatusDealNotFoundCleanup:              "DealStatusDealNotFoundCleanup",
	DealStatusFinalizingBlockstore:             "DealStatusFinalizingBlockstore",
}

DealStatuses maps deal status to a human readable representation

View Source 
var DefaultPaymentInterval = uint64(1 << 20)

DefaultPaymentInterval is the baseline interval, set to 1Mb if the miner does not explicitly set it otherwise

View Source 
var DefaultPaymentIntervalIncrease = uint64(1 << 20)

DefaultPaymentIntervalIncrease is the amount interval increases on each payment, set to to 1Mb if the miner does not explicitly set it otherwise

View Source 
var DefaultPricePerByte = abi.NewTokenAmount(2)

DefaultPricePerByte is the charge per byte retrieved if the miner does not specifically set it

View Source 
var DefaultUnsealPrice = abi.NewTokenAmount(0)

DefaultUnsealPrice is the default charge to unseal a sector for retrieval

View Source 
var ProviderEvents = map[ProviderEvent]string{
	ProviderEventOpen:                   "ProviderEventOpen",
	ProviderEventDealNotFound:           "ProviderEventDealNotFound",
	ProviderEventDealRejected:           "ProviderEventDealRejected",
	ProviderEventDealAccepted:           "ProviderEventDealAccepted",
	ProviderEventBlockSent:              "ProviderEventBlockSent",
	ProviderEventBlocksCompleted:        "ProviderEventBlocksCompleted",
	ProviderEventPaymentRequested:       "ProviderEventPaymentRequested",
	ProviderEventSaveVoucherFailed:      "ProviderEventSaveVoucherFailed",
	ProviderEventPartialPaymentReceived: "ProviderEventPartialPaymentReceived",
	ProviderEventPaymentReceived:        "ProviderEventPaymentReceived",
	ProviderEventComplete:               "ProviderEventComplete",
	ProviderEventUnsealError:            "ProviderEventUnsealError",
	ProviderEventUnsealComplete:         "ProviderEventUnsealComplete",
	ProviderEventDataTransferError:      "ProviderEventDataTransferError",
	ProviderEventCancelComplete:         "ProviderEventCancelComplete",
	ProviderEventCleanupComplete:        "ProviderEventCleanupComplete",
	ProviderEventMultiStoreError:        "ProviderEventMultiStoreError",
	ProviderEventClientCancelled:        "ProviderEventClientCancelled",
	ProviderEventLastPaymentRequested:   "ProviderEventLastPaymentRequested",
	ProviderEventProcessPayment:         "ProviderEventProcessPayment",
}

ProviderEvents is a human readable map of provider event name -> event description

View Source 
var QueryResponseUndefined = QueryResponse{}

QueryResponseUndefined is an empty QueryResponse

View Source 
var QueryUndefined = Query{}

QueryUndefined is a query with no values

View Source 
var SignatureBindnodeOption = bindnode.TypedBytesConverter(&crypto.Signature{}, signatureFromBytes, signatureToBytes)

SignatureBindnodeOption converts a filecoin Signature type to and from a Bytes field in a schema

View Source 
var TokenAmountBindnodeOption = bindnode.TypedBytesConverter(&abi.TokenAmount{}, tokenAmountFromBytes, tokenAmountToBytes)

TokenAmountBindnodeOption converts a filecoin abi.TokenAmount type to and from a Bytes field in a schema

Functions

func IsTerminalError 

func IsTerminalError(status DealStatus) bool

IsTerminalError returns true if this status indicates processing of this deal is complete with an error

func IsTerminalStatus 

func IsTerminalStatus(status DealStatus) bool

IsTerminalStatus returns true if this status indicates processing of a deal is complete (either success or error)

func IsTerminalSuccess 

func IsTerminalSuccess(status DealStatus) bool

IsTerminalSuccess returns true if this status indicates processing of this deal is complete with a success

func NewShortfallError 

func NewShortfallError(shortfall abi.TokenAmount) error

NewShortfallError returns a new error indicating a shortfall of funds

Types

type Ask 

type Ask struct {
	PricePerByte            abi.TokenAmount
	UnsealPrice             abi.TokenAmount
	PaymentInterval         uint64
	PaymentIntervalIncrease uint64
}

func (*Ask) MarshalCBOR 

func (t *Ask) MarshalCBOR(w io.Writer) error

func (*Ask) UnmarshalCBOR 

func (t *Ask) UnmarshalCBOR(r io.Reader) (err error)

type AskStore 

type AskStore interface {
	GetAsk() *Ask
	SetAsk(ask *Ask) error
}

AskStore is an interface which provides access to a persisted retrieval Ask

type BlockstoreAccessor 

type BlockstoreAccessor interface {
	Get(DealID, PayloadCID) (bstore.Blockstore, error)
	Done(DealID) error
}

BlockstoreAccessor is used by the retrieval market client to get a blockstore when needed, concretely to store blocks received from the provider. This abstraction allows the caller to provider any blockstore implementation: a CARv2 file, an IPFS blockstore, or something else.

type CborGenCompatibleNode 

type CborGenCompatibleNode struct {
	Node datamodel.Node
}

CborGenCompatibleNode is for cbor-gen / go-ipld-prime compatibility, to replace Deferred types that are used to represent datamodel.Nodes. This shouldn't be used as a pointer (nullable/optional) as it can consume "Null" tokens and therefore be a Null. Instead, use CborGenCompatibleNode#IsNull to check for null status.

func (CborGenCompatibleNode) IsNull 

func (sn CborGenCompatibleNode) IsNull() bool

func (*CborGenCompatibleNode) MarshalCBOR 

func (sn *CborGenCompatibleNode) MarshalCBOR(w io.Writer) error

MarshalCBOR is for cbor-gen compatibility

func (*CborGenCompatibleNode) UnmarshalCBOR 

func (sn *CborGenCompatibleNode) UnmarshalCBOR(r io.Reader) error

UnmarshalCBOR is for cbor-gen compatibility

type ChannelAvailableFunds 

type ChannelAvailableFunds struct {
	// ConfirmedAmt is the amount of funds that have been confirmed on-chain
	// for the channel
	ConfirmedAmt abi.TokenAmount
	// PendingAmt is the amount of funds that are pending confirmation on-chain
	PendingAmt abi.TokenAmount
	// PendingWaitSentinel can be used with PaychGetWaitReady to wait for
	// confirmation of pending funds
	PendingWaitSentinel *cid.Cid
	// QueuedAmt is the amount that is queued up behind a pending request
	QueuedAmt abi.TokenAmount
	// VoucherRedeemedAmt is the amount that is redeemed by vouchers on-chain
	// and in the local datastore
	VoucherReedeemedAmt abi.TokenAmount
}

ChannelAvailableFunds provides information about funds in a channel

type ClientDealState 

type ClientDealState struct {
	DealProposal
	StoreID *uint64
	// Set when the data transfer is started
	ChannelID            *datatransfer.ChannelID
	LastPaymentRequested bool
	AllBlocksReceived    bool
	TotalFunds           abi.TokenAmount
	ClientWallet         address.Address
	MinerWallet          address.Address
	PaymentInfo          *PaymentInfo
	Status               DealStatus
	Sender               peer.ID
	TotalReceived        uint64
	Message              string
	BytesPaidFor         uint64
	CurrentInterval      uint64
	PaymentRequested     abi.TokenAmount
	FundsSpent           abi.TokenAmount
	UnsealFundsPaid      abi.TokenAmount
	WaitMsgCID           *cid.Cid // the CID of any message the client deal is waiting for
	VoucherShortfall     abi.TokenAmount
	LegacyProtocol       bool
}

ClientDealState is the current state of a deal from the point of view of a retrieval client

func (*ClientDealState) MarshalCBOR 

func (t *ClientDealState) MarshalCBOR(w io.Writer) error

func (*ClientDealState) NextInterval 

func (deal *ClientDealState) NextInterval() uint64

func (*ClientDealState) UnmarshalCBOR 

func (t *ClientDealState) UnmarshalCBOR(r io.Reader) (err error)

type ClientEvent 

type ClientEvent uint64

ClientEvent is an event that occurs in a deal lifecycle on the client 

const (
	// ClientEventOpen indicates a deal was initiated
	ClientEventOpen ClientEvent = iota

	// ClientEventWriteDealProposalErrored means a network error writing a deal proposal
	ClientEventWriteDealProposalErrored

	// ClientEventDealProposed means a deal was successfully sent to a miner
	ClientEventDealProposed

	// ClientEventDealRejected means a deal was rejected by the provider
	ClientEventDealRejected

	// ClientEventDealNotFound means a provider could not find a piece for a deal
	ClientEventDealNotFound

	// ClientEventDealAccepted means a provider accepted a deal
	ClientEventDealAccepted

	// ClientEventProviderCancelled means a provider has sent a message to cancel a deal
	ClientEventProviderCancelled

	// ClientEventUnknownResponseReceived means a client received a response it doesn't
	// understand from the provider
	ClientEventUnknownResponseReceived

	// ClientEventPaymentChannelErrored means there was a failure creating a payment channel
	ClientEventPaymentChannelErrored

	// ClientEventAllocateLaneErrored means there was a failure creating a lane in a payment channel
	ClientEventAllocateLaneErrored

	// ClientEventPaymentChannelCreateInitiated means we are waiting for a message to
	// create a payment channel to appear on chain
	ClientEventPaymentChannelCreateInitiated

	// ClientEventPaymentChannelReady means the newly created payment channel is ready for the
	// deal to resume
	ClientEventPaymentChannelReady

	// ClientEventPaymentChannelAddingFunds mean we are waiting for funds to be
	// added to a payment channel
	ClientEventPaymentChannelAddingFunds

	// ClientEventPaymentChannelAddFundsErrored means that adding funds to the payment channel
	// failed
	ClientEventPaymentChannelAddFundsErrored

	// ClientEventLastPaymentRequested indicates the provider requested a final payment
	ClientEventLastPaymentRequested

	// ClientEventAllBlocksReceived indicates the provider has sent all blocks
	ClientEventAllBlocksReceived

	// ClientEventPaymentRequested indicates the provider requested a payment
	ClientEventPaymentRequested

	// ClientEventUnsealPaymentRequested indicates the provider requested a payment for unsealing the sector
	ClientEventUnsealPaymentRequested

	// ClientEventBlocksReceived indicates the provider has sent blocks
	ClientEventBlocksReceived

	// ClientEventSendFunds emits when we reach the threshold to send the next payment
	ClientEventSendFunds

	// ClientEventFundsExpended indicates a deal has run out of funds in the payment channel
	// forcing the client to add more funds to continue the deal
	ClientEventFundsExpended // when totalFunds is expended

	// ClientEventBadPaymentRequested indicates the provider asked for funds
	// in a way that does not match the terms of the deal
	ClientEventBadPaymentRequested

	// ClientEventCreateVoucherFailed indicates an error happened creating a payment voucher
	ClientEventCreateVoucherFailed

	// ClientEventWriteDealPaymentErrored indicates a network error trying to write a payment
	ClientEventWriteDealPaymentErrored

	// ClientEventPaymentSent indicates a payment was sent to the provider
	ClientEventPaymentSent

	// ClientEventComplete is fired when the provider sends a message
	// indicating that a deal has completed
	ClientEventComplete

	// ClientEventDataTransferError emits when something go wrong at the data transfer level
	ClientEventDataTransferError

	// ClientEventCancelComplete happens when a deal cancellation is transmitted to the provider
	ClientEventCancelComplete

	// ClientEventEarlyTermination indications a provider send a deal complete without sending all data
	ClientEventEarlyTermination

	// ClientEventCompleteVerified means that a provider completed without requesting a final payment but
	// we verified we received all data
	ClientEventCompleteVerified

	// ClientEventLaneAllocated is called when a lane is allocated
	ClientEventLaneAllocated

	// ClientEventVoucherShortfall means we tried to create a voucher but did not have enough funds in channel
	// to create it
	ClientEventVoucherShortfall

	// ClientEventRecheckFunds runs when an external caller indicates there may be new funds in a payment channel
	ClientEventRecheckFunds

	// ClientEventCancel runs when a user cancels a deal
	ClientEventCancel

	// ClientEventWaitForLastBlocks is fired when the provider has told
	// the client that all blocks were sent for the deal, and the client is
	// waiting for the last blocks to arrive
	ClientEventWaitForLastBlocks

	// ClientEventPaymentChannelSkip is fired when the total deal price is zero
	// so there's no need to set up a payment channel
	ClientEventPaymentChannelSkip

	// ClientEventPaymentNotSent indicates that payment was requested, but no
	// payment was actually due, so a voucher was not sent to the provider
	ClientEventPaymentNotSent

	// ClientEventBlockstoreFinalized is fired when the blockstore has been
	// finalized after receiving all blocks
	ClientEventBlockstoreFinalized

	// ClientEventFinalizeBlockstoreErrored is fired when there is an error
	// finalizing the blockstore
	ClientEventFinalizeBlockstoreErrored
)

func (ClientEvent) String 

func (e ClientEvent) String() string

type ClientSubscriber 

type ClientSubscriber func(event ClientEvent, state ClientDealState)

ClientSubscriber is a callback that is registered to listen for retrieval events

type DealID 

type DealID uint64

DealID is an identifier for a retrieval deal (unique to a client)

func (DealID) String 

func (d DealID) String() string

type DealPayment 

type DealPayment struct {
	ID             DealID
	PaymentChannel address.Address
	PaymentVoucher *paychtypes.SignedVoucher
}

DealPayment is a payment for an in progress retrieval deal

func DealPaymentFromNode 

func DealPaymentFromNode(node datamodel.Node) (*DealPayment, error)

func (*DealPayment) MarshalCBOR 

func (t *DealPayment) MarshalCBOR(w io.Writer) error

func (*DealPayment) UnmarshalCBOR 

func (t *DealPayment) UnmarshalCBOR(r io.Reader) (err error)

type DealProposal 

type DealProposal struct {
	PayloadCID cid.Cid
	ID         DealID
	Params
}

DealProposal is a proposal for a new retrieval deal

func DealProposalFromNode 

func DealProposalFromNode(node datamodel.Node) (*DealProposal, error)

func (*DealProposal) MarshalCBOR 

func (t *DealProposal) MarshalCBOR(w io.Writer) error

func (*DealProposal) UnmarshalCBOR 

func (t *DealProposal) UnmarshalCBOR(r io.Reader) (err error)

type DealResponse 

type DealResponse struct {
	Status DealStatus
	ID     DealID

	// payment required to proceed
	PaymentOwed abi.TokenAmount

	Message string
}

DealResponse is a response to a retrieval deal proposal

func DealResponseFromNode 

func DealResponseFromNode(node datamodel.Node) (*DealResponse, error)

func (*DealResponse) MarshalCBOR 

func (t *DealResponse) MarshalCBOR(w io.Writer) error

func (*DealResponse) UnmarshalCBOR 

func (t *DealResponse) UnmarshalCBOR(r io.Reader) (err error)

type DealStatus 

type DealStatus uint64

DealStatus is the status of a retrieval deal returned by a provider in a DealResponse 

const (
	// DealStatusNew is a deal that nothing has happened with yet
	DealStatusNew DealStatus = iota

	// DealStatusUnsealing means the provider is unsealing data
	DealStatusUnsealing

	// DealStatusUnsealed means the provider has finished unsealing data
	DealStatusUnsealed

	// DealStatusWaitForAcceptance means we're waiting to hear back if the provider accepted our deal
	DealStatusWaitForAcceptance

	// DealStatusPaymentChannelCreating is the status set while waiting for the
	// payment channel creation to complete
	DealStatusPaymentChannelCreating

	// DealStatusPaymentChannelAddingFunds is the status when we are waiting for funds
	// to finish being sent to the payment channel
	DealStatusPaymentChannelAddingFunds

	// DealStatusAccepted means a deal has been accepted by a provider
	// and its is ready to proceed with retrieval
	DealStatusAccepted

	// DealStatusFundsNeededUnseal means a deal has been accepted by a provider
	// and payment is needed to unseal the data
	DealStatusFundsNeededUnseal

	// DealStatusFailing indicates something went wrong during a retrieval,
	// and we are cleaning up before terminating with an error
	DealStatusFailing

	// DealStatusRejected indicates the provider rejected a client's deal proposal
	// for some reason
	DealStatusRejected

	// DealStatusFundsNeeded indicates the provider needs a payment voucher to
	// continue processing the deal
	DealStatusFundsNeeded

	// DealStatusSendFunds indicates the client is now going to send funds because we reached the threshold of the last payment
	DealStatusSendFunds

	// DealStatusSendFundsLastPayment indicates the client is now going to send final funds because
	// we reached the threshold of the final payment
	DealStatusSendFundsLastPayment

	// DealStatusOngoing indicates the provider is continuing to process a deal
	DealStatusOngoing

	// DealStatusFundsNeededLastPayment indicates the provider needs a payment voucher
	// in order to complete a deal
	DealStatusFundsNeededLastPayment

	// DealStatusCompleted indicates a deal is complete
	DealStatusCompleted

	// DealStatusDealNotFound indicates an update was received for a deal that could
	// not be identified
	DealStatusDealNotFound

	// DealStatusErrored indicates a deal has terminated in an error
	DealStatusErrored

	// DealStatusBlocksComplete indicates that all blocks have been processed for the piece
	DealStatusBlocksComplete

	// DealStatusFinalizing means the last payment has been received and
	// we are just confirming the deal is complete
	DealStatusFinalizing

	// DealStatusCompleting is just an inbetween state to perform final cleanup of
	// complete deals
	DealStatusCompleting

	// DealStatusCheckComplete is used for when the provided completes without a last payment
	// requested cycle, to verify we have received all blocks
	DealStatusCheckComplete

	// DealStatusCheckFunds means we are looking at the state of funding for the channel to determine
	// if more money is incoming
	DealStatusCheckFunds

	// DealStatusInsufficientFunds indicates we have depleted funds for the retrieval payment channel
	// - we can resume after funds are added
	DealStatusInsufficientFunds

	// DealStatusPaymentChannelAllocatingLane is the status when we are making a lane for this channel
	DealStatusPaymentChannelAllocatingLane

	// DealStatusCancelling means we are cancelling an inprogress deal
	DealStatusCancelling

	// DealStatusCancelled means a deal has been cancelled
	DealStatusCancelled

	// DealStatusRetryLegacy means we're attempting the deal proposal for a second time using the legacy datatype
	DealStatusRetryLegacy

	// DealStatusWaitForAcceptanceLegacy means we're waiting to hear the results on the legacy protocol
	DealStatusWaitForAcceptanceLegacy

	// DealStatusClientWaitingForLastBlocks means that the provider has told
	// the client that all blocks were sent for the deal, and the client is
	// waiting for the last blocks to arrive. This should only happen when
	// the deal price per byte is zero (if it's not zero the provider asks
	// for final payment after sending the last blocks).
	DealStatusClientWaitingForLastBlocks

	// DealStatusPaymentChannelAddingInitialFunds means that a payment channel
	// exists from an earlier deal between client and provider, but we need
	// to add funds to the channel for this particular deal
	DealStatusPaymentChannelAddingInitialFunds

	// DealStatusErroring means that there was an error and we need to
	// do some cleanup before moving to the error state
	DealStatusErroring

	// DealStatusRejecting means that the deal was rejected and we need to do
	// some cleanup before moving to the rejected state
	DealStatusRejecting

	// DealStatusDealNotFoundCleanup means that the deal was not found and we
	// need to do some cleanup before moving to the not found state
	DealStatusDealNotFoundCleanup

	// DealStatusFinalizingBlockstore means that all blocks have been received,
	// and the blockstore is being finalized
	DealStatusFinalizingBlockstore
)

func (DealStatus) String 

func (s DealStatus) String() string

type Params 

type Params struct {
	Selector                CborGenCompatibleNode // V1
	PieceCID                *cid.Cid
	PricePerByte            abi.TokenAmount
	PaymentInterval         uint64 // when to request payment
	PaymentIntervalIncrease uint64
	UnsealPrice             abi.TokenAmount
}

Params are the parameters requested for a retrieval deal proposal

func NewParamsV0 

func NewParamsV0(pricePerByte abi.TokenAmount, paymentInterval uint64, paymentIntervalIncrease uint64) Params

NewParamsV0 generates parameters for a retrieval deal, which is always a whole piece deal

func NewParamsV1 

func NewParamsV1(pricePerByte abi.TokenAmount, paymentInterval uint64, paymentIntervalIncrease uint64, sel datamodel.Node, pieceCid *cid.Cid, unsealPrice abi.TokenAmount) (Params, error)

NewParamsV1 generates parameters for a retrieval deal, including a selector

func (Params) IntervalLowerBound 

func (p Params) IntervalLowerBound(currentInterval uint64) uint64

func (*Params) MarshalCBOR 

func (t *Params) MarshalCBOR(w io.Writer) error

func (Params) NextInterval 

func (p Params) NextInterval(fundsReceived abi.TokenAmount) uint64

NextInterval produces the maximum data that can be transferred before more payment is request

func (Params) OutstandingBalance 

func (p Params) OutstandingBalance(fundsReceived abi.TokenAmount, sent uint64, inFinalization bool) big.Int

OutstandingBalance produces the amount owed based on the deal params for the given transfer state and funds received

func (Params) SelectorSpecified 

func (p Params) SelectorSpecified() bool

func (*Params) UnmarshalCBOR 

func (t *Params) UnmarshalCBOR(r io.Reader) (err error)

type PayloadCID 

type PayloadCID = cid.Cid

type PaymentInfo 

type PaymentInfo struct {
	PayCh address.Address
	Lane  uint64
}

PaymentInfo is the payment channel and lane for a deal, once it is setup

func (*PaymentInfo) MarshalCBOR 

func (t *PaymentInfo) MarshalCBOR(w io.Writer) error

func (*PaymentInfo) UnmarshalCBOR 

func (t *PaymentInfo) UnmarshalCBOR(r io.Reader) (err error)

type PricingInput 

type PricingInput struct {
	// PayloadCID is the cid of the payload to retrieve.
	PayloadCID cid.Cid
	// PieceCID is the cid of the Piece from which the Payload will be retrieved.
	PieceCID cid.Cid
	// PieceSize is the size of the Piece from which the payload will be retrieved.
	PieceSize abi.UnpaddedPieceSize
	// Client is the peerID of the retrieval client.
	Client peer.ID
	// VerifiedDeal is true if there exists a verified storage deal for the PayloadCID.
	VerifiedDeal bool
	// Unsealed is true if there exists an unsealed sector from which we can retrieve the given payload.
	Unsealed bool
	// CurrentAsk is the current configured ask in the ask-store.
	CurrentAsk Ask
}

PricingInput provides input parameters required to price a retrieval deal.

type ProviderDealIdentifier 

type ProviderDealIdentifier struct {
	Receiver peer.ID
	DealID   DealID
}

ProviderDealIdentifier is a value that uniquely identifies a deal

func (ProviderDealIdentifier) String 

func (p ProviderDealIdentifier) String() string

type ProviderDealState 

type ProviderDealState struct {
	DealProposal
	StoreID uint64

	ChannelID     *datatransfer.ChannelID
	PieceInfo     *piecestore.PieceInfo
	Status        DealStatus
	Receiver      peer.ID
	FundsReceived abi.TokenAmount
	Message       string
}

ProviderDealState is the current state of a deal from the point of view of a retrieval provider

func (ProviderDealState) Identifier 

func (pds ProviderDealState) Identifier() ProviderDealIdentifier

Identifier provides a unique id for this provider deal

func (*ProviderDealState) MarshalCBOR 

func (t *ProviderDealState) MarshalCBOR(w io.Writer) error

func (*ProviderDealState) UnmarshalCBOR 

func (t *ProviderDealState) UnmarshalCBOR(r io.Reader) (err error)

type ProviderEvent 

type ProviderEvent uint64

ProviderEvent is an event that occurs in a deal lifecycle on the provider 

const (
	// ProviderEventOpen indicates a new deal was received from a client
	ProviderEventOpen ProviderEvent = iota

	// ProviderEventDealNotFound happens when the provider cannot find the piece for the
	// deal proposed by the client
	ProviderEventDealNotFound

	// ProviderEventDealRejected happens when a provider rejects a deal proposed
	// by the client
	ProviderEventDealRejected

	// ProviderEventDealAccepted happens when a provider accepts a deal
	ProviderEventDealAccepted

	// ProviderEventBlockSent happens when the provider reads another block
	// in the piece
	ProviderEventBlockSent

	// ProviderEventBlocksCompleted happens when the provider reads the last block
	// in the piece
	ProviderEventBlocksCompleted

	// ProviderEventPaymentRequested happens when a provider asks for payment from
	// a client for blocks sent
	ProviderEventPaymentRequested

	// ProviderEventSaveVoucherFailed happens when an attempt to save a payment
	// voucher fails
	ProviderEventSaveVoucherFailed

	// ProviderEventPartialPaymentReceived happens when a provider receives and processes
	// a payment that is less than what was requested to proceed with the deal
	ProviderEventPartialPaymentReceived

	// ProviderEventPaymentReceived happens when a provider receives a payment
	// and resumes processing a deal
	ProviderEventPaymentReceived

	// ProviderEventComplete indicates a retrieval deal was completed for a client
	ProviderEventComplete

	// ProviderEventUnsealError emits when something wrong occurs while unsealing data
	ProviderEventUnsealError

	// ProviderEventUnsealComplete emits when the unsealing process is done
	ProviderEventUnsealComplete

	// ProviderEventDataTransferError emits when something go wrong at the data transfer level
	ProviderEventDataTransferError

	// ProviderEventCancelComplete happens when a deal cancellation is transmitted to the provider
	ProviderEventCancelComplete

	// ProviderEventCleanupComplete happens when a deal is finished cleaning up and enters a complete state
	ProviderEventCleanupComplete

	// ProviderEventMultiStoreError occurs when an error happens attempting to operate on the multistore
	ProviderEventMultiStoreError

	// ProviderEventClientCancelled happens when the provider gets a cancel message from the client's data transfer
	ProviderEventClientCancelled

	// ProviderEventLastPaymentRequested is emitted when the transfer goes into finalization
	ProviderEventLastPaymentRequested

	// ProviderEventProcessPayment occurs when the provider has received a voucher from the client that it needs
	// to process as a payment
	ProviderEventProcessPayment
)

type ProviderQueryEvent 

type ProviderQueryEvent struct {
	Response QueryResponse
	Error    error
}

type ProviderQueryEventSubscriber 

type ProviderQueryEventSubscriber func(evt ProviderQueryEvent)

ProviderQueryEventSubscriber is a callback that is registered to listen for query message events

type ProviderSubscriber 

type ProviderSubscriber func(event ProviderEvent, state ProviderDealState)

ProviderSubscriber is a callback that is registered to listen for retrieval events on a provider

type ProviderValidationEvent 

type ProviderValidationEvent struct {
	IsRestart bool
	Receiver  peer.ID
	Proposal  *DealProposal
	BaseCid   cid.Cid
	Selector  ipld.Node
	Response  *DealResponse
	Error     error
}

type ProviderValidationSubscriber 

type ProviderValidationSubscriber func(evt ProviderValidationEvent)

ProviderValidationSubscriber is a callback that is registered to listen for validation events

type Query 

type Query struct {
	PayloadCID  cid.Cid // V0
	QueryParams         // V1
}

Query is a query to a given provider to determine information about a piece they may have available for retrieval

func NewQueryV0 

func NewQueryV0(payloadCID cid.Cid) Query

NewQueryV0 creates a V0 query (which only specifies a payload)

func NewQueryV1 

func NewQueryV1(payloadCID cid.Cid, pieceCID *cid.Cid) Query

NewQueryV1 creates a V1 query (which has an optional pieceCID)

func (*Query) MarshalCBOR 

func (t *Query) MarshalCBOR(w io.Writer) error

func (*Query) UnmarshalCBOR 

func (t *Query) UnmarshalCBOR(r io.Reader) (err error)

type QueryItemStatus 

type QueryItemStatus uint64

QueryItemStatus (V1) indicates whether the requested part of a piece (payload or selector) is available for retrieval 

const (
	// QueryItemAvailable indicates requested part of the piece is available to be
	// served
	QueryItemAvailable QueryItemStatus = iota

	// QueryItemUnavailable indicates the piece either does not contain the requested
	// item or it cannot be served
	QueryItemUnavailable

	// QueryItemUnknown indicates the provider cannot determine if the given item
	// is part of the requested piece (for example, if the piece is sealed and the
	// miner does not maintain a payload CID index)
	QueryItemUnknown
)

type QueryParams 

type QueryParams struct {
	PieceCID *cid.Cid // optional, query if miner has this cid in this piece. some miners may not be able to respond.

}

QueryParams - V1 - indicate what specific information about a piece that a retrieval client is interested in, as well as specific parameters the client is seeking for the retrieval deal

func (*QueryParams) MarshalCBOR 

func (t *QueryParams) MarshalCBOR(w io.Writer) error

func (*QueryParams) UnmarshalCBOR 

func (t *QueryParams) UnmarshalCBOR(r io.Reader) (err error)

type QueryResponse 

type QueryResponse struct {
	Status        QueryResponseStatus
	PieceCIDFound QueryItemStatus // V1 - if a PieceCID was requested, the result

	Size uint64 // Total size of piece in bytes

	PaymentAddress             address.Address // address to send funds to -- may be different than miner addr
	MinPricePerByte            abi.TokenAmount
	MaxPaymentInterval         uint64
	MaxPaymentIntervalIncrease uint64
	Message                    string
	UnsealPrice                abi.TokenAmount
}

QueryResponse is a miners response to a given retrieval query

func (*QueryResponse) MarshalCBOR 

func (t *QueryResponse) MarshalCBOR(w io.Writer) error

func (QueryResponse) PieceRetrievalPrice 

func (qr QueryResponse) PieceRetrievalPrice() abi.TokenAmount

PieceRetrievalPrice is the total price to retrieve the piece (size * MinPricePerByte + UnsealedPrice)

func (*QueryResponse) UnmarshalCBOR 

func (t *QueryResponse) UnmarshalCBOR(r io.Reader) (err error)

type QueryResponseStatus 

type QueryResponseStatus uint64

QueryResponseStatus indicates whether a queried piece is available 

const (
	// QueryResponseAvailable indicates a provider has a piece and is prepared to
	// return it
	QueryResponseAvailable QueryResponseStatus = iota

	// QueryResponseUnavailable indicates a provider either does not have or cannot
	// serve the queried piece to the client
	QueryResponseUnavailable

	// QueryResponseError indicates something went wrong generating a query response
	QueryResponseError
)

type RetrievalClient 

type RetrievalClient interface {

	// NextID generates a new deal ID.
	NextID() DealID

	// Start initializes the client by running migrations
	Start(ctx context.Context) error

	// OnReady registers a listener for when the client comes on line
	OnReady(shared.ReadyFunc)

	// Find Providers finds retrieval providers who may be storing a given piece
	FindProviders(payloadCID cid.Cid) []RetrievalPeer

	// Query asks a provider for information about a piece it is storing
	Query(
		ctx context.Context,
		p RetrievalPeer,
		payloadCID cid.Cid,
		params QueryParams,
	) (QueryResponse, error)

	// Retrieve retrieves all or part of a piece with the given retrieval parameters
	Retrieve(
		ctx context.Context,
		id DealID,
		payloadCID cid.Cid,
		params Params,
		totalFunds abi.TokenAmount,
		p RetrievalPeer,
		clientWallet address.Address,
		minerWallet address.Address,
	) (DealID, error)

	// SubscribeToEvents listens for events that happen related to client retrievals
	SubscribeToEvents(subscriber ClientSubscriber) Unsubscribe

	// TryRestartInsufficientFunds attempts to restart any deals stuck in the insufficient funds state
	// after funds are added to a given payment channel
	TryRestartInsufficientFunds(paymentChannel address.Address) error

	// CancelDeal attempts to cancel an inprogress deal
	CancelDeal(id DealID) error

	// GetDeal returns a given deal by deal ID, if it exists
	GetDeal(dealID DealID) (ClientDealState, error)

	// ListDeals returns all deals
	ListDeals() (map[DealID]ClientDealState, error)
}

RetrievalClient is a client interface for making retrieval deals

type RetrievalClientNode 

type RetrievalClientNode interface {
	GetChainHead(ctx context.Context) (shared.TipSetToken, abi.ChainEpoch, error)

	// GetOrCreatePaymentChannel sets up a new payment channel if one does not exist
	// between a client and a miner and ensures the client has the given amount of funds available in the channel
	GetOrCreatePaymentChannel(ctx context.Context, clientAddress, minerAddress address.Address,
		clientFundsAvailable abi.TokenAmount, tok shared.TipSetToken) (address.Address, cid.Cid, error)

	// CheckAvailableFunds returns the amount of current and incoming funds in a channel
	CheckAvailableFunds(ctx context.Context, paymentChannel address.Address) (ChannelAvailableFunds, error)

	// Allocate late creates a lane within a payment channel so that calls to
	// CreatePaymentVoucher will automatically make vouchers only for the difference
	// in total
	AllocateLane(ctx context.Context, paymentChannel address.Address) (uint64, error)

	// CreatePaymentVoucher creates a new payment voucher in the given lane for a
	// given payment channel so that all the payment vouchers in the lane add up
	// to the given amount (so the payment voucher will be for the difference)
	CreatePaymentVoucher(ctx context.Context, paymentChannel address.Address, amount abi.TokenAmount,
		lane uint64, tok shared.TipSetToken) (*paychtypes.SignedVoucher, error)

	// WaitForPaymentChannelReady just waits for the payment channel's pending operations to complete
	WaitForPaymentChannelReady(ctx context.Context, waitSentinel cid.Cid) (address.Address, error)

	// GetKnownAddresses gets any on known multiaddrs for a given address, so we can add to the peer store
	GetKnownAddresses(ctx context.Context, p RetrievalPeer, tok shared.TipSetToken) ([]ma.Multiaddr, error)
}

RetrievalClientNode are the node dependencies for a RetrievalClient

type RetrievalPeer 

type RetrievalPeer struct {
	Address  address.Address
	ID       peer.ID // optional
	PieceCID *cid.Cid
}

RetrievalPeer is a provider address/peer.ID pair (everything needed to make deals for with a miner)

func (*RetrievalPeer) MarshalCBOR 

func (t *RetrievalPeer) MarshalCBOR(w io.Writer) error

func (*RetrievalPeer) UnmarshalCBOR 

func (t *RetrievalPeer) UnmarshalCBOR(r io.Reader) (err error)

type RetrievalProvider 

type RetrievalProvider interface {
	// Start begins listening for deals on the given host
	Start(ctx context.Context) error

	// OnReady registers a listener for when the provider comes on line
	OnReady(shared.ReadyFunc)

	// Stop stops handling incoming requests
	Stop() error

	// SetAsk sets the retrieval payment parameters that this miner will accept
	SetAsk(ask *Ask)

	// GetAsk returns the retrieval providers pricing information
	GetAsk() *Ask

	// GetDynamicAsk quotes a dynamic price for the retrieval deal by calling the user configured
	// dynamic pricing function. It passes the static price parameters set in the Ask Store to the pricing function.
	GetDynamicAsk(ctx context.Context, input PricingInput, storageDeals []abi.DealID) (Ask, error)

	// SubscribeToEvents listens for events that happen related to client retrievals
	SubscribeToEvents(subscriber ProviderSubscriber) Unsubscribe

	// SubscribeToQueryEvents subscribes to an event that is fired when a message
	// is received on the query protocol
	SubscribeToQueryEvents(subscriber ProviderQueryEventSubscriber) Unsubscribe

	// SubscribeToValidationEvents subscribes to an event that is fired when the
	// provider validates a request for data
	SubscribeToValidationEvents(subscriber ProviderValidationSubscriber) Unsubscribe

	ListDeals() map[ProviderDealIdentifier]ProviderDealState
}

RetrievalProvider is an interface by which a provider configures their retrieval operations and monitors deals received and process

type RetrievalProviderNode 

type RetrievalProviderNode interface {
	GetChainHead(ctx context.Context) (shared.TipSetToken, abi.ChainEpoch, error)

	// returns the worker address associated with a miner
	GetMinerWorkerAddress(ctx context.Context, miner address.Address, tok shared.TipSetToken) (address.Address, error)
	SavePaymentVoucher(ctx context.Context, paymentChannel address.Address, voucher *paychtypes.SignedVoucher, proof []byte, expectedAmount abi.TokenAmount, tok shared.TipSetToken) (abi.TokenAmount, error)

	GetRetrievalPricingInput(ctx context.Context, pieceCID cid.Cid, storageDeals []abi.DealID) (PricingInput, error)
}

RetrievalProviderNode are the node dependencies for a RetrievalProvider

type RetrieveResponse 

type RetrieveResponse struct {
	DealID      DealID
	CarFilePath string
}

type SectorAccessor 

type SectorAccessor interface {
	UnsealSector(ctx context.Context, sectorID abi.SectorNumber, offset abi.UnpaddedPieceSize, length abi.UnpaddedPieceSize) (io.ReadCloser, error)
	IsUnsealed(ctx context.Context, sectorID abi.SectorNumber, offset abi.UnpaddedPieceSize, length abi.UnpaddedPieceSize) (bool, error)
}

SectorAccessor provides methods to unseal and get the seal status of a sector

type ShortfallError 

type ShortfallError struct {
	// contains filtered or unexported fields
}

ShortfallErorr is an error that indicates a short fall of funds

func (ShortfallError) Error 

func (se ShortfallError) Error() string

func (ShortfallError) Shortfall 

func (se ShortfallError) Shortfall() abi.TokenAmount

Shortfall returns the numerical value of the shortfall

type Unsubscribe 

type Unsubscribe func()

Unsubscribe is a function that unsubscribes a subscriber for either the client or the provider

Source Files

View all Source files

Directories

Path Synopsis
Package retrievalimpl provides the primary implementation of retrieval market top level interfaces interfaces
 Package retrievalimpl provides the primary implementation of retrieval market top level interfaces interfaces
askstore
clientstates
Package clientstates contains state machine logic relating to the `RetrievalClient`.
 Package clientstates contains state machine logic relating to the `RetrievalClient`.
dtutils
Package dtutils provides event listeners for the client and provider to listen for events on the data transfer module and dispatch FSM events based on them
 Package dtutils provides event listeners for the client and provider to listen for events on the data transfer module and dispatch FSM events based on them
providerstates
Package providerstates contains state machine logic relating to the `RetrievalProvider`.
 Package providerstates contains state machine logic relating to the `RetrievalProvider`.
requestvalidation
testnodes
Package testnodes contains stubbed implementations of the RetrievalProviderNode and RetrievalClientNode interface to simulate communications with a filecoin node
 Package testnodes contains stubbed implementations of the RetrievalProviderNode and RetrievalClientNode interface to simulate communications with a filecoin node
maptypes
Package network providers an abstraction over a libp2p host for managing retrieval's Libp2p protocols:
 Package network providers an abstraction over a libp2p host for managing retrieval's Libp2p protocols:
Package testing provides test implementations of retieval market interfaces
 Package testing provides test implementations of retieval market interfaces

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.