piecestore

README

piecestore

The piecestore module is a simple encapsulation of two data stores, one for PieceInfo and another for CIDInfo. The piecestore's main goal is to help storagemarket module and retrievalmarket module find where sealed data lives inside of sectors. Storage market writes the data, and retrieval market reads it.

Both markets use CIDInfo to look up a Piece that contains the payload, and then use PieceInfo to find the sector that contains the piece.

The storage market has to write this data before it completes the deal in order to later look up the payload when the data is served.

Installation

go get github.com/filecoin-project/go-fil-markets/piecestore

PieceStore

PieceStore is primary export of this module. It is a database of piece info that can be modified and queried. The PieceStore interface is implemented in piecestore.go.

It has two stores, one for PieceInfo keyed by pieceCID, and another for CIDInfo, keyed by payloadCID. These keys are of type cid.CID; see github.com/ipfs/go-cid.

To initialize a PieceStore

func NewPieceStore(ds datastore.Batching) PieceStore

Parameters

• ds datastore.Batching is a datastore for the deal's state machine. It is typically the node's own datastore that implements the IPFS datastore.Batching interface. See github.com/ipfs/go-datastore.

PieceStore implements the following functions:

• AddDealForPiece
• AddPieceBlockLocations
• GetPieceInfo
• GetCIDInfo

Please the tests for more information about expected behavior.

Documentation

Index

• Variables
• type BlockLocation
  • func (t *BlockLocation) MarshalCBOR(w io.Writer) error
  • func (t *BlockLocation) UnmarshalCBOR(r io.Reader) (err error)
• type CIDInfo
  • func (t *CIDInfo) MarshalCBOR(w io.Writer) error
  • func (t *CIDInfo) UnmarshalCBOR(r io.Reader) (err error)
• type DealInfo
  • func (t *DealInfo) MarshalCBOR(w io.Writer) error
  • func (t *DealInfo) UnmarshalCBOR(r io.Reader) (err error)
• type PieceBlockLocation
  • func (t *PieceBlockLocation) MarshalCBOR(w io.Writer) error
  • func (t *PieceBlockLocation) UnmarshalCBOR(r io.Reader) (err error)
• type PieceInfo
  • func (pi PieceInfo) Defined() bool
  • func (t *PieceInfo) MarshalCBOR(w io.Writer) error
  • func (t *PieceInfo) UnmarshalCBOR(r io.Reader) (err error)
• type PieceStore

Variables

var CIDInfoUndefined = CIDInfo{}

CIDInfoUndefined is cid info with no information

var PieceInfoUndefined = PieceInfo{}

PieceInfoUndefined is piece info with no information

Types

type BlockLocation

type BlockLocation struct {
	RelOffset uint64
	BlockSize uint64
}

BlockLocation is information about where a given block is relative to the overall piece

func (*BlockLocation) MarshalCBOR

func (t *BlockLocation) MarshalCBOR(w io.Writer) error

func (*BlockLocation) UnmarshalCBOR

func (t *BlockLocation) UnmarshalCBOR(r io.Reader) (err error)

type CIDInfo

type CIDInfo struct {
	CID                 cid.Cid
	PieceBlockLocations []PieceBlockLocation
}

CIDInfo is information about where a given CID will live inside a piece

func (*CIDInfo) MarshalCBOR

func (t *CIDInfo) MarshalCBOR(w io.Writer) error

func (*CIDInfo) UnmarshalCBOR

func (t *CIDInfo) UnmarshalCBOR(r io.Reader) (err error)

type DealInfo

type DealInfo struct {
	DealID   abi.DealID
	SectorID abi.SectorNumber
	Offset   abi.PaddedPieceSize
	Length   abi.PaddedPieceSize
}

DealInfo is information about a single deal for a given piece

func (*DealInfo) MarshalCBOR

func (t *DealInfo) MarshalCBOR(w io.Writer) error

func (*DealInfo) UnmarshalCBOR

func (t *DealInfo) UnmarshalCBOR(r io.Reader) (err error)

type PieceBlockLocation

type PieceBlockLocation struct {
	BlockLocation
	PieceCID cid.Cid
}

PieceBlockLocation is block information along with the pieceCID of the piece the block is inside of

func (*PieceBlockLocation) MarshalCBOR

func (t *PieceBlockLocation) MarshalCBOR(w io.Writer) error

func (*PieceBlockLocation) UnmarshalCBOR

func (t *PieceBlockLocation) UnmarshalCBOR(r io.Reader) (err error)

type PieceInfo

type PieceInfo struct {
	PieceCID cid.Cid
	Deals    []DealInfo
}

PieceInfo is metadata about a piece a provider may be storing based on its PieceCID -- so that, given a pieceCID during retrieval, the miner can determine how to unseal it if needed

func (PieceInfo) Defined

func (pi PieceInfo) Defined() bool

func (*PieceInfo) MarshalCBOR

func (t *PieceInfo) MarshalCBOR(w io.Writer) error

func (*PieceInfo) UnmarshalCBOR

func (t *PieceInfo) UnmarshalCBOR(r io.Reader) (err error)

type PieceStore

type PieceStore interface {
	Start(ctx context.Context) error
	OnReady(ready shared.ReadyFunc)
	AddDealForPiece(pieceCID cid.Cid, payloadCid cid.Cid, dealInfo DealInfo) error
	AddPieceBlockLocations(pieceCID cid.Cid, blockLocations map[cid.Cid]BlockLocation) error
	GetPieceInfo(pieceCID cid.Cid) (PieceInfo, error)
	GetCIDInfo(payloadCID cid.Cid) (CIDInfo, error)
	ListCidInfoKeys() ([]cid.Cid, error)
	ListPieceInfoKeys() ([]cid.Cid, error)
}

PieceStore is a saved database of piece info that can be modified and queried