carstore

Documentation

Index

• Constants
• Variables
• type AdBlock
  • func (a AdBlock) Advertisement() (schema.Advertisement, error)
• type CarReader
  • func NewReader(fileStore filestore.Interface, options ...Option) (*CarReader, error)
  • func (cr CarReader) CarPath(adCid cid.Cid) string
  • func (cr CarReader) Compression() string
  • func (cr CarReader) Read(ctx context.Context, adCid cid.Cid, skipEntries bool) (*AdBlock, error)
  • func (cr CarReader) ReadHead(ctx context.Context, provider peer.ID) (cid.Cid, error)
• type CarWriter
  • func NewWriter(dstore datastore.Batching, fileStore filestore.Interface, options ...Option) (*CarWriter, error)
  • func (cw *CarWriter) CarPath(adCid cid.Cid) string
  • func (cw *CarWriter) CleanupAdData(ctx context.Context, adCid cid.Cid, skipEntries bool) error
  • func (cw *CarWriter) Compression() string
  • func (cw *CarWriter) Delete(ctx context.Context, adCid cid.Cid) error
  • func (cw *CarWriter) DeleteHead(ctx context.Context, provider peer.ID) error
  • func (cw *CarWriter) Write(ctx context.Context, adCid cid.Cid, skipEntries, noOverwrite bool) (*filestore.File, error)
  • func (cw *CarWriter) WriteChain(ctx context.Context, adCid cid.Cid, noOverwrite bool) (int, error)
  • func (cw *CarWriter) WriteHead(ctx context.Context, adCid cid.Cid, provider peer.ID) (*filestore.File, error)
• type EntryBlock
  • func (e EntryBlock) EntryChunk() (*schema.EntryChunk, error)
• type Option
  • func WithCompress(alg string) Option

Constants

const (
	CarFileSuffix  = ".car"
	GzipFileSuffix = ".gz"
	HeadFileSuffix = ".head"
)

File suffixes.

const (
	Gzip = "gzip"
)

Compression methods.

Variables

var ErrHAMT = errors.New("hamt entries not supported")

Types

type AdBlock

type AdBlock struct {
	Cid     cid.Cid
	Data    []byte
	Entries <-chan EntryBlock
}

AdBlock contains schema.Advertisement data.

func (AdBlock) Advertisement

func (a AdBlock) Advertisement() (schema.Advertisement, error)

type CarReader

type CarReader struct {
	// contains filtered or unexported fields
}

func NewReader

func NewReader(fileStore filestore.Interface, options ...Option) (*CarReader, error)

NewReader creates a CarReader that reads CAR files from the given filestore and returns advertisements and entries.

func (CarReader) CarPath

func (cr CarReader) CarPath(adCid cid.Cid) string

CarPath returns the name of the CAR file being read.

func (CarReader) Compression

func (cr CarReader) Compression() string

Compression returns the name of the compression used to compress CAR files.

func (CarReader) Read

func (cr CarReader) Read(ctx context.Context, adCid cid.Cid, skipEntries bool) (*AdBlock, error)

Read reads an advertisement CAR file, identitfied by the advertisement CID and returns the advertisement data and a channel to read blocks of multihash entries. Returns fs.ErrNotExist if file is not found.

func (CarReader) ReadHead

func (cr CarReader) ReadHead(ctx context.Context, provider peer.ID) (cid.Cid, error)

ReadHead reads the advertisement CID from the provider's head file. The head file contains the CID of the latest advertisement for a provider. Returns fs.ErrNotExist if head file is not found.

type CarWriter

type CarWriter struct {
	// contains filtered or unexported fields
}

CarWriter writes advertisements and entries, that are stored in a datastore, to CAR files. Each advertisement and its associated multihash entries blocks, or HAMT, are written to a single CAR file. The original chunks or HAMT are preserved, as opposed to storing only multihashes, for the purpose of being able to verify links from advertisements and verify the advertisement signature. Such verification may be necessary when fetching car files from a location that is not completely trusted.

func NewWriter

func NewWriter(dstore datastore.Batching, fileStore filestore.Interface, options ...Option) (*CarWriter, error)

NewWriter create a new CarWriter that reads advertisement data from the given datastore and writes car files to the specified directory.

func (*CarWriter) CarPath

func (cw *CarWriter) CarPath(adCid cid.Cid) string

CarPath returns the name of the CAR file being written.

func (*CarWriter) CleanupAdData

func (cw *CarWriter) CleanupAdData(ctx context.Context, adCid cid.Cid, skipEntries bool) error

CleanupAdData removes advertisement and associated entries block data from the data store.

func (*CarWriter) Compression

func (cw *CarWriter) Compression() string

Compression returns the name of the compression used to compress CAR files.

func (*CarWriter) Delete

func (cw *CarWriter) Delete(ctx context.Context, adCid cid.Cid) error

func (*CarWriter) DeleteHead

func (cw *CarWriter) DeleteHead(ctx context.Context, provider peer.ID) error

func (*CarWriter) Write

func (cw *CarWriter) Write(ctx context.Context, adCid cid.Cid, skipEntries, noOverwrite bool) (*filestore.File, error)

Write reads the advertisement, specified by CID, from the datastore and writes the advertisement and its entries into a CAR file. The CAR file is named with the advertisement CID, and is stored using the FileStore provided when the CarWriter was created. The advertisement and entries data are always removed from the datastore.

If the CAR file already exists, it is overwritten unless noOverwrite is true. If noOverwrite is true, and the file exists, then fs.ErrExist is returned along with the filestore File information describing the existing file.

The CAR file is written without entries if skipEntries is true. The purpose of this to create a CAR file, to maintain the link in the advertisement chain, when it is know that a later advertisement deletes this advertisement's entries.

func (*CarWriter) WriteChain

func (cw *CarWriter) WriteChain(ctx context.Context, adCid cid.Cid, noOverwrite bool) (int, error)

WriteChain reads the advertisement, specified by CID, from the datastore and writes it and its entries into a CAR file. If the advertisement has a previous advertisement, then it is written also. This continues until there are no more previous advertisements in the datastore or until an advertisement does not have a previous.

func (*CarWriter) WriteHead

func (cw *CarWriter) WriteHead(ctx context.Context, adCid cid.Cid, provider peer.ID) (*filestore.File, error)

type EntryBlock

type EntryBlock struct {
	Cid  cid.Cid
	Data []byte
	Err  error
}

EntryBlock contains schema.EntryChunk data.

func (EntryBlock) EntryChunk

func (e EntryBlock) EntryChunk() (*schema.EntryChunk, error)

type Option

type Option func(*config) error

Option is a function that sets a value in a config.

func WithCompress

func WithCompress(alg string) Option

WithCompress configures file compression.