carstore

Documentation

Index

• Constants
• Variables
• type AdBlock
  • func (a AdBlock) Advertisement() (schema.Advertisement, error)
• type CarReader
  • func NewReader(fileStore filestore.Interface, options ...Option) (*CarReader, error)
  • func (cr CarReader) Compression() string
  • func (cr CarReader) Read(ctx context.Context, adCid cid.Cid, skipEntries bool) (*AdBlock, error)
  • func (cr CarReader) ReadHead(ctx context.Context, publisher peer.ID) (cid.Cid, error)
• type CarWriter
  • func NewWriter(dstore datastore.Batching, fileStore filestore.Interface, options ...Option) (*CarWriter, error)
  • func (cw *CarWriter) Compression() string
  • func (cw *CarWriter) Write(ctx context.Context, adCid cid.Cid, skipEntries, overWrite bool) (*filestore.File, error)
  • func (cw *CarWriter) WriteChain(ctx context.Context, adCid cid.Cid, overWrite bool) (int, error)
  • func (cw *CarWriter) WriteHead(ctx context.Context, adCid cid.Cid, publisher peer.ID) (*filestore.File, error)
• type EntryBlock
  • func (e EntryBlock) EntryChunk() (*schema.EntryChunk, error)
• type Option
  • func WithCompress(alg string) Option

Constants

const (
	CarFileSuffix  = ".car"
	GzipFileSuffix = ".gz"
	HeadFileSuffix = ".head"
)

File suffixes.

const (
	Gzip = "gzip"
)

Compression methods.

Variables

var ErrHAMT = errors.New("hamt entries not supported")

Types

type AdBlock

type AdBlock struct {
	Cid     cid.Cid
	Data    []byte
	Entries <-chan EntryBlock
}

AdBlock contains schema.Advertisement dat.

func (AdBlock) Advertisement

func (a AdBlock) Advertisement() (schema.Advertisement, error)

type CarReader

type CarReader struct {
	// contains filtered or unexported fields
}

func NewReader

func NewReader(fileStore filestore.Interface, options ...Option) (*CarReader, error)

NewReader creates a CarReader that reads CAR files from the given filestore and returns advertisements and entries.

func (CarReader) Compression

func (cr CarReader) Compression() string

func (CarReader) Read

func (cr CarReader) Read(ctx context.Context, adCid cid.Cid, skipEntries bool) (*AdBlock, error)

Read reads an advertisement CAR file, identitfied by the advertisement CID and returns the advertisement data and a channel to read blocks of multihash entries. Returns fs.ErrNotExist if file is not found.

func (CarReader) ReadHead

func (cr CarReader) ReadHead(ctx context.Context, publisher peer.ID) (cid.Cid, error)

ReadHead reads the advertisement CID from the publisher's head file. The head file contains the CID of the latest advertisement for an advertisement publisher. Returns fs.ErrNotExist if head file is not found.

type CarWriter

type CarWriter struct {
	// contains filtered or unexported fields
}

CarWriter writes advertisements and entries, that are stored in a datastore, to CAR files. Each advertisement and its associated multihash entries blocks, or HAMT, are written to a single CAR file. The original chunks or HAMT are preserved, as opposed to storing only multihashes, for the purpose of being able to verify links from advertisements and verify the advertisement signature. Such verification may be necessary when fetching car files from a location that is not completely trusted.

func NewWriter

func NewWriter(dstore datastore.Batching, fileStore filestore.Interface, options ...Option) (*CarWriter, error)

NewWriter create a new CarWriter that reads advertisement data from the given datastore and writes car files to the specified directory.

func (*CarWriter) Compression

func (cw *CarWriter) Compression() string

Compression returns the name of the compression used to compress CAR files.

func (*CarWriter) Write

func (cw *CarWriter) Write(ctx context.Context, adCid cid.Cid, skipEntries, overWrite bool) (*filestore.File, error)

Write reads the advertisement, specified by CID, from the datastore and writes the advertisement and its entries into a CAR file. The CAR file is named with the advertisement CID, and is stored using the FileStore provided when the CarWriter was created. The advertisement and entries data are always removed from the datastore.

If the CAR file already exists, it is not overwritten and fs.ErrExist is returned. When this happens the filestore.File information describing the existing file is also returned.

The CAR file is written without entries if skipEntries is true. The purpose of this to create a CAR file, to maintain the link in the advertisement chain, when it is know that a later advertisement deletes this advertisement's entries.

func (*CarWriter) WriteChain

func (cw *CarWriter) WriteChain(ctx context.Context, adCid cid.Cid, overWrite bool) (int, error)

WriteChain reads the advertisement, specified by CID, from the datastore and writes it and its entries into a CAR file. If the advertisement has a previous advertisement, then it is written also. This continues until there are no more previous advertisements in the datastore or until an advertisement does not have a previous.

func (*CarWriter) WriteHead

func (cw *CarWriter) WriteHead(ctx context.Context, adCid cid.Cid, publisher peer.ID) (*filestore.File, error)

type EntryBlock

type EntryBlock struct {
	Cid  cid.Cid
	Data []byte
	Err  error
}

EntryBlock contains schema.EntryChunk data.

func (EntryBlock) EntryChunk

func (e EntryBlock) EntryChunk() (*schema.EntryChunk, error)

type Option

type Option func(*config) error

Option is a function that sets a value in a config.

func WithCompress

func WithCompress(alg string) Option

WithCompress configures file compression.