carbs

README

🍔 Carbs

Car Blockstore provides a read-only blockstore interface directly reading out of a car file.

License

Carbs is dual-licensed under Apache 2.0 and MIT terms:

Apache License, Version 2.0, (LICENSE or http://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

Documentation

Index

• Variables
• func Generate(path string, codec IndexCodec) error
• func Save(i Index, path string) error
• type Carbs
  • func Load(path string, noPersist bool) (*Carbs, error)
  • func (c *Carbs) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error)
  • func (c *Carbs) DeleteBlock(_ cid.Cid) error
  • func (c *Carbs) Get(key cid.Cid) (blocks.Block, error)
  • func (c *Carbs) GetSize(key cid.Cid) (int, error)
  • func (c *Carbs) Has(key cid.Cid) (bool, error)
  • func (c *Carbs) HashOnRead(enabled bool)
  • func (c *Carbs) Put(blocks.Block) error
  • func (c *Carbs) PutMany([]blocks.Block) error
  • func (c *Carbs) Read(idx int64) (cid.Cid, []byte, error)
  • func (c *Carbs) Roots() ([]cid.Cid, error)
• type Index
  • func GenerateIndex(store io.ReaderAt, size int64, codec IndexCodec, verbose bool) (Index, error)
  • func Restore(path string) (Index, error)
• type IndexCls
• type IndexCodec
• type Record

Variables

var IndexAtlas = map[IndexCodec]IndexCls{
	IndexHashed:       mkHashed,
	IndexSorted:       mkSorted,
	IndexSingleSorted: mkSingleSorted,
	IndexGobHashed:    mkGobHashed,
}

IndexAtlas holds known index formats

Functions

func Generate

func Generate(path string, codec IndexCodec) error

Generate walks a car file and generates an index of cid->byte offset in it.

func Save

func Save(i Index, path string) error

Save writes a generated index for a car at `path`

Types

type Carbs

type Carbs struct {
	// contains filtered or unexported fields
}

Carbs provides a read-only Car Block Store.

func Load

func Load(path string, noPersist bool) (*Carbs, error)

Load opens a carbs data store, generating an index if it does not exist

func (*Carbs) AllKeysChan

func (c *Carbs) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error)

AllKeysChan returns the list of keys in the store

func (*Carbs) DeleteBlock

func (c *Carbs) DeleteBlock(_ cid.Cid) error

DeleteBlock doesn't delete a block on RO blockstore

func (*Carbs) Get

func (c *Carbs) Get(key cid.Cid) (blocks.Block, error)

Get gets a block from the store

func (*Carbs) GetSize

func (c *Carbs) GetSize(key cid.Cid) (int, error)

GetSize gets how big a item is

func (*Carbs) Has

func (c *Carbs) Has(key cid.Cid) (bool, error)

Has indicates if the store has a cid

func (*Carbs) HashOnRead

func (c *Carbs) HashOnRead(enabled bool)

HashOnRead does nothing

func (*Carbs) Put

func (c *Carbs) Put(blocks.Block) error

Put does nothing on a ro store

func (*Carbs) PutMany

func (c *Carbs) PutMany([]blocks.Block) error

PutMany does nothing on a ro store

func (*Carbs) Read

func (c *Carbs) Read(idx int64) (cid.Cid, []byte, error)

func (*Carbs) Roots

func (c *Carbs) Roots() ([]cid.Cid, error)

Roots returns the root CIDs of the backing car

type Index

type Index interface {
	Codec() IndexCodec
	Marshal(w io.Writer) error
	Unmarshal(r io.Reader) error
	Get(cid.Cid) (uint64, error)
	Load([]Record) error
}

Index provides an interface for figuring out where in the car a given cid begins

func GenerateIndex

func GenerateIndex(store io.ReaderAt, size int64, codec IndexCodec, verbose bool) (Index, error)

GenerateIndex provides a low-level interface to create an index over a reader to a car stream.

func Restore

func Restore(path string) (Index, error)

Restore loads an index from an on-disk representation.

type IndexCls

type IndexCls func() Index

IndexCls is a constructor for an index type

type IndexCodec

type IndexCodec int

IndexCodec is used as a multicodec identifier for carbs index files

const (
	IndexHashed IndexCodec = iota + 0x300000
	IndexSorted
	IndexSingleSorted
	IndexGobHashed
)

IndexCodec table is a first var-int in carbs indexes

type Record

type Record struct {
	cid.Cid
	// contains filtered or unexported fields
}

Record is a pre-processed record of a car item and location.