pebbleds

README

go-ds-pebble: Pebble-backed datastore

🐣 Status: experimental A datastore implementation using cockroachdb/pebble (a native-Go RocksDB equivalent) as a backend.

This is a simple adapter to plug in cockroachdb/pebble as a backend anywhere that accepts a go-datastore.

Amongst other software, this includes:

• go-ipfs
• go-libp2p
• Lotus, the reference Filecoin implementation written in Go.

Status

This implementation is experimental. It is currently exercised against the test suite under go-datastore.

The road to maturity includes:

• Benchmarks against go-ds-badger, go-ds-badger2, and go-ds-leveldb.
• Exposing metrics.
• Testing it in IPFS, libp2p, and Lotus, and characterising its behaviour, performance and footprint under real, practical workloads.

Contribute

Feel free to join in. All welcome. Open an issue!

This repository falls under the IPFS Code of Conduct.

Want to hack on IPFS?

License

Dual-licensed under MIT + Apache 2.0

Documentation

Index

• type Batch
  • func (b *Batch) Commit(ctx context.Context) error
  • func (b *Batch) Delete(ctx context.Context, key ds.Key) error
  • func (b *Batch) Put(ctx context.Context, key ds.Key, value []byte) error
• type Datastore
  • func NewDatastore(path string, opts *pebble.Options) (*Datastore, error)
  • func (d *Datastore) Batch(ctx context.Context) (ds.Batch, error)
  • func (d *Datastore) Close() error
  • func (d *Datastore) Delete(ctx context.Context, key ds.Key) error
  • func (d *Datastore) DiskUsage(ctx context.Context) (uint64, error)
  • func (d *Datastore) Get(ctx context.Context, key ds.Key) (value []byte, err error)
  • func (d *Datastore) GetSize(ctx context.Context, key ds.Key) (size int, _ error)
  • func (d *Datastore) Has(ctx context.Context, key ds.Key) (exists bool, _ error)
  • func (d *Datastore) Put(ctx context.Context, key ds.Key, value []byte) error
  • func (d *Datastore) Query(ctx context.Context, q query.Query) (query.Results, error)
  • func (d *Datastore) Sync(ctx context.Context, _ ds.Key) error

Types

type Batch

type Batch struct {
	// contains filtered or unexported fields
}

func (*Batch) Commit

func (b *Batch) Commit(ctx context.Context) error

func (*Batch) Delete

func (b *Batch) Delete(ctx context.Context, key ds.Key) error

func (*Batch) Put

func (b *Batch) Put(ctx context.Context, key ds.Key, value []byte) error

type Datastore

type Datastore struct {
	// contains filtered or unexported fields
}

Datastore is a pebble-backed github.com/ipfs/go-datastore.Datastore.

It supports batching. It does not support TTL or transactions, because pebble doesn't have those features.

func NewDatastore

func NewDatastore(path string, opts *pebble.Options) (*Datastore, error)

NewDatastore creates a pebble-backed datastore.

Users can provide pebble options or rely on Pebble's defaults. There is one exception though: opts.Comparer.Split is always overwriten, so that lookups can take advantage of bloom filters. This assumes a regular datastore usage where advanced key-versioning and performance features that Pebbles are offers are unused, but instead we care more about responding quickly to Has() and Get() lookups, particularly when keys are not in the datastore.

func (*Datastore) Batch

func (d *Datastore) Batch(ctx context.Context) (ds.Batch, error)

func (*Datastore) Close

func (d *Datastore) Close() error

func (*Datastore) Delete

func (d *Datastore) Delete(ctx context.Context, key ds.Key) error

func (*Datastore) DiskUsage

func (d *Datastore) DiskUsage(ctx context.Context) (uint64, error)

DiskUsage implements the PersistentDatastore interface and returns current size on disk.

func (*Datastore) Get

func (d *Datastore) Get(ctx context.Context, key ds.Key) (value []byte, err error)

Get reads a key from the datastore.

func (*Datastore) GetSize

func (d *Datastore) GetSize(ctx context.Context, key ds.Key) (size int, _ error)

func (*Datastore) Has

func (d *Datastore) Has(ctx context.Context, key ds.Key) (exists bool, _ error)

Has can be used to check whether a key is stored in the datastore. Has() calls are not cheaper than Get() though. In Pebble, lookups for existing keys will also read the values. Avoid using Has() if you later expect to read the key anyways. Has() calls for non-existing keys should take advantage of bloom filters and avoid reads.

func (*Datastore) Put

func (d *Datastore) Put(ctx context.Context, key ds.Key, value []byte) error

func (*Datastore) Query

func (d *Datastore) Query(ctx context.Context, q query.Query) (query.Results, error)

func (*Datastore) Sync

func (d *Datastore) Sync(ctx context.Context, _ ds.Key) error