pebbleds

README

go-ds-pebble: Pebble-backed datastore

🐣 Status: experimental A datastore implementation using cockroachdb/pebble (a native-Go RocksDB equivalent) as a backend.

This is a simple adapter to plug in cockroachdb/pebble as a backend anywhere that accepts a go-datastore.

Amongst other software, this includes:

• go-ipfs
• go-libp2p
• Lotus, the reference Filecoin implementation written in Go.

Status

This implementation is experimental. It is currently exercised against the test suite under go-datastore.

The road to maturity includes:

• Benchmarks against go-ds-badger, go-ds-badger2, and go-ds-leveldb.
• Exposing metrics.
• Testing it in IPFS, libp2p, and Lotus, and characterising its behaviour, performance and footprint under real, practical workloads.

Contribute

Feel free to join in. All welcome. Open an issue!

This repository falls under the IPFS Code of Conduct.

Want to hack on IPFS?

License

Dual-licensed under MIT + Apache 2.0

Documentation

Index

• type Batch
  • func (b *Batch) Commit(ctx context.Context) error
  • func (b *Batch) Delete(ctx context.Context, key ds.Key) error
  • func (b *Batch) Put(ctx context.Context, key ds.Key, value []byte) error
• type Datastore
  • func NewDatastore(path string, options ...Option) (*Datastore, error)
  • func (d *Datastore) Batch(ctx context.Context) (ds.Batch, error)
  • func (d *Datastore) Close() error
  • func (d *Datastore) Delete(ctx context.Context, key ds.Key) error
  • func (d *Datastore) DiskUsage(ctx context.Context) (uint64, error)
  • func (d *Datastore) Get(ctx context.Context, key ds.Key) (value []byte, err error)
  • func (d *Datastore) GetSize(ctx context.Context, key ds.Key) (size int, _ error)
  • func (d *Datastore) Has(ctx context.Context, key ds.Key) (exists bool, _ error)
  • func (d *Datastore) Put(ctx context.Context, key ds.Key, value []byte) error
  • func (d *Datastore) Query(ctx context.Context, q query.Query) (query.Results, error)
  • func (d *Datastore) Sync(ctx context.Context, _ ds.Key) error
• type Option
  • func WithCacheSize(size int64) Option
  • func WithPebbleDB(db *pebble.DB) Option
  • func WithPebbleOpts(opts *pebble.Options) Option

Types

type Batch

type Batch struct {
	// contains filtered or unexported fields
}

func (*Batch) Commit

func (b *Batch) Commit(ctx context.Context) error

func (*Batch) Delete

func (b *Batch) Delete(ctx context.Context, key ds.Key) error

func (*Batch) Put

func (b *Batch) Put(ctx context.Context, key ds.Key, value []byte) error

type Datastore

type Datastore struct {
	// contains filtered or unexported fields
}

Datastore is a pebble-backed github.com/ipfs/go-datastore.Datastore.

It supports batching. It does not support TTL or transactions, because pebble doesn't have those features.

func NewDatastore

func NewDatastore(path string, options ...Option) (*Datastore, error)

NewDatastore creates a pebble-backed datastore.

Users can provide pebble options using WithPebbleOpts or rely on Pebble's defaults. Any pebble options that are not assigned a value are assigned pebble's default value for the option.

func (*Datastore) Batch

func (d *Datastore) Batch(ctx context.Context) (ds.Batch, error)

func (*Datastore) Close

func (d *Datastore) Close() error

func (*Datastore) Delete

func (d *Datastore) Delete(ctx context.Context, key ds.Key) error

func (*Datastore) DiskUsage

func (d *Datastore) DiskUsage(ctx context.Context) (uint64, error)

DiskUsage implements the PersistentDatastore interface and returns current size on disk.

func (*Datastore) Get

func (d *Datastore) Get(ctx context.Context, key ds.Key) (value []byte, err error)

Get reads a key from the datastore.

func (*Datastore) GetSize

func (d *Datastore) GetSize(ctx context.Context, key ds.Key) (size int, _ error)

func (*Datastore) Has

func (d *Datastore) Has(ctx context.Context, key ds.Key) (exists bool, _ error)

Has can be used to check whether a key is stored in the datastore. Has() calls are not cheaper than Get() though. In Pebble, lookups for existing keys will also read the values. Avoid using Has() if you later expect to read the key anyways.

func (*Datastore) Put

func (d *Datastore) Put(ctx context.Context, key ds.Key, value []byte) error

func (*Datastore) Query

func (d *Datastore) Query(ctx context.Context, q query.Query) (query.Results, error)

func (*Datastore) Sync

func (d *Datastore) Sync(ctx context.Context, _ ds.Key) error

type Option

type Option func(*config)

func WithCacheSize

func WithCacheSize(size int64) Option

WithCacheSize configures the size of pebble's shared block cache. A value of 0 (the default) uses the default cache size.

func WithPebbleDB

func WithPebbleDB(db *pebble.DB) Option

WithPebbleDB is used to configure the Datastore with a custom DB.

func WithPebbleOpts

func WithPebbleOpts(opts *pebble.Options) Option

WithPebbleOpts sets any/all configurable values for pebble. If not set, the default configuration values are used. Any unspecified value in opts is replaced by the default value.