Documentation ¶
Index ¶
- Constants
- Variables
- func DefaultPebbleOptions(cache *pebble.Cache, comparer *pebble.Comparer) *pebble.Options
- func IsBootstrapped(db *pebble.DB) (bool, error)
- func NewBootstrappedRegistersWithPathForTest(tb testing.TB, dir string, first, latest uint64) *pebble.DB
- func OpenDefaultPebbleDB(dir string) (*pebble.DB, error)
- func OpenRegisterPebbleDB(dir string) (*pebble.DB, error)
- func ReadHeightsFromBootstrappedDB(db *pebble.DB) (firstHeight uint64, latestHeight uint64, err error)
- func RunWithRegistersStorageAtInitialHeights(tb testing.TB, first uint64, latest uint64, f func(r *Registers))
- type Batch
- type Cache
- type CacheBackend
- type CacheType
- type ChunkDataPacks
- func (ch *ChunkDataPacks) BatchRemove(chunkID flow.Identifier, batch storage.BatchStorage) error
- func (ch *ChunkDataPacks) ByChunkID(chunkID flow.Identifier) (*flow.ChunkDataPack, error)
- func (ch *ChunkDataPacks) Remove(cs []flow.Identifier) error
- func (ch *ChunkDataPacks) Store(cs []*flow.ChunkDataPack) error
- type ReadCache
- type RegisterBootstrap
- type Registers
- type RegistersCache
Constants ¶
const DefaultCacheSize = uint(10_000)
const DefaultPebbleCacheSize = 1 << 20
const ( // MinLookupKeyLen defines the minimum length for a valid lookup key // // Lookup keys use the following format: // [code] [owner] / [key] / [height] // Where: // - code: 1 byte indicating the type of data stored // - owner: optional variable length field // - key: optional variable length field // - height: 8 bytes representing the block height (uint64) // - separator: '/' is used to separate variable length fields (required 2) // // Therefore the minimum key would be 3 bytes + # of bytes for height // [code] / / [height] MinLookupKeyLen = 3 + registers.HeightSuffixLen )
Variables ¶
var ErrAlreadyBootstrapped = errors.New("found latest key set on badger instance, DB is already bootstrapped")
ErrAlreadyBootstrapped is the sentinel error for an already bootstrapped pebble instance
Functions ¶
func DefaultPebbleOptions ¶
DefaultPebbleOptions returns an optimized set of pebble options. This is mostly copied form pebble's nightly performance benchmark.
func IsBootstrapped ¶
IsBootstrapped returns true if the db is bootstrapped otherwise return false it returns error if the db is corrupted or other exceptions
func NewBootstrappedRegistersWithPathForTest ¶ added in v0.32.7
func OpenDefaultPebbleDB ¶ added in v0.33.36
OpenDefaultPebbleDB opens a pebble database using default options, such as cache size and comparer
func OpenRegisterPebbleDB ¶
OpenRegisterPebbleDB opens the database The difference between OpenDefaultPebbleDB is that it uses a customized comparer (NewMVCCComparer) which is needed to implement finding register values at any given height using pebble's SeekPrefixGE function
func ReadHeightsFromBootstrappedDB ¶
func ReadHeightsFromBootstrappedDB(db *pebble.DB) (firstHeight uint64, latestHeight uint64, err error)
ReadHeightsFromBootstrappedDB reads the first and latest height from a bootstrapped register db If the register db is not bootstrapped, it returns storage.ErrNotBootstrapped If the register db is corrupted, it returns an error
Types ¶
type Batch ¶ added in v0.33.36
type Batch struct {
// contains filtered or unexported fields
}
type Cache ¶ added in v0.33.36
type Cache[K comparable, V any] struct { // contains filtered or unexported fields }
Cache is a read-through cache for underlying storage layer. Note: when a resource is not found in the cache nor the underlying storage, then it will not be cached. In other words, finding the missing item again will query the underlying storage again.
func (*Cache[K, V]) Get ¶ added in v0.33.36
Get will try to retrieve the resource from cache first, and then from the injected. During normal operations, the following error returns are expected:
- `storage.ErrNotFound` if key is unknown.
func (*Cache[K, V]) Insert ¶ added in v0.33.36
func (c *Cache[K, V]) Insert(key K, resource V)
Insert will add a resource directly to the cache with the given ID assuming the resource has been added to storage already.
func (*Cache[K, V]) IsCached ¶ added in v0.33.36
IsCached returns true if the key exists in the cache. It DOES NOT check whether the key exists in the underlying data store.
type CacheBackend ¶ added in v0.33.12
type ChunkDataPacks ¶ added in v0.33.36
type ChunkDataPacks struct {
// contains filtered or unexported fields
}
func NewChunkDataPacks ¶ added in v0.33.36
func NewChunkDataPacks(collector module.CacheMetrics, db *pebble.DB, collections storage.Collections, byChunkIDCacheSize uint) *ChunkDataPacks
func (*ChunkDataPacks) BatchRemove ¶ added in v0.33.36
func (ch *ChunkDataPacks) BatchRemove(chunkID flow.Identifier, batch storage.BatchStorage) error
BatchRemove is not used in pebble implementation
func (*ChunkDataPacks) ByChunkID ¶ added in v0.33.36
func (ch *ChunkDataPacks) ByChunkID(chunkID flow.Identifier) (*flow.ChunkDataPack, error)
ByChunkID finds the chunk data pack by chunk ID. it returns storage.ErrNotFound if not found other errors are exceptions
func (*ChunkDataPacks) Remove ¶ added in v0.33.36
func (ch *ChunkDataPacks) Remove(cs []flow.Identifier) error
Remove removes chunk data packs by IDs, it removes them atomically. Any errors are exceptions
func (*ChunkDataPacks) Store ¶ added in v0.33.36
func (ch *ChunkDataPacks) Store(cs []*flow.ChunkDataPack) error
Store stores the given chunk data pack lists, it stores them atomically. Any error are exceptions
type ReadCache ¶ added in v0.33.12
type ReadCache struct {
// contains filtered or unexported fields
}
func (*ReadCache) Get ¶ added in v0.33.12
func (c *ReadCache) Get(key string) (flow.RegisterValue, error)
Get will try to retrieve the resource from cache first, and then from the injected. During normal operations, the following error returns are expected:
- `storage.ErrNotFound` if key is unknown.
func (*ReadCache) Insert ¶ added in v0.33.12
func (c *ReadCache) Insert(key string, resource flow.RegisterValue)
Insert will add a resource directly to the cache with the given ID
type RegisterBootstrap ¶
type RegisterBootstrap struct {
// contains filtered or unexported fields
}
func NewRegisterBootstrap ¶
func NewRegisterBootstrap( db *pebble.DB, checkpointFile string, rootHeight uint64, rootHash ledger.RootHash, log zerolog.Logger, ) (*RegisterBootstrap, error)
NewRegisterBootstrap creates the bootstrap object for reading checkpoint data and the height tracker in pebble This object must be initialized and RegisterBootstrap.IndexCheckpointFile must be run to have the pebble db instance in the correct state to initialize a Registers store.
func (*RegisterBootstrap) IndexCheckpointFile ¶
func (b *RegisterBootstrap) IndexCheckpointFile(ctx context.Context, workerCount int) error
IndexCheckpointFile indexes the checkpoint file in the Dir provided
type Registers ¶
type Registers struct {
// contains filtered or unexported fields
}
Registers library that implements pebble storage for registers given a pebble instance with root block and root height populated
func NewBootstrappedRegistersWithPath ¶
NewBootstrappedRegistersWithPath initializes a new Registers instance with a pebble db if the database is not initialized, it close the database and return storage.ErrNotBootstrapped
func NewRegisters ¶
NewRegisters takes a populated pebble instance with LatestHeight and FirstHeight set. return storage.ErrNotBootstrapped if they those two keys are unavailable as it implies a uninitialized state return other error if database is in a corrupted state
func (*Registers) FirstHeight ¶
FirstHeight first indexed height found in the store, typically root block for the spork
func (*Registers) Get ¶
func (s *Registers) Get( reg flow.RegisterID, height uint64, ) (flow.RegisterValue, error)
Get returns the most recent updated payload for the given RegisterID. "most recent" means the updates happens most recent up the given height.
For example, if there are 2 values stored for register A at height 6 and 11, then GetPayload(13, A) would return the value at height 11.
- storage.ErrNotFound if no register values are found - storage.ErrHeightNotIndexed if the requested height is out of the range of stored heights
func (*Registers) LatestHeight ¶
LatestHeight Gets the latest height of complete registers available
func (*Registers) Store ¶
func (s *Registers) Store( entries flow.RegisterEntries, height uint64, ) error
Store sets the given entries in a batch. This function is expected to be called at one batch per height, sequentially. Under normal conditions, it should be called wth the value of height set to LatestHeight + 1 CAUTION: This function is not safe for concurrent use.
type RegistersCache ¶ added in v0.33.12
type RegistersCache struct { *Registers // contains filtered or unexported fields }
func NewRegistersCache ¶ added in v0.33.12
func NewRegistersCache(registers *Registers, cacheType CacheType, size uint, metrics module.CacheMetrics) (*RegistersCache, error)
NewRegistersCache wraps a read cache around Get requests to a underlying Registers object.
func (*RegistersCache) Get ¶ added in v0.33.12
func (c *RegistersCache) Get( reg flow.RegisterID, height uint64, ) (flow.RegisterValue, error)
Get returns the most recent updated payload for the given RegisterID. "most recent" means the updates happens most recent up the given height.
For example, if there are 2 values stored for register A at height 6 and 11, then GetPayload(13, A) would return the value at height 11.
- storage.ErrNotFound if no register values are found - storage.ErrHeightNotIndexed if the requested height is out of the range of stored heights