objectstorage

package
v1.0.0-beta.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 12, 2022 License: Apache-2.0, BSD-2-Clause Imports: 8 Imported by: 11

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewStoreWithRealm 

func NewStoreWithRealm(store kvstore.KVStore, packagePrefix byte, storagePrefix byte) kvstore.KVStore

NewStoreWithRealm creates a new kvstore.KVStore with the given the store and prefixes.

Types

type CachedObject 

type CachedObject[T StorableObject] struct {
	// contains filtered or unexported fields
}

CachedObject is a wrapper around a value that is stored in the object storage. It provides necessary function that object storage needs to correctly handle the object.

func NewEmptyCachedObject 

func NewEmptyCachedObject[T StorableObject](key []byte) (result *CachedObject[T])

NewEmptyCachedObject creates an "empty" CachedObject, that is not part of any ObjectStorage.

Sometimes, we want to be able to offer a "filtered view" on the ObjectStorage and therefore be able to return an "empty" value on load operations even if the underlying object exists (i.e. the value tangle on top of the normal tangle only returns value transactions in its load operations).

func (*CachedObject[T]) BatchWrite 

func (c *CachedObject[T]) BatchWrite(batchedMuts kvstore.BatchedMutations)

BatchWrite checks if the cachedObject should be persisted. If all checks pass, the cachedObject is marshaled and added to the BatchedMutations. Do not call this method for objects that should not be persisted.

func (*CachedObject[T]) BatchWriteDone 

func (c *CachedObject[T]) BatchWriteDone()

BatchWriteDone is called after the cachedObject was persisted. It releases the cachedObject from the cache if no consumers are left and it was not modified in the meantime.

func (*CachedObject[T]) BatchWriteScheduled 

func (c *CachedObject[T]) BatchWriteScheduled() bool

BatchWriteScheduled returns true if the cachedObject is already scheduled for a BatchWrite operation.

func (*CachedObject[T]) Consume 

func (c *CachedObject[T]) Consume(consumer func(T), forceRelease ...bool) bool

Consume directly consumes the StorableObject. This method automatically Release()s the object when the callback is done. Returns true if the callback was called.

func (*CachedObject[T]) Exists 

func (c *CachedObject[T]) Exists() bool

Exists returns true if the StorableObject in this container does exist (could be found in the database and was not marked as deleted).

func (*CachedObject[T]) Get 

func (c *CachedObject[T]) Get() (result T)

Get retrieves the StorableObject, that is cached in this container.

func (*CachedObject[T]) Key 

func (c *CachedObject[T]) Key() []byte

Key returns the object storage key that is used to address the object.

func (*CachedObject[T]) RTransaction 

func (c *CachedObject[T]) RTransaction(callback func(object T), identifiers ...interface{}) *CachedObject[T]

RTransaction is a synchronization primitive that executes the callback together with other RTransactions but never together with a normal Transaction.

The identifiers allow to define the scope of the RTransaction. RTransactions with different scopes can run at the same time independently of other RTransactions and act as if they are secured by different mutexes.

It is also possible to provide multiple identifiers and the callback waits until all of them can be acquired at the same time. In contrast to normal mutexes where acquiring multiple locks can lead to deadlocks, this method is deadlock safe.

Note: It is the equivalent of a mutex.RLock/RUnlock.

func (*CachedObject[T]) Release 

func (c *CachedObject[T]) Release(force ...bool)

Release the object, to be picked up by the persistence layer (as soon as all consumers are done).

func (*CachedObject[T]) ResetBatchWriteScheduled 

func (c *CachedObject[T]) ResetBatchWriteScheduled()

ResetBatchWriteScheduled resets the flag that the cachedObject is scheduled for a BatchWrite operation.

func (*CachedObject[T]) Retain 

func (c *CachedObject[T]) Retain() *CachedObject[T]

Retain registers a new consumer for this cached object.

func (*CachedObject[T]) Transaction 

func (c *CachedObject[T]) Transaction(callback func(object T), identifiers ...interface{}) *CachedObject[T]

Transaction is a synchronization primitive that executes the callback atomically which means that if multiple Transactions are being started from different goroutines, then only one of them can run at the same time.

The identifiers allow to define the scope of the Transaction. Transactions with different scopes can run at the same time and act as if they are secured by different mutexes.

It is also possible to provide multiple identifiers and the callback waits until all of them can be acquired at the same time. In contrast to normal mutexes where acquiring multiple locks can lead to deadlocks, this method is deadlock safe.

Note: It is the equivalent of a mutex.Lock/Unlock.

func (*CachedObject[T]) Unwrap 

func (c *CachedObject[T]) Unwrap() (result T, exists bool)

Unwrap returns the underlying object with correct type.

type CachedObjects 

type CachedObjects[T StorableObject] []*CachedObject[T]

CachedObjects represents a collection of CachedObject objects.

func (CachedObjects[T]) Consume 

func (c CachedObjects[T]) Consume(consumer func(T), forceRelease ...bool) (consumed bool)

Consume iterates over the CachedObjects, unwraps them and passes a type-casted version to the consumer (if the object is not empty - it exists). It automatically releases the object when the consumer finishes. It returns true, if at least one object was consumed.

func (CachedObjects[T]) Exists 

func (c CachedObjects[T]) Exists() (exists []bool)

Exists returns a slice of boolean values to indicate whether element at a given index exists.

func (CachedObjects[T]) Release 

func (c CachedObjects[T]) Release(force ...bool)

Release is a utility function that allows us to release all CachedObjects in the collection.

func (CachedObjects[T]) String 

func (c CachedObjects[T]) String() string

String returns a human readable version of the CachedObjects.

func (CachedObjects[T]) Unwrap 

func (c CachedObjects[T]) Unwrap(skip ...bool) (unwrappedChildBranches []T)

Unwrap is the type-casted equivalent of Get. It returns a slice of unwrapped objects and optionally skips any objects that do not exist or are deleted, sets default type value for missing elements.

type Events 

type Events = objectstorage.Events

type IteratorOption 

type IteratorOption = objectstorage.IteratorOption

func WithIteratorMaxIterations 

func WithIteratorMaxIterations(maxIterations int) IteratorOption

WithIteratorMaxIterations is used to stop the iteration after a certain amount of iterations. 0 disables the limit.

func WithIteratorPrefix 

func WithIteratorPrefix(prefix []byte) IteratorOption

WithIteratorPrefix is used to iterate a subset of elements with a defined prefix.

func WithIteratorSkipCache 

func WithIteratorSkipCache(skipCache bool) IteratorOption

WithIteratorSkipCache is used to skip the elements in the cache.

func WithIteratorSkipStorage 

func WithIteratorSkipStorage(skipStorage bool) IteratorOption

WithIteratorSkipStorage is used to skip the elements in the storage.

type IteratorOptions 

type IteratorOptions = objectstorage.IteratorOptions

type LeakDetectionOptions 

type LeakDetectionOptions = objectstorage.LeakDetectionOptions

type LeakDetectionWrapper 

type LeakDetectionWrapper = objectstorage.LeakDetectionWrapper

type ObjectStorage 

type ObjectStorage[T StorableObject] struct {
	Events *Events

	*objectstorage.ObjectStorage
}

ObjectStorage is a manual cache which keeps objects as long as consumers are using it.

func NewInterfaceStorage 

func NewInterfaceStorage[T StorableObject](store kvstore.KVStore, objectFactory StorableObjectFactory, optionalOptions ...Option) (newObjectStorage *ObjectStorage[T])

NewInterfaceStorage is the constructor for the ObjectStorage that stores interface types.

func NewStructStorage 

func NewStructStorage[U any, T PtrStorableObject[U]](store kvstore.KVStore, optionalOptions ...Option) (newObjectStorage *ObjectStorage[T])

NewStructStorage is the constructor for the ObjectStorage that stores struct types.

func (*ObjectStorage[T]) ComputeIfAbsent 

func (o *ObjectStorage[T]) ComputeIfAbsent(key []byte, remappingFunction func(key []byte) T) *CachedObject[T]

ComputeIfAbsent computes and returns the default value if the given key is not in the ObjectStorage.

func (*ObjectStorage[T]) Contains 

func (o *ObjectStorage[T]) Contains(key []byte, options ...ReadOption) (result bool)

Contains returns true if the given key is in the ObjectStorage.

func (*ObjectStorage[T]) Delete 

func (o *ObjectStorage[T]) Delete(key []byte)

Delete performs a "blind delete", where we do not check the object's existence. blindDelete is used to delete without accessing the value log.

func (*ObjectStorage[T]) DeleteIfPresent 

func (o *ObjectStorage[T]) DeleteIfPresent(key []byte) bool

DeleteIfPresent deletes an element and return true if the element was deleted.

func (*ObjectStorage[T]) DeleteIfPresentAndReturn 

func (o *ObjectStorage[T]) DeleteIfPresentAndReturn(key []byte) T

DeleteIfPresentAndReturn deletes an element and returns it. If the element does not exist then the return value is nil.

func (*ObjectStorage[T]) Flush 

func (o *ObjectStorage[T]) Flush()

Flush writes all objects from cache to the underlying store.

func (*ObjectStorage[T]) ForEach 

func (o *ObjectStorage[T]) ForEach(consumer func(key []byte, cachedObject *CachedObject[T]) bool, options ...IteratorOption)

ForEach calls the consumer function on every object residing within the cache and the underlying persistence layer.

func (*ObjectStorage[T]) ForEachKeyOnly 

func (o *ObjectStorage[T]) ForEachKeyOnly(consumer func(key []byte) bool, options ...IteratorOption)

ForEachKeyOnly calls the consumer function on every storage key residing within the cache and the underlying persistence layer.

func (*ObjectStorage[T]) FreeMemory 

func (o *ObjectStorage[T]) FreeMemory()

FreeMemory copies the content of the internal maps to newly created maps. This is necessary, otherwise the GC is not able to free the memory used by the old maps. "delete" doesn't shrink the maximum memory used by the map, since it only marks the entry as deleted.

func (*ObjectStorage[T]) Get 

func (o *ObjectStorage[T]) Get(key []byte) *CachedObject[T]

Get returns the object for the given key from cache. If object is not in cache, returns nil.

func (*ObjectStorage[T]) GetSize 

func (o *ObjectStorage[T]) GetSize() int

GetSize returns the size of the ObjectStorage.

func (*ObjectStorage[T]) Load 

func (o *ObjectStorage[T]) Load(key []byte) *CachedObject[T]

Load returns the object for the given key. If object is not found in cache, tries to load the object from underlying store. Can only be used with persistence enabled.

func (*ObjectStorage[T]) Prune 

func (o *ObjectStorage[T]) Prune() error

Prune removes all values from the ObjectStorage

func (*ObjectStorage[T]) Put 

func (o *ObjectStorage[T]) Put(object T) *CachedObject[T]

Put adds the given object in the ObjectStorage cache.

func (*ObjectStorage[T]) ReleaseExecutor 

func (o *ObjectStorage[T]) ReleaseExecutor() (releaseExecutor *timedexecutor.TimedExecutor)

ReleaseExecutor returns the executor that schedules releases of CachedObjects after the configured CacheTime.

func (*ObjectStorage[T]) Shutdown 

func (o *ObjectStorage[T]) Shutdown()

Shutdown shuts down the ObjectStorage.

func (*ObjectStorage[T]) Store 

func (o *ObjectStorage[T]) Store(object T) *CachedObject[T]

Store stores the given object in the ObjectStorage.

func (*ObjectStorage[T]) StoreIfAbsent 

func (o *ObjectStorage[T]) StoreIfAbsent(object T) (result *CachedObject[T], stored bool)

StoreIfAbsent stores an object only if it was not stored before. In contrast to "ComputeIfAbsent", this method does not access the value log. If the object was not stored, then the returned CachedObject is nil and does not need to be Released.

type Option 

type Option = objectstorage.Option

func CacheTime 

func CacheTime(duration time.Duration) Option

CacheTime sets the time after which the object is evicted from the cache.

func KeysOnly 

func KeysOnly(keysOnly bool) Option

KeysOnly is used to store only the keys of the elements.

func LeakDetectionEnabled 

func LeakDetectionEnabled(leakDetectionEnabled bool, options ...LeakDetectionOptions) Option

LeakDetectionEnabled enables the leak detection of the object storage.

func LogAccess 

func LogAccess(fileName string, commandsFilter ...debug.Command) Option

LogAccess sets up a logger that logs all calls to the underlying store in the given file. It is possible to filter the logged commands by providing an optional filter flag.

func OnEvictionCallback 

func OnEvictionCallback[T StorableObject](cb func(cachedObject *CachedObject[T])) Option

OnEvictionCallback sets a function that is called on eviction of the object.

func OverrideLeakDetectionWrapper 

func OverrideLeakDetectionWrapper[T StorableObject](wrapperFunc func(cachedObject *CachedObject[T]) LeakDetectionWrapper) Option

OverrideLeakDetectionWrapper is used to override the default leak detection wrapper.

func PartitionKey 

func PartitionKey(keyPartitions ...int) Option

PartitionKey sets the partition sizes of the key.

func PersistenceEnabled 

func PersistenceEnabled(persistenceEnabled bool) Option

PersistenceEnabled enables the persistence of the object storage.

func ReleaseExecutorWorkerCount 

func ReleaseExecutorWorkerCount(releaseExecutorWorkerCount int) Option

ReleaseExecutorWorkerCount sets the number of workers that execute the scheduled eviction of the objects in parallel (whenever they become due).

func StoreOnCreation 

func StoreOnCreation(store bool) Option

StoreOnCreation writes an object directly to the persistence layer on creation.

type Options 

type Options = objectstorage.Options

type PtrStorableObject 

type PtrStorableObject[T any] interface {
	*T

	StorableObject
}

PtrStorableObject is a wrapper type that holds a pointer to a StorableObject.

type ReadOption 

type ReadOption = objectstorage.ReadOption

func WithReadSkipCache 

func WithReadSkipCache(skipCache bool) ReadOption

WithReadSkipCache is used to skip the elements in the cache.

func WithReadSkipStorage 

func WithReadSkipStorage(skipStorage bool) ReadOption

WithReadSkipStorage is used to skip the elements in the storage.

type ReadOptions 

type ReadOptions = objectstorage.ReadOptions

type StorableObject 

type StorableObject interface {
	FromObjectStorage(key, data []byte) error

	objectstorage.StorableObject
}

StorableObject is an interface to be implemented by an object that is stored in the ObjectStorage.

type StorableObjectFactory 

type StorableObjectFactory func(key []byte, data []byte) (result StorableObject, err error)

StorableObjectFactory is a function that creates new StorageObject from a key and data.

type StorableObjectFlags 

type StorableObjectFlags = objectstorage.StorableObjectFlags

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.