stoabs

package module
v1.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 8, 2024 License: GPL-3.0 Imports: 12 Imported by: 5

README

Golang Storage Abstraction (go-stoabs)

BBolt

Redis

When creating a Redis KVStore it tests the connection using Redis' PING command.

Due to the simple API of the library, the Redis adapter only supports reading/writing byte arrays. The behavior when reading any other Redis type (e.g. a list or set) is undefined.

Transaction Isolation

Redis doesn't have actual transactions, so this library simulates them by using the MULTI/EXEC/DISCARD commands. That way all commands are guaranteed to be executed atomically.

As a consequence, a value that hasn't been committed yet can't be read. In other words, don't try to read a value from a key that was written to in the same transaction. Subsequently, changes from other writers (from the same process or remote) are reflected immediately in the current transaction: if a key is read twice, there's no guarantee the returned value will be equal.

If the application requires exclusive write access to a store it can lock the database, to assert there are no other active writers:

store.Write(func (tx stoabs.WriteTx) error { 
	// do something with tx
}, stoabs.WithWriteLock())

The lock is released when the transaction is committed or rolled back. The lock is subject to the prefix (CreateRedisStore(prefix string, ...)) the store was created with, meaning other stores with the same prefix will have the same lock.

Redis locks are implemented using (Redsync)[https://github.com/go-redsync/redsync].

Unsupported features
  • Clustering

Documentation

Overview

Code generated by MockGen. DO NOT EDIT. Source: store.go

Generated by this command:

mockgen -destination=mock.go -package stoabs -source=store.go

Package stoabs is a generated GoMock package.

Index

Constants

View Source
const DefaultTransactionTimeout = 30 * time.Second

Variables

View Source
var ErrCommitFailed = DatabaseError(errors.New("unable to commit transaction"))

ErrCommitFailed is returned when the commit of transaction fails. Is also a ErrDatabase.

View Source
var ErrKeyNotFound = errors.New("key not found")

ErrKeyNotFound is returned when the requested key does not exist

View Source
var ErrStoreIsClosed = DatabaseError(errors.New("database not open"))

ErrStoreIsClosed is returned when an operation is executed on a closed store. Is also a ErrDatabase.

Functions

func DatabaseError added in v1.2.0

func DatabaseError(err error) error

DatabaseError wraps the given error in ErrDatabase if it isn't already in the error chain.

Types

type AfterCommitOption

type AfterCommitOption struct {
	// contains filtered or unexported fields
}

AfterCommitOption see AfterCommit

func (AfterCommitOption) Invoke

func (o AfterCommitOption) Invoke(opts []TxOption)

Invoke calls all functions registered with the AfterCommitOption.

type BytesKey

type BytesKey []byte

BytesKey is a type helper for a byte slice as Key

func (BytesKey) Bytes

func (b BytesKey) Bytes() []byte

func (BytesKey) Equals

func (b BytesKey) Equals(other Key) bool

func (BytesKey) FromBytes

func (b BytesKey) FromBytes(i []byte) (Key, error)

func (BytesKey) FromString

func (b BytesKey) FromString(i string) (Key, error)

func (BytesKey) Next

func (b BytesKey) Next() Key

func (BytesKey) String

func (b BytesKey) String() string

type CallerFn

type CallerFn func(key Key, value []byte) error

CallerFn is the function type which is called for each key value pair when using Iterate() or Range()

type Config

type Config struct {
	Log                *logrus.Logger
	NoSync             bool
	LockAcquireTimeout time.Duration
}

Config specifies the configuration for a KVStore.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns the default configuration.

type ErrDatabase added in v1.2.0

type ErrDatabase struct {
	// contains filtered or unexported fields
}

ErrDatabase signals that the wrapped error is related to database access, or due to context cancellation/timeout. The action that resulted in this error may succeed when retried.

func (ErrDatabase) Error added in v1.2.0

func (e ErrDatabase) Error() string

func (ErrDatabase) Is added in v1.2.0

func (e ErrDatabase) Is(other error) bool

func (ErrDatabase) Unwrap added in v1.2.0

func (e ErrDatabase) Unwrap() error

type HashKey

type HashKey [32]byte

HashKey is a type helper for a 256 bits hash as Key

func (HashKey) Bytes

func (s HashKey) Bytes() []byte

func (HashKey) Equals

func (s HashKey) Equals(other Key) bool

func (HashKey) FromBytes

func (s HashKey) FromBytes(i []byte) (Key, error)

func (HashKey) FromString

func (s HashKey) FromString(i string) (Key, error)

func (HashKey) Next

func (s HashKey) Next() Key

func (HashKey) String

func (s HashKey) String() string

type KVStore

type KVStore interface {
	Store
	// Write starts a writable transaction and passes it to the given function.
	// Callers should not try to read values which are written in the same transactions, and thus haven't been committed yet.
	// The result when doing so depends on transaction isolation of the underlying database.
	// The passed context can be used to cancel long-running operations or the final commit of the transaction.
	Write(ctx context.Context, fn func(WriteTx) error, opts ...TxOption) error
	// Read starts a read-only transaction and passes it to the given function.
	// The passed context can be used to cancel long-running read operations.
	Read(ctx context.Context, fn func(ReadTx) error) error
	// WriteShelf starts a writable transaction, open a writer for the specified shelf and passes it to the given function.
	// If the shelf does not exist, it will be created.
	// The same semantics of Write apply.
	// The passed context can be used to cancel long-running operations or the final commit of the transaction.
	WriteShelf(ctx context.Context, shelfName string, fn func(Writer) error) error
	// ReadShelf starts a read-only transaction, open a reader for the specified shelf and passes it to the given function.
	// If the shelf does not exist, the function is not called.
	// The passed context can be used to cancel long-running read operations.
	ReadShelf(ctx context.Context, shelfName string, fn func(Reader) error) error
}

KVStore defines the interface for a key-value store. Writing to it is done in callbacks passed to the Write-functions. If the callback returns an error, the transaction is rolled back. Methods return a ErrDatabase when the context has been cancelled or timed-out.

type Key

type Key interface {
	fmt.Stringer
	// FromString creates a new instance of this Key, parses it from the given string and returns it. It does not modify this key.
	FromString(string) (Key, error)
	// Bytes returns a byte representation of this key
	Bytes() []byte
	// FromBytes creates a new instance of this Key, parses it from the given byte slice and returns it. It does not modify this key.
	FromBytes([]byte) (Key, error)
	// Next returns the next logical key. The next for the number 1 would be 2. The next for the string "a" would be "b"
	Next() Key
	// Equals returns true when the given key is of same type and value.
	Equals(other Key) bool
}

Key is an abstraction for a key in a key/value pair. The underlying implementation determines if the string or byte representation is used.

func NewHashKey

func NewHashKey(bytes [32]byte) Key

NewHashKey creates a new HashKey from bytes

type MockKVStore

type MockKVStore struct {
	// contains filtered or unexported fields
}

MockKVStore is a mock of KVStore interface.

func NewMockKVStore

func NewMockKVStore(ctrl *gomock.Controller) *MockKVStore

NewMockKVStore creates a new mock instance.

func (*MockKVStore) Close

func (m *MockKVStore) Close(ctx context.Context) error

Close mocks base method.

func (*MockKVStore) EXPECT

func (m *MockKVStore) EXPECT() *MockKVStoreMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockKVStore) Read

func (m *MockKVStore) Read(ctx context.Context, fn func(ReadTx) error) error

Read mocks base method.

func (*MockKVStore) ReadShelf

func (m *MockKVStore) ReadShelf(ctx context.Context, shelfName string, fn func(Reader) error) error

ReadShelf mocks base method.

func (*MockKVStore) Write

func (m *MockKVStore) Write(ctx context.Context, fn func(WriteTx) error, opts ...TxOption) error

Write mocks base method.

func (*MockKVStore) WriteShelf

func (m *MockKVStore) WriteShelf(ctx context.Context, shelfName string, fn func(Writer) error) error

WriteShelf mocks base method.

type MockKVStoreMockRecorder

type MockKVStoreMockRecorder struct {
	// contains filtered or unexported fields
}

MockKVStoreMockRecorder is the mock recorder for MockKVStore.

func (*MockKVStoreMockRecorder) Close

func (mr *MockKVStoreMockRecorder) Close(ctx any) *gomock.Call

Close indicates an expected call of Close.

func (*MockKVStoreMockRecorder) Read

func (mr *MockKVStoreMockRecorder) Read(ctx, fn any) *gomock.Call

Read indicates an expected call of Read.

func (*MockKVStoreMockRecorder) ReadShelf

func (mr *MockKVStoreMockRecorder) ReadShelf(ctx, shelfName, fn any) *gomock.Call

ReadShelf indicates an expected call of ReadShelf.

func (*MockKVStoreMockRecorder) Write

func (mr *MockKVStoreMockRecorder) Write(ctx, fn any, opts ...any) *gomock.Call

Write indicates an expected call of Write.

func (*MockKVStoreMockRecorder) WriteShelf

func (mr *MockKVStoreMockRecorder) WriteShelf(ctx, shelfName, fn any) *gomock.Call

WriteShelf indicates an expected call of WriteShelf.

type MockReadTx

type MockReadTx struct {
	// contains filtered or unexported fields
}

MockReadTx is a mock of ReadTx interface.

func NewMockReadTx

func NewMockReadTx(ctrl *gomock.Controller) *MockReadTx

NewMockReadTx creates a new mock instance.

func (*MockReadTx) EXPECT

func (m *MockReadTx) EXPECT() *MockReadTxMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockReadTx) GetShelfReader

func (m *MockReadTx) GetShelfReader(shelfName string) Reader

GetShelfReader mocks base method.

func (*MockReadTx) Store

func (m *MockReadTx) Store() KVStore

Store mocks base method.

func (*MockReadTx) Unwrap

func (m *MockReadTx) Unwrap() any

Unwrap mocks base method.

type MockReadTxMockRecorder

type MockReadTxMockRecorder struct {
	// contains filtered or unexported fields
}

MockReadTxMockRecorder is the mock recorder for MockReadTx.

func (*MockReadTxMockRecorder) GetShelfReader

func (mr *MockReadTxMockRecorder) GetShelfReader(shelfName any) *gomock.Call

GetShelfReader indicates an expected call of GetShelfReader.

func (*MockReadTxMockRecorder) Store

func (mr *MockReadTxMockRecorder) Store() *gomock.Call

Store indicates an expected call of Store.

func (*MockReadTxMockRecorder) Unwrap

func (mr *MockReadTxMockRecorder) Unwrap() *gomock.Call

Unwrap indicates an expected call of Unwrap.

type MockReader

type MockReader struct {
	// contains filtered or unexported fields
}

MockReader is a mock of Reader interface.

func NewMockReader

func NewMockReader(ctrl *gomock.Controller) *MockReader

NewMockReader creates a new mock instance.

func (*MockReader) EXPECT

func (m *MockReader) EXPECT() *MockReaderMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockReader) Empty added in v1.7.1

func (m *MockReader) Empty() (bool, error)

Empty mocks base method.

func (*MockReader) Get

func (m *MockReader) Get(key Key) ([]byte, error)

Get mocks base method.

func (*MockReader) Iterate

func (m *MockReader) Iterate(callback CallerFn, keyType Key) error

Iterate mocks base method.

func (*MockReader) Range

func (m *MockReader) Range(from, to Key, callback CallerFn, stopAtNil bool) error

Range mocks base method.

func (*MockReader) Stats

func (m *MockReader) Stats() ShelfStats

Stats mocks base method.

type MockReaderMockRecorder

type MockReaderMockRecorder struct {
	// contains filtered or unexported fields
}

MockReaderMockRecorder is the mock recorder for MockReader.

func (*MockReaderMockRecorder) Empty added in v1.7.1

func (mr *MockReaderMockRecorder) Empty() *gomock.Call

Empty indicates an expected call of Empty.

func (*MockReaderMockRecorder) Get

func (mr *MockReaderMockRecorder) Get(key any) *gomock.Call

Get indicates an expected call of Get.

func (*MockReaderMockRecorder) Iterate

func (mr *MockReaderMockRecorder) Iterate(callback, keyType any) *gomock.Call

Iterate indicates an expected call of Iterate.

func (*MockReaderMockRecorder) Range

func (mr *MockReaderMockRecorder) Range(from, to, callback, stopAtNil any) *gomock.Call

Range indicates an expected call of Range.

func (*MockReaderMockRecorder) Stats

func (mr *MockReaderMockRecorder) Stats() *gomock.Call

Stats indicates an expected call of Stats.

type MockStore

type MockStore struct {
	// contains filtered or unexported fields
}

MockStore is a mock of Store interface.

func NewMockStore

func NewMockStore(ctrl *gomock.Controller) *MockStore

NewMockStore creates a new mock instance.

func (*MockStore) Close

func (m *MockStore) Close(ctx context.Context) error

Close mocks base method.

func (*MockStore) EXPECT

func (m *MockStore) EXPECT() *MockStoreMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

type MockStoreMockRecorder

type MockStoreMockRecorder struct {
	// contains filtered or unexported fields
}

MockStoreMockRecorder is the mock recorder for MockStore.

func (*MockStoreMockRecorder) Close

func (mr *MockStoreMockRecorder) Close(ctx any) *gomock.Call

Close indicates an expected call of Close.

type MockTxOption

type MockTxOption struct {
	// contains filtered or unexported fields
}

MockTxOption is a mock of TxOption interface.

func NewMockTxOption

func NewMockTxOption(ctrl *gomock.Controller) *MockTxOption

NewMockTxOption creates a new mock instance.

func (*MockTxOption) EXPECT

EXPECT returns an object that allows the caller to indicate expected use.

type MockTxOptionMockRecorder

type MockTxOptionMockRecorder struct {
	// contains filtered or unexported fields
}

MockTxOptionMockRecorder is the mock recorder for MockTxOption.

type MockWriteTx

type MockWriteTx struct {
	// contains filtered or unexported fields
}

MockWriteTx is a mock of WriteTx interface.

func NewMockWriteTx

func NewMockWriteTx(ctrl *gomock.Controller) *MockWriteTx

NewMockWriteTx creates a new mock instance.

func (*MockWriteTx) EXPECT

func (m *MockWriteTx) EXPECT() *MockWriteTxMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockWriteTx) GetShelfReader

func (m *MockWriteTx) GetShelfReader(shelfName string) Reader

GetShelfReader mocks base method.

func (*MockWriteTx) GetShelfWriter

func (m *MockWriteTx) GetShelfWriter(shelfName string) Writer

GetShelfWriter mocks base method.

func (*MockWriteTx) Store

func (m *MockWriteTx) Store() KVStore

Store mocks base method.

func (*MockWriteTx) Unwrap

func (m *MockWriteTx) Unwrap() any

Unwrap mocks base method.

type MockWriteTxMockRecorder

type MockWriteTxMockRecorder struct {
	// contains filtered or unexported fields
}

MockWriteTxMockRecorder is the mock recorder for MockWriteTx.

func (*MockWriteTxMockRecorder) GetShelfReader

func (mr *MockWriteTxMockRecorder) GetShelfReader(shelfName any) *gomock.Call

GetShelfReader indicates an expected call of GetShelfReader.

func (*MockWriteTxMockRecorder) GetShelfWriter

func (mr *MockWriteTxMockRecorder) GetShelfWriter(shelfName any) *gomock.Call

GetShelfWriter indicates an expected call of GetShelfWriter.

func (*MockWriteTxMockRecorder) Store

func (mr *MockWriteTxMockRecorder) Store() *gomock.Call

Store indicates an expected call of Store.

func (*MockWriteTxMockRecorder) Unwrap

func (mr *MockWriteTxMockRecorder) Unwrap() *gomock.Call

Unwrap indicates an expected call of Unwrap.

type MockWriter

type MockWriter struct {
	// contains filtered or unexported fields
}

MockWriter is a mock of Writer interface.

func NewMockWriter

func NewMockWriter(ctrl *gomock.Controller) *MockWriter

NewMockWriter creates a new mock instance.

func (*MockWriter) Delete

func (m *MockWriter) Delete(key Key) error

Delete mocks base method.

func (*MockWriter) EXPECT

func (m *MockWriter) EXPECT() *MockWriterMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockWriter) Empty added in v1.7.1

func (m *MockWriter) Empty() (bool, error)

Empty mocks base method.

func (*MockWriter) Get

func (m *MockWriter) Get(key Key) ([]byte, error)

Get mocks base method.

func (*MockWriter) Iterate

func (m *MockWriter) Iterate(callback CallerFn, keyType Key) error

Iterate mocks base method.

func (*MockWriter) Put

func (m *MockWriter) Put(key Key, value []byte) error

Put mocks base method.

func (*MockWriter) Range

func (m *MockWriter) Range(from, to Key, callback CallerFn, stopAtNil bool) error

Range mocks base method.

func (*MockWriter) Stats

func (m *MockWriter) Stats() ShelfStats

Stats mocks base method.

type MockWriterMockRecorder

type MockWriterMockRecorder struct {
	// contains filtered or unexported fields
}

MockWriterMockRecorder is the mock recorder for MockWriter.

func (*MockWriterMockRecorder) Delete

func (mr *MockWriterMockRecorder) Delete(key any) *gomock.Call

Delete indicates an expected call of Delete.

func (*MockWriterMockRecorder) Empty added in v1.7.1

func (mr *MockWriterMockRecorder) Empty() *gomock.Call

Empty indicates an expected call of Empty.

func (*MockWriterMockRecorder) Get

func (mr *MockWriterMockRecorder) Get(key any) *gomock.Call

Get indicates an expected call of Get.

func (*MockWriterMockRecorder) Iterate

func (mr *MockWriterMockRecorder) Iterate(callback, keyType any) *gomock.Call

Iterate indicates an expected call of Iterate.

func (*MockWriterMockRecorder) Put

func (mr *MockWriterMockRecorder) Put(key, value any) *gomock.Call

Put indicates an expected call of Put.

func (*MockWriterMockRecorder) Range

func (mr *MockWriterMockRecorder) Range(from, to, callback, stopAtNil any) *gomock.Call

Range indicates an expected call of Range.

func (*MockWriterMockRecorder) Stats

func (mr *MockWriterMockRecorder) Stats() *gomock.Call

Stats indicates an expected call of Stats.

type NilReader

type NilReader struct{}

NilReader is a shelfReader that always returns nil. It can be used when shelves do not exist.

func (NilReader) Empty added in v1.7.0

func (n NilReader) Empty() (bool, error)

func (NilReader) Get

func (n NilReader) Get(_ Key) ([]byte, error)

func (NilReader) Iterate

func (n NilReader) Iterate(_ CallerFn, _ Key) error

func (NilReader) Range

func (n NilReader) Range(_ Key, to Key, _ CallerFn, _ bool) error

func (NilReader) Stats

func (n NilReader) Stats() ShelfStats

type OnRollbackOption

type OnRollbackOption struct {
	// contains filtered or unexported fields
}

OnRollbackOption see OnRollback

func (OnRollbackOption) Invoke

func (o OnRollbackOption) Invoke(opts []TxOption)

Invoke calls all functions registered with the OnRollbackOption.

type Option

type Option func(config *Config)

func WithLockAcquireTimeout

func WithLockAcquireTimeout(value time.Duration) Option

WithLockAcquireTimeout overrides the default timeout for acquiring a lock.

func WithLogger

func WithLogger(log *logrus.Logger) Option

WithLogger overrides the default logger.

func WithNoSync

func WithNoSync() Option

WithNoSync specifies that the database should not flush its data to disk. Support depends on the underlying database.

type ReadTx

type ReadTx interface {
	// GetShelfReader returns the specified shelf for reading. If it doesn't exist, a NilReader is returned that will return nil for all read operations.
	GetShelfReader(shelfName string) Reader
	// Store returns the KVStore on which the transaction is started
	Store() KVStore
	// Unwrap returns the underlying, database specific transaction object. If not supported, it returns nil.
	Unwrap() interface{}
}

ReadTx is used to read from a KVStore.

type Reader

type Reader interface {
	// Empty returns true if the shelf contains no data
	Empty() (bool, error)
	// Get returns the value for the given key.
	// If the key does not exist it returns ErrKeyNotFound.
	// Returns a ErrDatabase if unsuccessful.
	Get(key Key) ([]byte, error)
	// Iterate walks over all key/value pairs for this shelf. Ordering is not guaranteed.
	// The caller will have to supply the correct key type, such that the keys can be parsed.
	Iterate(callback CallerFn, keyType Key) error
	// Range calls the callback for each key/value pair on this shelf from (inclusive) and to (exclusive) given keys.
	// Ordering is guaranteed and determined by the type of Key given.
	// If stopAtNil is true the operation stops when a non-existing key is encountered.
	Range(from Key, to Key, callback CallerFn, stopAtNil bool) error
	// Stats returns statistics about the shelf.
	Stats() ShelfStats
}

Reader is used to read from a shelf.

type ShelfStats

type ShelfStats struct {
	// NumEntries holds the number of entries in the shelf.
	NumEntries uint
	// ShelfSize holds the current shelf size in bytes.
	ShelfSize uint
}

ShelfStats contains statistics about a shelf.

type Store

type Store interface {
	// Close releases all resources associated with the store. It is safe to call multiple (subsequent) times.
	// The context being passed can be used to specify a timeout for the close operation.
	// Returns a ErrDatabase if unsuccessful,
	Close(ctx context.Context) error
}

type TxOption

type TxOption interface{}

TxOption holds options for store transactions.

func AfterCommit

func AfterCommit(fn func()) TxOption

AfterCommit specifies a function that will be called after a transaction is successfully committed. There can be multiple AfterCommit functions, which will be called in order.

func OnRollback

func OnRollback(fn func()) TxOption

OnRollback specifies a function that will be called after a transaction is successfully rolled back. There can be multiple OnRollback functions, which will be called in order.

func WithWriteLock

func WithWriteLock() TxOption

WithWriteLock is a transaction option that acquires a write lock for the entire store, making sure there are no concurrent writeable transactions. The lock is released when the transaction finishes in any way (commit/rollback).

type Uint32Key

type Uint32Key uint32

Uint32Key is a type helper for a uint32 as Key

func (Uint32Key) Bytes

func (u Uint32Key) Bytes() []byte

func (Uint32Key) Equals

func (u Uint32Key) Equals(other Key) bool

func (Uint32Key) FromBytes

func (u Uint32Key) FromBytes(i []byte) (Key, error)

func (Uint32Key) FromString

func (u Uint32Key) FromString(i string) (Key, error)

func (Uint32Key) Next

func (u Uint32Key) Next() Key

func (Uint32Key) String

func (u Uint32Key) String() string

type WriteLockOption

type WriteLockOption struct {
}

WriteLockOption see WithWriteLock

func (WriteLockOption) Enabled

func (w WriteLockOption) Enabled(opts []TxOption) bool

Enabled returns whether the WithWriteLock option was specified.

type WriteTx

type WriteTx interface {
	ReadTx
	// GetShelfWriter returns the specified shelf for writing. If it doesn't exist, it will be created.
	// To keep the API easy to use it doesn't return an error when it fails, but any call to the returned Writer will return the error that occurred.
	GetShelfWriter(shelfName string) Writer
}

WriteTx is used to write to a KVStore.

type Writer

type Writer interface {
	Reader

	// Put stores the given key and value in the shelf.
	// Returns a ErrDatabase if unsuccessful.
	Put(key Key, value []byte) error
	// Delete removes the given key from the shelf.
	// Returns a ErrDatabase if unsuccessful.
	Delete(key Key) error
}

Writer is used to write to a shelf.

func NewErrorWriter added in v1.5.0

func NewErrorWriter(err error) Writer

NewErrorWriter returns a Writer that will return the error for every method

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL