stoabs

package module
v1.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 8, 2022 License: GPL-3.0 Imports: 12 Imported by: 5

README

Golang Storage Abstraction (go-stoabs)

BBolt

Redis

When creating a Redis KVStore it tests the connection using Redis' PING command.

Due to the simple API of the library, the Redis adapter only supports reading/writing byte arrays. The behavior when reading any other Redis type (e.g. a list or set) is undefined.

Transaction Isolation

Redis doesn't have actual transactions, so this library simulates them by using the MULTI/EXEC/DISCARD commands. That way all commands are guaranteed to be executed atomically.

As a consequence, a value that hasn't been committed yet can't be read. In other words, don't try to read a value from a key that was written to in the same transaction. Subsequently, changes from other writers (from the same process or remote) are reflected immediately in the current transaction: if a key is read twice, there's no guarantee the returned value will be equal.

If the application requires exclusive write access to a store it can lock the database, to assert there are no other active writers:

store.Write(func (tx stoabs.WriteTx) error { 
	// do something with tx
}, stoabs.WithWriteLock())

The lock is released when the transaction is committed or rolled back. The lock is subject to the prefix (CreateRedisStore(prefix string, ...)) the store was created with, meaning other stores with the same prefix will have the same lock.

Redis locks are implemented using (Redsync)[https://github.com/go-redsync/redsync].

Unsupported features
  • Clustering

Documentation

Overview

Package stoabs is a generated GoMock package.

Index

Constants

View Source 
const DefaultTransactionTimeout = 30 * time.Second

Variables

View Source 
var ErrCommitFailed = DatabaseError(errors.New("unable to commit transaction"))

ErrCommitFailed is returned when the commit of transaction fails. Is also a ErrDatabase.

View Source 
var ErrStoreIsClosed = DatabaseError(errors.New("database not open"))

ErrStoreIsClosed is returned when an operation is executed on a closed store. Is also a ErrDatabase.

Functions

func DatabaseError added in v1.2.0 

func DatabaseError(err error) error

DatabaseError wraps the given error in ErrDatabase if it isn't already in the error chain.

Types

type AfterCommitOption 

type AfterCommitOption struct {
	// contains filtered or unexported fields
}

AfterCommitOption see AfterCommit

func (AfterCommitOption) Invoke 

func (o AfterCommitOption) Invoke(opts []TxOption)

Invoke calls all functions registered with the AfterCommitOption.

type BytesKey 

type BytesKey []byte

BytesKey is a type helper for a byte slice as Key

func (BytesKey) Bytes 

func (b BytesKey) Bytes() []byte

func (BytesKey) Equals 

func (b BytesKey) Equals(other Key) bool

func (BytesKey) FromBytes 

func (b BytesKey) FromBytes(i []byte) (Key, error)

func (BytesKey) FromString 

func (b BytesKey) FromString(i string) (Key, error)

func (BytesKey) Next 

func (b BytesKey) Next() Key

func (BytesKey) String 

func (b BytesKey) String() string

type CallerFn 

type CallerFn func(key Key, value []byte) error

CallerFn is the function type which is called for each key value pair when using Iterate() or Range()

type Config 

type Config struct {
	Log                *logrus.Logger
	NoSync             bool
	LockAcquireTimeout time.Duration
}

Config specifies the configuration for a KVStore.

func DefaultConfig 

func DefaultConfig() Config

DefaultConfig returns the default configuration.

type ErrDatabase added in v1.2.0 

type ErrDatabase struct {
	// contains filtered or unexported fields
}

ErrDatabase signals that the wrapped error is related to database access, or due to context cancellation/timeout. The action that resulted in this error may succeed when retried.

func (ErrDatabase) Error added in v1.2.0 

func (e ErrDatabase) Error() string

func (ErrDatabase) Is added in v1.2.0 

func (e ErrDatabase) Is(other error) bool

func (ErrDatabase) Unwrap added in v1.2.0 

func (e ErrDatabase) Unwrap() error

type HashKey 

type HashKey [32]byte

HashKey is a type helper for a 256 bits hash as Key

func (HashKey) Bytes 

func (s HashKey) Bytes() []byte

func (HashKey) Equals 

func (s HashKey) Equals(other Key) bool

func (HashKey) FromBytes 

func (s HashKey) FromBytes(i []byte) (Key, error)

func (HashKey) FromString 

func (s HashKey) FromString(i string) (Key, error)

func (HashKey) Next 

func (s HashKey) Next() Key

func (HashKey) String 

func (s HashKey) String() string

type KVStore 

type KVStore interface {
	Store
	// Write starts a writable transaction and passes it to the given function.
	// Callers should not try to read values which are written in the same transactions, and thus haven't been committed yet.
	// The result when doing so depends on transaction isolation of the underlying database.
	// The passed context can be used to cancel long-running operations or the final commit of the transaction.
	Write(ctx context.Context, fn func(WriteTx) error, opts ...TxOption) error
	// Read starts a read-only transaction and passes it to the given function.
	// The passed context can be used to cancel long-running read operations.
	Read(ctx context.Context, fn func(ReadTx) error) error
	// WriteShelf starts a writable transaction, open a writer for the specified shelf and passes it to the given function.
	// If the shelf does not exist, it will be created.
	// The same semantics of Write apply.
	// The passed context can be used to cancel long-running operations or the final commit of the transaction.
	WriteShelf(ctx context.Context, shelfName string, fn func(Writer) error) error
	// ReadShelf starts a read-only transaction, open a reader for the specified shelf and passes it to the given function.
	// If the shelf does not exist, the function is not called.
	// The passed context can be used to cancel long-running read operations.
	ReadShelf(ctx context.Context, shelfName string, fn func(Reader) error) error
}

KVStore defines the interface for a key-value store. Writing to it is done in callbacks passed to the Write-functions. If the callback returns an error, the transaction is rolled back. Methods return a ErrDatabase when the context has been cancelled or timed-out.

type Key 

type Key interface {
	fmt.Stringer
	// FromString creates a new instance of this Key, parses it from the given string and returns it. It does not modify this key.
	FromString(string) (Key, error)
	// Bytes returns a byte representation of this key
	Bytes() []byte
	// FromBytes creates a new instance of this Key, parses it from the given byte slice and returns it. It does not modify this key.
	FromBytes([]byte) (Key, error)
	// Next returns the next logical key. The next for the number 1 would be 2. The next for the string "a" would be "b"
	Next() Key
	// Equals returns true when the given key is of same type and value.
	Equals(other Key) bool
}

Key is an abstraction for a key in a key/value pair. The underlying implementation determines if the string or byte representation is used.

func NewHashKey 

func NewHashKey(bytes [32]byte) Key

NewHashKey creates a new HashKey from bytes

type MockKVStore 

type MockKVStore struct {
	// contains filtered or unexported fields
}

MockKVStore is a mock of KVStore interface.

func NewMockKVStore 

func NewMockKVStore(ctrl *gomock.Controller) *MockKVStore

NewMockKVStore creates a new mock instance.

func (*MockKVStore) Close 

func (m *MockKVStore) Close(ctx context.Context) error

Close mocks base method.

func (*MockKVStore) EXPECT 

func (m *MockKVStore) EXPECT() *MockKVStoreMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockKVStore) Read 

func (m *MockKVStore) Read(ctx context.Context, fn func(ReadTx) error) error

Read mocks base method.

func (*MockKVStore) ReadShelf 

func (m *MockKVStore) ReadShelf(ctx context.Context, shelfName string, fn func(Reader) error) error

ReadShelf mocks base method.

func (*MockKVStore) Write 

func (m *MockKVStore) Write(ctx context.Context, fn func(WriteTx) error, opts ...TxOption) error

Write mocks base method.

func (*MockKVStore) WriteShelf 

func (m *MockKVStore) WriteShelf(ctx context.Context, shelfName string, fn func(Writer) error) error

WriteShelf mocks base method.

type MockKVStoreMockRecorder 

type MockKVStoreMockRecorder struct {
	// contains filtered or unexported fields
}

MockKVStoreMockRecorder is the mock recorder for MockKVStore.

func (*MockKVStoreMockRecorder) Close 

func (mr *MockKVStoreMockRecorder) Close(ctx interface{}) *gomock.Call

Close indicates an expected call of Close.

func (*MockKVStoreMockRecorder) Read 

func (mr *MockKVStoreMockRecorder) Read(ctx, fn interface{}) *gomock.Call

Read indicates an expected call of Read.

func (*MockKVStoreMockRecorder) ReadShelf 

func (mr *MockKVStoreMockRecorder) ReadShelf(ctx, shelfName, fn interface{}) *gomock.Call

ReadShelf indicates an expected call of ReadShelf.

func (*MockKVStoreMockRecorder) Write 

func (mr *MockKVStoreMockRecorder) Write(ctx, fn interface{}, opts ...interface{}) *gomock.Call

Write indicates an expected call of Write.

func (*MockKVStoreMockRecorder) WriteShelf 

func (mr *MockKVStoreMockRecorder) WriteShelf(ctx, shelfName, fn interface{}) *gomock.Call

WriteShelf indicates an expected call of WriteShelf.

type MockReadTx 

type MockReadTx struct {
	// contains filtered or unexported fields
}

MockReadTx is a mock of ReadTx interface.

func NewMockReadTx 

func NewMockReadTx(ctrl *gomock.Controller) *MockReadTx

NewMockReadTx creates a new mock instance.

func (*MockReadTx) EXPECT 

func (m *MockReadTx) EXPECT() *MockReadTxMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockReadTx) GetShelfReader 

func (m *MockReadTx) GetShelfReader(shelfName string) Reader

GetShelfReader mocks base method.

func (*MockReadTx) Store 

func (m *MockReadTx) Store() KVStore

Store mocks base method.

func (*MockReadTx) Unwrap 

func (m *MockReadTx) Unwrap() interface{}

Unwrap mocks base method.

type MockReadTxMockRecorder 

type MockReadTxMockRecorder struct {
	// contains filtered or unexported fields
}

MockReadTxMockRecorder is the mock recorder for MockReadTx.

func (*MockReadTxMockRecorder) GetShelfReader 

func (mr *MockReadTxMockRecorder) GetShelfReader(shelfName interface{}) *gomock.Call

GetShelfReader indicates an expected call of GetShelfReader.

func (*MockReadTxMockRecorder) Store 

func (mr *MockReadTxMockRecorder) Store() *gomock.Call

Store indicates an expected call of Store.

func (*MockReadTxMockRecorder) Unwrap 

func (mr *MockReadTxMockRecorder) Unwrap() *gomock.Call

Unwrap indicates an expected call of Unwrap.

type MockReader 

type MockReader struct {
	// contains filtered or unexported fields
}

MockReader is a mock of Reader interface.

func NewMockReader 

func NewMockReader(ctrl *gomock.Controller) *MockReader

NewMockReader creates a new mock instance.

func (*MockReader) EXPECT 

func (m *MockReader) EXPECT() *MockReaderMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockReader) Get 

func (m *MockReader) Get(key Key) ([]byte, error)

Get mocks base method.

func (*MockReader) Iterate 

func (m *MockReader) Iterate(callback CallerFn, keyType Key) error

Iterate mocks base method.

func (*MockReader) Range 

func (m *MockReader) Range(from, to Key, callback CallerFn, stopAtNil bool) error

Range mocks base method.

func (*MockReader) Stats 

func (m *MockReader) Stats() ShelfStats

Stats mocks base method.

type MockReaderMockRecorder 

type MockReaderMockRecorder struct {
	// contains filtered or unexported fields
}

MockReaderMockRecorder is the mock recorder for MockReader.

func (*MockReaderMockRecorder) Get 

func (mr *MockReaderMockRecorder) Get(key interface{}) *gomock.Call

Get indicates an expected call of Get.

func (*MockReaderMockRecorder) Iterate 

func (mr *MockReaderMockRecorder) Iterate(callback, keyType interface{}) *gomock.Call

Iterate indicates an expected call of Iterate.

func (*MockReaderMockRecorder) Range 

func (mr *MockReaderMockRecorder) Range(from, to, callback, stopAtNil interface{}) *gomock.Call

Range indicates an expected call of Range.

func (*MockReaderMockRecorder) Stats 

func (mr *MockReaderMockRecorder) Stats() *gomock.Call

Stats indicates an expected call of Stats.

type MockStore 

type MockStore struct {
	// contains filtered or unexported fields
}

MockStore is a mock of Store interface.

func NewMockStore 

func NewMockStore(ctrl *gomock.Controller) *MockStore

NewMockStore creates a new mock instance.

func (*MockStore) Close 

func (m *MockStore) Close(ctx context.Context) error

Close mocks base method.

func (*MockStore) EXPECT 

func (m *MockStore) EXPECT() *MockStoreMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

type MockStoreMockRecorder 

type MockStoreMockRecorder struct {
	// contains filtered or unexported fields
}

MockStoreMockRecorder is the mock recorder for MockStore.

func (*MockStoreMockRecorder) Close 

func (mr *MockStoreMockRecorder) Close(ctx interface{}) *gomock.Call

Close indicates an expected call of Close.

type MockTxOption 

type MockTxOption struct {
	// contains filtered or unexported fields
}

MockTxOption is a mock of TxOption interface.

func NewMockTxOption 

func NewMockTxOption(ctrl *gomock.Controller) *MockTxOption

NewMockTxOption creates a new mock instance.

func (*MockTxOption) EXPECT 

func (m *MockTxOption) EXPECT() *MockTxOptionMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

type MockTxOptionMockRecorder 

type MockTxOptionMockRecorder struct {
	// contains filtered or unexported fields
}

MockTxOptionMockRecorder is the mock recorder for MockTxOption.

type MockWriteTx 

type MockWriteTx struct {
	// contains filtered or unexported fields
}

MockWriteTx is a mock of WriteTx interface.

func NewMockWriteTx 

func NewMockWriteTx(ctrl *gomock.Controller) *MockWriteTx

NewMockWriteTx creates a new mock instance.

func (*MockWriteTx) EXPECT 

func (m *MockWriteTx) EXPECT() *MockWriteTxMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockWriteTx) GetShelfReader 

func (m *MockWriteTx) GetShelfReader(shelfName string) Reader

GetShelfReader mocks base method.

func (*MockWriteTx) GetShelfWriter 

func (m *MockWriteTx) GetShelfWriter(shelfName string) Writer

GetShelfWriter mocks base method.

func (*MockWriteTx) Store 

func (m *MockWriteTx) Store() KVStore

Store mocks base method.

func (*MockWriteTx) Unwrap 

func (m *MockWriteTx) Unwrap() interface{}

Unwrap mocks base method.

type MockWriteTxMockRecorder 

type MockWriteTxMockRecorder struct {
	// contains filtered or unexported fields
}

MockWriteTxMockRecorder is the mock recorder for MockWriteTx.

func (*MockWriteTxMockRecorder) GetShelfReader 

func (mr *MockWriteTxMockRecorder) GetShelfReader(shelfName interface{}) *gomock.Call

GetShelfReader indicates an expected call of GetShelfReader.

func (*MockWriteTxMockRecorder) GetShelfWriter 

func (mr *MockWriteTxMockRecorder) GetShelfWriter(shelfName interface{}) *gomock.Call

GetShelfWriter indicates an expected call of GetShelfWriter.

func (*MockWriteTxMockRecorder) Store 

func (mr *MockWriteTxMockRecorder) Store() *gomock.Call

Store indicates an expected call of Store.

func (*MockWriteTxMockRecorder) Unwrap 

func (mr *MockWriteTxMockRecorder) Unwrap() *gomock.Call

Unwrap indicates an expected call of Unwrap.

type MockWriter 

type MockWriter struct {
	// contains filtered or unexported fields
}

MockWriter is a mock of Writer interface.

func NewMockWriter 

func NewMockWriter(ctrl *gomock.Controller) *MockWriter

NewMockWriter creates a new mock instance.

func (*MockWriter) Delete 

func (m *MockWriter) Delete(key Key) error

Delete mocks base method.

func (*MockWriter) EXPECT 

func (m *MockWriter) EXPECT() *MockWriterMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockWriter) Get 

func (m *MockWriter) Get(key Key) ([]byte, error)

Get mocks base method.

func (*MockWriter) Iterate 

func (m *MockWriter) Iterate(callback CallerFn, keyType Key) error

Iterate mocks base method.

func (*MockWriter) Put 

func (m *MockWriter) Put(key Key, value []byte) error

Put mocks base method.

func (*MockWriter) Range 

func (m *MockWriter) Range(from, to Key, callback CallerFn, stopAtNil bool) error

Range mocks base method.

func (*MockWriter) Stats 

func (m *MockWriter) Stats() ShelfStats

Stats mocks base method.

type MockWriterMockRecorder 

type MockWriterMockRecorder struct {
	// contains filtered or unexported fields
}

MockWriterMockRecorder is the mock recorder for MockWriter.

func (*MockWriterMockRecorder) Delete 

func (mr *MockWriterMockRecorder) Delete(key interface{}) *gomock.Call

Delete indicates an expected call of Delete.

func (*MockWriterMockRecorder) Get 

func (mr *MockWriterMockRecorder) Get(key interface{}) *gomock.Call

Get indicates an expected call of Get.

func (*MockWriterMockRecorder) Iterate 

func (mr *MockWriterMockRecorder) Iterate(callback, keyType interface{}) *gomock.Call

Iterate indicates an expected call of Iterate.

func (*MockWriterMockRecorder) Put 

func (mr *MockWriterMockRecorder) Put(key, value interface{}) *gomock.Call

Put indicates an expected call of Put.

func (*MockWriterMockRecorder) Range 

func (mr *MockWriterMockRecorder) Range(from, to, callback, stopAtNil interface{}) *gomock.Call

Range indicates an expected call of Range.

func (*MockWriterMockRecorder) Stats 

func (mr *MockWriterMockRecorder) Stats() *gomock.Call

Stats indicates an expected call of Stats.

type NilReader 

type NilReader struct{}

NilReader is a shelfReader that always returns nil. It can be used when shelves do not exist.

func (NilReader) Get 

func (n NilReader) Get(_ Key) ([]byte, error)

func (NilReader) Iterate 

func (n NilReader) Iterate(_ CallerFn, _ Key) error

func (NilReader) Range 

func (n NilReader) Range(_ Key, to Key, _ CallerFn, _ bool) error

func (NilReader) Stats 

func (n NilReader) Stats() ShelfStats

type OnRollbackOption 

type OnRollbackOption struct {
	// contains filtered or unexported fields
}

OnRollbackOption see OnRollback

func (OnRollbackOption) Invoke 

func (o OnRollbackOption) Invoke(opts []TxOption)

Invoke calls all functions registered with the OnRollbackOption.

type Option 

type Option func(config *Config)

func WithLockAcquireTimeout 

func WithLockAcquireTimeout(value time.Duration) Option

WithLockAcquireTimeout overrides the default timeout for acquiring a lock.

func WithLogger 

func WithLogger(log *logrus.Logger) Option

WithLogger overrides the default logger.

func WithNoSync 

func WithNoSync() Option

WithNoSync specifies that the database should not flush its data to disk. Support depends on the underlying database.

type ReadTx 

type ReadTx interface {
	// GetShelfReader returns the specified shelf for reading. If it doesn't exist, a NilReader is returned that will return nil for all read operations.
	GetShelfReader(shelfName string) Reader
	// Store returns the KVStore on which the transaction is started
	Store() KVStore
	// Unwrap returns the underlying, database specific transaction object. If not supported, it returns nil.
	Unwrap() interface{}
}

ReadTx is used to read from a KVStore.

type Reader 

type Reader interface {
	// Get returns the value for the given key. If it does not exist it returns nil.
	// Returns a ErrDatabase if unsuccessful.
	Get(key Key) ([]byte, error)
	// Iterate walks over all key/value pairs for this shelf. Ordering is not guaranteed.
	// The caller will have to supply the correct key type, such that the keys can be parsed.
	Iterate(callback CallerFn, keyType Key) error
	// Range calls the callback for each key/value pair on this shelf from (inclusive) and to (exclusive) given keys.
	// Ordering is guaranteed and determined by the type of Key given.
	// If stopAtNil is true the operation stops when a non-existing key is encountered.
	Range(from Key, to Key, callback CallerFn, stopAtNil bool) error
	// Stats returns statistics about the shelf.
	Stats() ShelfStats
}

Reader is used to read from a shelf.

type ShelfStats 

type ShelfStats struct {
	// NumEntries holds the number of entries in the shelf.
	NumEntries uint
	// ShelfSize holds the current shelf size in bytes.
	ShelfSize uint
}

ShelfStats contains statistics about a shelf.

type Store 

type Store interface {
	// Close releases all resources associated with the store. It is safe to call multiple (subsequent) times.
	// The context being passed can be used to specify a timeout for the close operation.
	// Returns a ErrDatabase if unsuccessful,
	Close(ctx context.Context) error
}

type TxOption 

type TxOption interface{}

TxOption holds options for store transactions.

func AfterCommit 

func AfterCommit(fn func()) TxOption

AfterCommit specifies a function that will be called after a transaction is successfully committed. There can be multiple AfterCommit functions, which will be called in order.

func OnRollback 

func OnRollback(fn func()) TxOption

OnRollback specifies a function that will be called after a transaction is successfully rolled back. There can be multiple OnRollback functions, which will be called in order.

func WithWriteLock 

func WithWriteLock() TxOption

WithWriteLock is a transaction option that acquires a write lock for the entire store, making sure there are no concurrent writeable transactions. The lock is released when the transaction finishes in any way (commit/rollback).

type Uint32Key 

type Uint32Key uint32

Uint32Key is a type helper for a uint32 as Key

func (Uint32Key) Bytes 

func (u Uint32Key) Bytes() []byte

func (Uint32Key) Equals 

func (u Uint32Key) Equals(other Key) bool

func (Uint32Key) FromBytes 

func (u Uint32Key) FromBytes(i []byte) (Key, error)

func (Uint32Key) FromString 

func (u Uint32Key) FromString(i string) (Key, error)

func (Uint32Key) Next 

func (u Uint32Key) Next() Key

func (Uint32Key) String 

func (u Uint32Key) String() string

type WriteLockOption 

type WriteLockOption struct {
}

WriteLockOption see WithWriteLock

func (WriteLockOption) Enabled 

func (w WriteLockOption) Enabled(opts []TxOption) bool

Enabled returns whether the WithWriteLock option was specified.

type WriteTx 

type WriteTx interface {
	ReadTx
	// GetShelfWriter returns the specified shelf for writing. If it doesn't exist, it will be created.
	// To keep the API easy to use it doesn't return an error when it fails, but any call to the returned Writer will return the error that occurred.
	GetShelfWriter(shelfName string) Writer
}

WriteTx is used to write to a KVStore.

type Writer 

type Writer interface {
	Reader

	// Put stores the given key and value in the shelf.
	// Returns a ErrDatabase if unsuccessful.
	Put(key Key, value []byte) error
	// Delete removes the given key from the shelf.
	// Returns a ErrDatabase if unsuccessful.
	Delete(key Key) error
}

Writer is used to write to a shelf.

func NewErrorWriter added in v1.5.0 

func NewErrorWriter(err error) Writer

NewErrorWriter returns a Writer that will return the error for every method

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.