storage

package
v1.7.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 23, 2024 License: MIT Imports: 19 Imported by: 136

Documentation

Overview

The storage package provides a key/value based interface for storing Kapacitor metadata. All services wishing to store data should use this interface.

The usage patterns for this storage layer are typical create/replace/delete/get/list operations. Typically objects are serialized and stored as the value. As a result, updates to a single field of an object can incur the cost to retrieve the entire object and store it again. In most cases this is acceptable since modifications are rare and object size is small.

A BoltDB backed implementation is also provided.

Index

Constants

View Source 
const (
	DefaultIDIndex = "id"
)

Variables

View Source 
var (
	ErrObjectExists   = errors.New("object already exists")
	ErrNoObjectExists = errors.New("no object exists")
)
View Source 
var (
	ErrNoKeyExists = errors.New("no key exists")
)

Common errors that can be returned

Functions

func DoListFunc 

func DoListFunc(list []*KeyValue, match func(value []byte) bool, offset, limit int) []string

Return a list of values from a list of KeyValues using an offset/limit bound and a match function.

func DoUpdate added in v1.2.0 

func DoUpdate(o TxOperator, f func(Tx) error) error

DoUpdate provides a complete implementation of Interface.Update for a TxOperator.

func DoView added in v1.2.0 

func DoView(o TxOperator, f func(ReadOnlyTx) error) error

View manages a read only transaction.

func ImpossibleTypeErr added in v1.2.0 

func ImpossibleTypeErr(exp interface{}, got interface{}) error

func VersionEasyJSONDecode added in v1.5.1 

func VersionEasyJSONDecode(data []byte, decF func(version int, dec *jlexer.Lexer) error) error

VersionEasyJSONDecode decodes and object that was encoded using VersionJSONEncode.

func VersionJSONDecode added in v1.2.0 

func VersionJSONDecode(data []byte, decF func(version int, dec *json.Decoder) error) error

VersionJSONDecode decodes and object that was encoded using VersionJSONEncode.

func VersionJSONEncode added in v1.2.0 

func VersionJSONEncode(version int, o interface{}) ([]byte, error)

VersionJSONEncode encodes an object as json wrapping it in a VersionWrapper struct.

Types

type APIServer added in v1.3.0 

type APIServer struct {
	Registrar *StoreActionerRegistrar
	DB        *bolt.DB

	HTTPDService interface {
		AddRoutes([]httpd.Route) error
		DelRoutes([]httpd.Route)
	}
	// contains filtered or unexported fields
}

func (*APIServer) Close added in v1.3.0 

func (s *APIServer) Close() error

func (*APIServer) Open added in v1.3.0 

func (s *APIServer) Open() error

type BinaryObject added in v1.2.0 

type BinaryObject interface {
	encoding.BinaryMarshaler
	encoding.BinaryUnmarshaler
	ObjectID() string
}

type Bolt 

type Bolt struct {
	// contains filtered or unexported fields
}

Bolt implementation of Store

func NewBolt 

func NewBolt(db *bolt.DB, bucket ...[]byte) *Bolt

func (*Bolt) BeginReadOnlyTx added in v1.2.0 

func (b *Bolt) BeginReadOnlyTx() (ReadOnlyTx, error)

func (*Bolt) BeginTx added in v1.2.0 

func (b *Bolt) BeginTx() (Tx, error)

func (*Bolt) Bucket added in v1.7.0 

func (b *Bolt) Bucket(bucket []byte) *Bolt

Bucket tells the Bolt to do following actions in a bucket. A nil bucket will return a *Bolt that is targeted to the root bucket.

func (*Bolt) Delete 

func (b *Bolt) Delete(key string) error

func (*Bolt) Exists 

func (b *Bolt) Exists(key string) (exists bool, err error)

func (*Bolt) Get 

func (b *Bolt) Get(key string) (kv *KeyValue, err error)

func (*Bolt) List 

func (b *Bolt) List(prefix string) (kvs []*KeyValue, err error)

func (*Bolt) Put 

func (b *Bolt) Put(key string, value []byte) error

func (*Bolt) Store added in v1.7.0 

func (b *Bolt) Store(buckets ...[]byte) Interface

Store tells the Bolt to do following actions in a bucket. A nil bucket will return a *Bolt that is targeted to the root bucket.

func (*Bolt) Update added in v1.2.0 

func (b *Bolt) Update(f func(tx Tx) error) error

func (*Bolt) View added in v1.2.0 

func (b *Bolt) View(f func(tx ReadOnlyTx) error) error

type Config 

type Config struct {
	// Path to a boltdb database file.
	BoltDBPath string `toml:"boltdb"`
}

func NewConfig 

func NewConfig() Config

func (Config) Validate 

func (c Config) Validate() error

type Diagnostic added in v1.4.0 

type Diagnostic interface {
	Error(msg string, err error)
	Info(msg string, ctx ...keyvalue.T)
}

type Index added in v1.2.0 

type Index struct {
	Name      string
	ValueFunc ValueFunc
	Unique    bool
}

func (Index) ValueOf added in v1.2.0 

func (idx Index) ValueOf(o BinaryObject) (string, error)

type IndexedStore added in v1.2.0 

type IndexedStore struct {
	// contains filtered or unexported fields
}

Indexed provides basic CRUD operations and maintains indexes.

func NewIndexedStore added in v1.2.0 

func NewIndexedStore(store Interface, c IndexedStoreConfig) (*IndexedStore, error)

func (*IndexedStore) Create added in v1.2.0 

func (s *IndexedStore) Create(o BinaryObject) error

func (*IndexedStore) CreateTx added in v1.3.0 

func (s *IndexedStore) CreateTx(tx Tx, o BinaryObject) error

func (*IndexedStore) Delete added in v1.2.0 

func (s *IndexedStore) Delete(id string) error

func (*IndexedStore) DeleteTx added in v1.3.0 

func (s *IndexedStore) DeleteTx(tx Tx, id string) error

func (*IndexedStore) Get added in v1.2.0 

func (s *IndexedStore) Get(id string) (o BinaryObject, err error)

func (*IndexedStore) GetTx added in v1.3.0 

func (s *IndexedStore) GetTx(tx ReadOperator, id string) (BinaryObject, error)

func (*IndexedStore) List added in v1.2.0 

func (s *IndexedStore) List(index, pattern string, offset, limit int) (objects []BinaryObject, err error)

List returns a list of objects that match a given pattern. If limit < 0, then no limit is enforced.

func (*IndexedStore) ListTx added in v1.3.0 

func (s *IndexedStore) ListTx(tx ReadOperator, index, pattern string, offset, limit int) ([]BinaryObject, error)

func (*IndexedStore) Put added in v1.2.0 

func (s *IndexedStore) Put(o BinaryObject) error

func (*IndexedStore) PutTx added in v1.3.0 

func (s *IndexedStore) PutTx(tx Tx, o BinaryObject) error

func (*IndexedStore) Rebuild added in v1.3.0 

func (s *IndexedStore) Rebuild() error

Rebuild completely rebuilds all indexes for the store.

func (*IndexedStore) RebuildTx added in v1.3.0 

func (s *IndexedStore) RebuildTx(tx Tx) error

Rebuild completely rebuilds all indexes for the store using the provided transaction.

func (*IndexedStore) Replace added in v1.2.0 

func (s *IndexedStore) Replace(o BinaryObject) error

func (*IndexedStore) ReplaceTx added in v1.3.0 

func (s *IndexedStore) ReplaceTx(tx Tx, o BinaryObject) error

func (*IndexedStore) ReverseList added in v1.2.0 

func (s *IndexedStore) ReverseList(index, pattern string, offset, limit int) (objects []BinaryObject, err error)

ReverseList returns a list of objects that match a given pattern, using reverse sort. If limit < 0, then no limit is enforced.

func (*IndexedStore) ReverseListTx added in v1.3.0 

func (s *IndexedStore) ReverseListTx(tx ReadOnlyTx, index, pattern string, offset, limit int) ([]BinaryObject, error)

func (*IndexedStore) Store added in v1.7.0 

func (s *IndexedStore) Store() Interface

type IndexedStoreConfig added in v1.2.0 

type IndexedStoreConfig struct {
	Prefix        string
	DataPrefix    string
	IndexesPrefix string
	NewObject     NewObjectF
	Indexes       []Index
}

func DefaultIndexedStoreConfig added in v1.2.0 

func DefaultIndexedStoreConfig(prefix string, newObject NewObjectF) IndexedStoreConfig

func (IndexedStoreConfig) Validate added in v1.2.0 

func (c IndexedStoreConfig) Validate() error

type Interface 

type Interface interface {

	// View creates a new read only transaction and always rolls it back.
	View(func(ReadOnlyTx) error) error

	// Update creates a new read-write transaction and always rolls it back.
	// If the function returns a nil error the transaction is committed, otherwise the error is returned.
	Update(func(Tx) error) error

	Store(Buckets ...[]byte) Interface
}

Common interface for interacting with a simple Key/Value storage

type KeyValue 

type KeyValue struct {
	Key   string
	Value []byte
}

type NewObjectF added in v1.2.0 

type NewObjectF func() BinaryObject

type ReadOnlyTx added in v1.2.0 

type ReadOnlyTx interface {
	ReadOperator

	// Bucket returns a ReadOnlyTx for that bucket. If the bucket doesn't exist Tx should be nil.
	Bucket(name []byte) ReadOnlyTx

	// Rollback signals that the transaction is complete.
	// If the transaction was not committed, then all changes are reverted.
	// Rollback must always be called for every transaction.
	Rollback() error
}

ReadOnlyTx provides an interface for performing read operations in a single transaction.

type ReadOperator added in v1.2.0 

type ReadOperator interface {
	// Get - Retrieve a value.
	Get(key string) (*KeyValue, error)
	// Exists - Check if a key exists>
	Exists(key string) (bool, error)
	// List all values with given prefix.
	List(prefix string) ([]*KeyValue, error)
}

ReadOperator provides an interface for performing read operations.

type Service 

type Service struct {
	HTTPDService interface {
		AddRoutes([]httpd.Route) error
		DelRoutes([]httpd.Route)
	}
	// contains filtered or unexported fields
}

func NewService 

func NewService(conf Config, d Diagnostic) *Service

func (*Service) Close 

func (s *Service) Close() error

func (*Service) CloseBolt added in v1.7.0 

func (s *Service) CloseBolt() error

func (*Service) Diagnostic added in v1.7.0 

func (s *Service) Diagnostic() Diagnostic

func (*Service) Open 

func (s *Service) Open() error

func (*Service) Path added in v1.7.0 

func (s *Service) Path() string

func (*Service) Register added in v1.3.0 

func (s *Service) Register(name string, store StoreActioner)

func (*Service) Store 

func (s *Service) Store(name string) Interface

Return a namespaced store. Calling Store with the same namespace returns the same Store.

func (*Service) Versions added in v1.3.0 

func (s *Service) Versions() Versions

type StoreActioner added in v1.3.0 

type StoreActioner interface {
	// Rebuild the entire store, this should be considered to be an expensive action.
	Rebuild() error
}

StoreActioner exposes and interface for various actions that can be performed on a store.

type StoreActionerRegistrar added in v1.3.0 

type StoreActionerRegistrar struct {
	// contains filtered or unexported fields
}

func NewStorageRegistrar added in v1.7.0 

func NewStorageRegistrar() *StoreActionerRegistrar

func (*StoreActionerRegistrar) Get added in v1.3.0 

func (sr *StoreActionerRegistrar) Get(name string) (store StoreActioner, ok bool)

func (*StoreActionerRegistrar) List added in v1.3.0 

func (sr *StoreActionerRegistrar) List() []string

func (*StoreActionerRegistrar) Register added in v1.3.0 

func (sr *StoreActionerRegistrar) Register(name string, store StoreActioner)

type Tx added in v1.2.0 

type Tx interface {
	ReadOperator
	WriteOperator

	// Cursor - returns a cursor for that bucket
	Cursor() *bbolt.Cursor

	// Bucket returns a Tx for that bucket.  If the bucket doesn't exist Tx should be nil.
	Bucket(name []byte) Tx

	// Commit finalizes the transaction.
	// Once a transaction is committed, rolling back the transaction has no effect.
	Commit() error

	// Rollback signals that the transaction is complete.
	// If the transaction was not committed, then all changes are reverted.
	// Rollback must always be called for every transaction.
	Rollback() error
}

Tx provides an interface for performing read and write storage operations in a single transaction.

type TxOperator added in v1.2.0 

type TxOperator interface {
	// BeginReadOnlyTx starts a new read only transaction. The transaction must be rolledback.
	// Leaving a transaction open can block other operations and otherwise
	// significantly degrade the performance of the storage backend.
	// A single go routine should only have one transaction open at a time.
	BeginReadOnlyTx() (ReadOnlyTx, error)
	// BeginTx starts a new transaction for reads and writes. The transaction must be committed or rolledback.
	// Leaving a transaction open can block other operations and otherwise
	// significantly degrade the performance of the storage backend.
	// A single go routine should only have one transaction open at a time.
	BeginTx() (Tx, error)
}

type ValueFunc added in v1.2.0 

type ValueFunc func(BinaryObject) (string, error)

type VersionWrapper added in v1.2.0 

type VersionWrapper struct {
	Version int              `json:"version"`
	Value   *json.RawMessage `json:"value"`
}

VersionWrapper wraps a structure with a version so that changes to the structure can be properly decoded.

func (VersionWrapper) MarshalEasyJSON added in v1.5.1 

func (v VersionWrapper) MarshalEasyJSON(w *jwriter.Writer)

MarshalEasyJSON supports easyjson.Marshaler interface

func (VersionWrapper) MarshalJSON added in v1.5.1 

func (v VersionWrapper) MarshalJSON() ([]byte, error)

MarshalJSON supports json.Marshaler interface

func (*VersionWrapper) UnmarshalEasyJSON added in v1.5.1 

func (v *VersionWrapper) UnmarshalEasyJSON(l *jlexer.Lexer)

UnmarshalEasyJSON supports easyjson.Unmarshaler interface

func (*VersionWrapper) UnmarshalJSON added in v1.5.1 

func (v *VersionWrapper) UnmarshalJSON(data []byte) error

UnmarshalJSON supports json.Unmarshaler interface

type Versions added in v1.3.0 

type Versions interface {
	Get(id string) (string, error)
	Set(id, version string) error
}

func NewVersions added in v1.3.0 

func NewVersions(store Interface) Versions

type WriteOperator added in v1.2.0 

type WriteOperator interface {

	// Put - Store a value.
	Put(key string, value []byte) error
	// Delete a key.
	// Deleting a non-existent key is not an error.
	Delete(key string) error
}

WriteOperator provides an interface for performing write operations.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.