keyval

Documentation

Overview

Package keyval provides an abstraction of a key-value data store and defines the keyval data broker API. The Data Broker API consists of the ProtoBroker API, the KeyValProtoWatcher API, and other supporting APIs. It is used to access data in a key-value store.

Index

• Constants
• Variables
• func ToChan(respCh chan BytesWatchResp, opts ...interface{}) func(dto BytesWatchResp)
• func ToChanProto(respCh chan datasync.ProtoWatchResp, opts ...interface{}) func(dto datasync.ProtoWatchResp)
• type BytesBroker
• type BytesBrokerWithAtomic
• type BytesKeyIterator
• type BytesKeyVal
• type BytesKeyValIterator
• type BytesKvPair
• type BytesTxn
• type BytesWatchResp
• type BytesWatcher
• type CoreBrokerWatcher
• type KvBytesPlugin
• type KvProtoPlugin
• type ProtoBroker
• type ProtoKeyIterator
• type ProtoKeyVal
• type ProtoKeyValIterator
• type ProtoKvPair
• type ProtoTxn
• type ProtoWatcher
• type Serializer
• type SerializerJSON
  • func (sj *SerializerJSON) Marshal(message proto.Message) ([]byte, error)
  • func (sj *SerializerJSON) Unmarshal(data []byte, protoData proto.Message) error
• type SerializerProto
  • func (sp *SerializerProto) Marshal(message proto.Message) ([]byte, error)
  • func (sp *SerializerProto) Unmarshal(data []byte, protoData proto.Message) error

Constants

const Root = ""

Root denotes that no prefix is prepended to the keys.

Variables

var DefaultMarshaler = &jsonpb.Marshaler{
	OrigName: true,
}

DefaultMarshaler is the marshaler used for JSON encoding. It uses original names (from .proto by default).

Functions

func ToChan

func ToChan(respCh chan BytesWatchResp, opts ...interface{}) func(dto BytesWatchResp)

ToChan creates a callback that can be passed to the Watch function in order to receive notifications through a channel. If the notification cannot be delivered until timeout, it is dropped.

func ToChanProto

func ToChanProto(respCh chan datasync.ProtoWatchResp, opts ...interface{}) func(dto datasync.ProtoWatchResp)

ToChanProto creates a callback that can be passed to the Watch function in order to receive JSON/protobuf-formatted notifications through a channel. If the notification cannot be delivered until timeout, it is dropped.

Types

type BytesBroker

type BytesBroker interface {
	// Put puts single key-value pair into etcd.
	// The behavior of put can be adjusted using PutOptions.
	Put(key string, data []byte, opts ...datasync.PutOption) error
	// NewTxn creates a transaction.
	NewTxn() BytesTxn
	// GetValue retrieves one item under the provided key.
	GetValue(key string) (data []byte, found bool, revision int64, err error)
	// ListValues returns an iterator that enables to traverse all items stored
	// under the provided <key>.
	ListValues(key string) (BytesKeyValIterator, error)
	// ListKeys returns an iterator that allows to traverse all keys from data
	// store that share the given <prefix>.
	ListKeys(prefix string) (BytesKeyIterator, error)
	// Delete removes data stored under the <key>.
	Delete(key string, opts ...datasync.DelOption) (existed bool, err error)
}

BytesBroker allows storing, retrieving and removing data in a key-value form.

type BytesBrokerWithAtomic

type BytesBrokerWithAtomic interface {
	BytesBroker

	// PutIfNotExists puts given key-value pair into the datastore if there is no value set for the key.
	// If the put was successful, <succeeded> is returned as true. If the key already exists, <succeeded> is returned
	// as false and the value for the key is untouched.
	PutIfNotExists(key string, data []byte) (succeeded bool, err error)

	// CompareAndSwap compares the value currently stored under the given key with the expected <oldData>,
	// and only if the expected and actual data match, the value is then changed to <newData>. The comparison and the
	// value change are executed together in a single transaction and cannot be interleaved with another operation for
	// that key.
	CompareAndSwap(key string, oldData, newData []byte) (swapped bool, err error)

	// CompareAndDelete compares the value currently stored under the given key with the expected <data>,
	// and only if the expected and actual data match, the value is then removed from the datastore. The comparison and
	// the value removal are executed together in a single transaction and cannot be interleaved with another operation
	// for that key.
	CompareAndDelete(key string, data []byte) (deleted bool, err error)
}

BytesBrokerWithAtomic extends BytesBroker with atomic operations. Currently only etcd plugin supports atomic operations.

type BytesKeyIterator

type BytesKeyIterator interface {
	// GetNext retrieves the following key from the context.
	// When there are no more keys to get, <stop> is returned as *true*
	// and <key>, <rev> are default values.
	GetNext() (key string, rev int64, stop bool)
}

BytesKeyIterator is an iterator returned by ListKeys call.

type BytesKeyVal

type BytesKeyVal interface {
	BytesKvPair
	datasync.WithRevision
}

BytesKeyVal represents a single item in data store.

type BytesKeyValIterator

type BytesKeyValIterator interface {
	// GetNext retrieves the following item from the context.
	// When there are no more items to get, <stop> is returned as *true*
	// and <kv> is simply *nil*.
	GetNext() (kv BytesKeyVal, stop bool)
}

BytesKeyValIterator is an iterator returned by ListValues call.

type BytesKvPair

type BytesKvPair interface {
	// GetValue returns the value of the pair.
	GetValue() []byte
	// GetPrevValue returns the previous value of the pair.
	GetPrevValue() []byte

	datasync.WithKey
}

BytesKvPair groups getters for a key-value pair.

type BytesTxn

type BytesTxn interface {
	// Put adds put operation (write raw <data> under the given <key>) into
	// the transaction.
	Put(key string, data []byte) BytesTxn
	// Delete adds delete operation (removal of <data> under the given <key>)
	// into the transaction.
	Delete(key string) BytesTxn
	// Commit tries to execute all the operations of the transaction.
	// In the end, either all of them have been successfully applied or none
	// of them and an error is returned.
	Commit(ctx context.Context) error
}

BytesTxn allows to group operations into the transaction. Transaction executes multiple operations in a more efficient way in contrast to executing them one by one.

type BytesWatchResp

type BytesWatchResp interface {
	BytesKvPair
	datasync.WithChangeType
	datasync.WithRevision
}

BytesWatchResp represents a notification about data change. It is sent through the respChan callback.

type BytesWatcher

type BytesWatcher interface {
	// Watch starts subscription for changes associated with the selected keys.
	// Watch events will be delivered to callback (not channel) <respChan>.
	// Channel <closeChan> can be used to close watching on respective key
	Watch(respChan func(BytesWatchResp), closeChan chan string, keys ...string) error
}

BytesWatcher defines API for monitoring changes in datastore.

type CoreBrokerWatcher

type CoreBrokerWatcher interface {
	BytesBroker
	BytesWatcher
	NewBroker(prefix string) BytesBroker
	NewWatcher(prefix string) BytesWatcher
	Close() error
}

CoreBrokerWatcher defines methods for full datastore access.

type KvBytesPlugin

type KvBytesPlugin interface {
	// NewBroker returns a BytesBroker instance that prepends given
	// <keyPrefix> to all keys in its calls.
	// To avoid using a prefix, pass keyval.Root constant as argument.
	NewBroker(keyPrefix string) BytesBroker
	// NewWatcher returns a BytesWatcher instance. Given <keyPrefix> is
	// prepended to keys during watch subscribe phase.
	// The prefix is removed from the key retrieved by GetKey() in BytesWatchResp.
	// To avoid using a prefix, pass keyval.Root constant as argument.
	NewWatcher(keyPrefix string) BytesWatcher
}

KvBytesPlugin provides unifying interface for different key-value datastore implementations.

type KvProtoPlugin

type KvProtoPlugin interface {
	// NewPrefixedBroker returns a ProtoBroker instance that prepends given
	// <keyPrefix> to all keys in its calls.
	// To avoid using a prefix, pass keyval.Root constant as the argument.
	NewBroker(keyPrefix string) ProtoBroker
	// NewPrefixedWatcher returns a ProtoWatcher instance. Given key prefix
	// is prepended to keys during watch subscribe phase.
	// The prefix is removed from the key retrieved by GetKey() in ProtoWatchResp.
	// To avoid using a prefix, pass keyval.Root constant as argument.
	NewWatcher(keyPrefix string) ProtoWatcher
	// Disabled returns true if there was no configuration and therefore agent
	// started without connectivity to a particular data store.
	Disabled() bool
	// OnConnect executes datasync callback if KV plugin is connected. If not, it gathers
	// these functions from all plugins using the specific KV plugin as dependency and
	// if delayed start is allowed, callbacks are executed after successful connection.
	OnConnect(func() error)
	// Returns key value store name.
	String() string
}

KvProtoPlugin provides unifying interface for different key-value datastore implementations.

type ProtoBroker

type ProtoBroker interface {
	// Put puts single key-value pair into key value store.
	datasync.KeyProtoValWriter
	// NewTxn creates a transaction.
	NewTxn() ProtoTxn
	// GetValue retrieves one item under the provided <key>. If the item exists,
	// it is unmarshaled into the <reqObj>.
	GetValue(key string, reqObj proto.Message) (found bool, revision int64, err error)
	// ListValues returns an iterator that enables to traverse all items stored
	// under the provided <key>.
	ListValues(key string) (ProtoKeyValIterator, error)
	// ListKeys returns an iterator that allows to traverse all keys from data
	// store that share the given <prefix>.
	ListKeys(prefix string) (ProtoKeyIterator, error)
	// Delete removes data stored under the <key>.
	Delete(key string, opts ...datasync.DelOption) (existed bool, err error)
}

ProtoBroker is a decorator that allows to read/write proto file modelled data. It marshals/unmarshals go structures to slice of bytes and vice versa behind the scenes.

type ProtoKeyIterator

type ProtoKeyIterator interface {
	// GetNext retrieves the following item from the context.
	GetNext() (key string, rev int64, stop bool)
	// Closer is needed for closing the iterator (please check error returned by Close method)
	io.Closer
}

ProtoKeyIterator is an iterator returned by ListKeys call.

type ProtoKeyVal

type ProtoKeyVal interface {
	ProtoKvPair
	datasync.WithRevision
}

ProtoKeyVal represents a single key-value pair.

type ProtoKeyValIterator

type ProtoKeyValIterator interface {
	// GetNext retrieves the following value from the context. GetValue is unmarshaled into the provided argument.
	GetNext() (kv ProtoKeyVal, stop bool)
	// Closer is needed for closing the iterator (please check error returned by Close method).
	io.Closer
}

ProtoKeyValIterator is an iterator returned by ListValues call.

type ProtoKvPair

type ProtoKvPair interface {
	datasync.LazyValue
	datasync.WithPrevValue
	datasync.WithKey
}

ProtoKvPair groups getter for single key-value pair.

type ProtoTxn

type ProtoTxn interface {
	// Put adds put operation (write formatted <data> under the given <key>)
	// into the transaction.
	Put(key string, data proto.Message) ProtoTxn
	// Delete adds delete operation (removal of <data> under the given <key>)
	// into the transaction.
	Delete(key string) ProtoTxn
	// Commit tries to execute all the operations of the transaction.
	// In the end, either all of them have been successfully applied or none
	// of them and an error is returned.
	Commit(ctx context.Context) error
}

ProtoTxn allows to group operations into the transaction. It is like BytesTxn, except that data are protobuf/JSON formatted. Transaction executes multiple operations in a more efficient way in contrast to executing them one by one.

type ProtoWatcher

type ProtoWatcher interface {
	// Watch starts monitoring changes associated with the keys.
	// Watch events will be delivered to callback (not channel) <respChan>.
	// Channel <closeChan> can be used to close watching on respective key
	Watch(respChan func(datasync.ProtoWatchResp), closeChan chan string, key ...string) error
}

ProtoWatcher defines API for monitoring changes in datastore. Changes are returned as protobuf/JSON-formatted data.

type Serializer

type Serializer interface {
	Unmarshal(data []byte, protoData proto.Message) error
	Marshal(message proto.Message) ([]byte, error)
}

Serializer is used to make conversions between raw and formatted data. Currently supported formats are JSON and protobuf.

type SerializerJSON

type SerializerJSON struct {
	ExpandEnvVars bool
}

SerializerJSON serializes proto message using JSON serializer.

func (*SerializerJSON) Marshal

func (sj *SerializerJSON) Marshal(message proto.Message) ([]byte, error)

Marshal serializes proto message to the slice of bytes using jsonpb marshaller to correctly marshal protobuf data.

func (*SerializerJSON) Unmarshal

func (sj *SerializerJSON) Unmarshal(data []byte, protoData proto.Message) error

Unmarshal deserializes data from slice of bytes into the provided protobuf message using jsonpb marshaller to correctly unmarshal protobuf data.

type SerializerProto

type SerializerProto struct{}

SerializerProto serializes proto message using proto serializer.

func (*SerializerProto) Marshal

func (sp *SerializerProto) Marshal(message proto.Message) ([]byte, error)

Marshal serializes data from proto message to the slice of bytes using proto marshaller.

func (*SerializerProto) Unmarshal

func (sp *SerializerProto) Unmarshal(data []byte, protoData proto.Message) error

Unmarshal deserializes data from slice of bytes into the provided protobuf message using proto marshaller.