storage

README

Storage

Status: under development; This is currently just the interface

A storage extension persists state beyond the collector process. Other components can request a storage client from the storage extension and use it to manage state.

The storage.Extension interface extends component.Extension by adding the following method:

GetClient(context.Context, component.Kind, component.ID, string) (Client, error)

The storage.Client interface contains the following methods:

Get(context.Context, string) ([]byte, error)
Set(context.Context, string, []byte) error
Delete(context.Context, string) error
Close(context.Context) error

It is possible to execute several operations in a single transaction via Batch. The method takes a collection of Operation arguments (each of which contains Key, Value and Type properties):

Batch(context.Context, ...Operation) error

The elements itself can be created using:

SetOperation(string, []byte) Operation
GetOperation(string) Operation
DeleteOperation(string) Operation

Get operation results are stored in-place into the given Operation and can be retrieved using its Value property.

Note: All methods should return error only if a problem occurred. (For example, if a file is no longer accessible, or if a remote service is unavailable.)

Note: It is the responsibility of each component to Close a storage client that it has requested.

Documentation

Overview

Package storage implements an extension that can persist state beyond the collector process.

Index

• type Client
  • func NewNopClient() Client
• type Extension
• type OpType
• type Operation
  • func DeleteOperation(key string) *Operation
  • func GetOperation(key string) *Operation
  • func SetOperation(key string, value []byte) *Operation

Types

type Client

type Client interface {
	// Get will retrieve data from storage that corresponds to the
	// specified key. It should return (nil, nil) if not found
	Get(ctx context.Context, key string) ([]byte, error)

	// Set will store data. The data can be retrieved by the same
	// component after a process restart, using the same key
	Set(ctx context.Context, key string, value []byte) error

	// Delete will delete data associated with the specified key
	Delete(ctx context.Context, key string) error

	// Batch handles specified operations in batch. Get operation results are put in-place
	Batch(ctx context.Context, ops ...*Operation) error

	// Close will release any resources held by the client
	Close(ctx context.Context) error
}

Client is the interface that storage clients must implement All methods should return error only if a problem occurred. This mirrors the behavior of a golang map:

• Set doesn't error if a key already exists - it just overwrites the value.
• Get doesn't error if a key is not found - it just returns nil.
• Delete doesn't error if the key doesn't exist - it just no-ops.

Similarly:

• Batch doesn't error if any of the above happens for either retrieved or updated keys

This also provides a way to differentiate data operations

[overwrite | not-found | no-op] from "real" problems

func NewNopClient

func NewNopClient() Client

NewNopClient returns a nop client

type Extension

type Extension interface {
	extension.Extension

	// GetClient will create a client for use by the specified component.
	// Each component can have multiple storages (e.g. one for each signal),
	// which can be identified using storageName parameter.
	// The component can use the client to manage state
	GetClient(ctx context.Context, kind component.Kind, id component.ID, storageName string) (Client, error)
}

Extension is the interface that storage extensions must implement

type OpType

type OpType int

const (
	Get OpType = iota
	Set
	Delete
)

type Operation

type Operation struct {
	// Key specifies key which is going to be get/set/deleted
	Key string
	// Value specifies value that is going to be set or holds result of get operation
	Value []byte
	// Type describes the operation type
	Type OpType
}

func DeleteOperation

func DeleteOperation(key string) *Operation

func GetOperation

func GetOperation(key string) *Operation

func SetOperation

func SetOperation(key string, value []byte) *Operation