cache

package
v3.65.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 10, 2022 License: MIT Imports: 38 Imported by: 0

Documentation

Overview

Package cache implements the types.Cache interface for storing and retrieving key/value pairs from a range of storage strategies.

Index

Constants

View Source
const (
	TypeAWSDynamoDB = "aws_dynamodb"
	TypeAWSS3       = "aws_s3"
	TypeDynamoDB    = "dynamodb"
	TypeFile        = "file"
	TypeMemcached   = "memcached"
	TypeMemory      = "memory"
	TypeMongoDB     = "mongodb"
	TypeMultilevel  = "multilevel"
	TypeRedis       = "redis"
	TypeRistretto   = "ristretto"
	TypeS3          = "s3"
)

String constants representing each cache type. Deprecated: Do not add new components here. Instead, use the public plugin APIs. Examples can be found in: ./internal/impl

Variables

View Source
var Constructors = map[string]TypeSpec{}

Constructors is a map of all cache types with their specs.

Functions

func Descriptions

func Descriptions() string

Descriptions returns a formatted string of descriptions for each type.

func DocumentPlugin

func DocumentPlugin(
	typeString, description string,
	configSanitiser PluginConfigSanitiser,
)

DocumentPlugin adds a description and an optional configuration sanitiser function to the definition of a registered plugin. This improves the documentation generated by PluginDescriptions.

func New

func New(
	conf Config,
	mgr types.Manager,
	log log.Modular,
	stats metrics.Type,
) (types.Cache, error)

New creates a cache type based on an cache configuration.

func NewAWSDynamoDB

func NewAWSDynamoDB(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewAWSDynamoDB creates a new DynamoDB cache type.

func NewAWSS3

func NewAWSS3(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewAWSS3 creates a new S3 cache type.

func NewDynamoDB

func NewDynamoDB(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewDynamoDB creates a new DynamoDB cache type.

func NewFile

func NewFile(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewFile creates a new File cache type.

func NewMemcached

func NewMemcached(
	conf Config, mgr types.Manager, log log.Modular, stats metrics.Type,
) (types.Cache, error)

NewMemcached returns a Memcached processor.

func NewMemory

func NewMemory(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewMemory creates a new Memory cache type.

func NewMultilevel

func NewMultilevel(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewMultilevel creates a new Multilevel cache type.

func NewRedis

func NewRedis(
	conf Config, mgr types.Manager, log log.Modular, stats metrics.Type,
) (types.Cache, error)

NewRedis returns a Redis processor.

func NewRistretto

func NewRistretto(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewRistretto creates a new Ristretto cache type.

func NewS3

func NewS3(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewS3 creates a new S3 cache type.

func PluginCount

func PluginCount() int

PluginCount returns the number of registered plugins. This does NOT count the standard set of components.

func PluginDescriptions

func PluginDescriptions() string

PluginDescriptions generates and returns a markdown formatted document listing each registered plugin and an example configuration for it.

func RegisterPlugin

func RegisterPlugin(
	typeString string,
	configConstructor PluginConfigConstructor,
	constructor PluginConstructor,
)

RegisterPlugin registers a plugin by a unique name so that it can be constructed similar to regular caches. If configuration is not needed for this plugin then configConstructor can be nil. A constructor for the plugin itself must be provided.

func SanitiseConfig

func SanitiseConfig(conf Config) (interface{}, error)

SanitiseConfig creates a sanitised version of a config.

func WalkConstructors

func WalkConstructors(fn func(ConstructorFunc, docs.ComponentSpec))

WalkConstructors iterates each component constructor.

Types

type Config

type Config struct {
	Label       string           `json:"label" yaml:"label"`
	Type        string           `json:"type" yaml:"type"`
	AWSDynamoDB DynamoDBConfig   `json:"aws_dynamodb" yaml:"aws_dynamodb"`
	AWSS3       S3Config         `json:"aws_s3" yaml:"aws_s3"`
	DynamoDB    DynamoDBConfig   `json:"dynamodb" yaml:"dynamodb"`
	File        FileConfig       `json:"file" yaml:"file"`
	Memcached   MemcachedConfig  `json:"memcached" yaml:"memcached"`
	Memory      MemoryConfig     `json:"memory" yaml:"memory"`
	MongoDB     MongoDBConfig    `json:"mongodb" yaml:"mongodb"`
	Multilevel  MultilevelConfig `json:"multilevel" yaml:"multilevel"`
	Plugin      interface{}      `json:"plugin,omitempty" yaml:"plugin,omitempty"`
	Redis       RedisConfig      `json:"redis" yaml:"redis"`
	Ristretto   RistrettoConfig  `json:"ristretto" yaml:"ristretto"`
	S3          S3Config         `json:"s3" yaml:"s3"`
}

Config is the all encompassing configuration struct for all cache types. Deprecated: Do not add new components here. Instead, use the public plugin APIs. Examples can be found in: ./internal/impl

func NewConfig

func NewConfig() Config

NewConfig returns a configuration struct fully populated with default values.

func (Config) Sanitised

func (conf Config) Sanitised(removeDeprecated bool) (interface{}, error)

Sanitised returns a sanitised version of the config, meaning sections that aren't relevant to behaviour are removed. Also optionally removes deprecated fields.

func (*Config) UnmarshalYAML

func (conf *Config) UnmarshalYAML(value *yaml.Node) error

UnmarshalYAML ensures that when parsing configs that are in a map or slice the default values are still applied.

type ConstructorFunc

type ConstructorFunc func(Config, types.Manager, log.Modular, metrics.Type) (types.Cache, error)

ConstructorFunc is a func signature able to construct a cache.

func GetDeprecatedPlugin

func GetDeprecatedPlugin(name string) (ConstructorFunc, bool)

GetDeprecatedPlugin returns a constructor for an old plugin if it exists.

type DynamoDB

type DynamoDB struct {
	// contains filtered or unexported fields
}

DynamoDB is a DynamoDB based cache implementation.

func (*DynamoDB) Add

func (d *DynamoDB) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*DynamoDB) CloseAsync

func (d *DynamoDB) CloseAsync()

CloseAsync shuts down the cache.

func (*DynamoDB) Delete

func (d *DynamoDB) Delete(key string) error

Delete attempts to remove a key.

func (*DynamoDB) Get

func (d *DynamoDB) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

func (*DynamoDB) Set

func (d *DynamoDB) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*DynamoDB) SetMulti

func (d *DynamoDB) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, if any keys fail to be set an error is returned.

func (*DynamoDB) WaitForClose

func (d *DynamoDB) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type DynamoDBConfig

type DynamoDBConfig struct {
	ConsistentRead bool   `json:"consistent_read" yaml:"consistent_read"`
	DataKey        string `json:"data_key" yaml:"data_key"`
	HashKey        string `json:"hash_key" yaml:"hash_key"`
	Table          string `json:"table" yaml:"table"`
	TTL            string `json:"ttl" yaml:"ttl"`
	TTLKey         string `json:"ttl_key" yaml:"ttl_key"`
	retries.Config `json:",inline" yaml:",inline"`
	// contains filtered or unexported fields
}

DynamoDBConfig contains config fields for the DynamoDB cache type.

func NewDynamoDBConfig

func NewDynamoDBConfig() DynamoDBConfig

NewDynamoDBConfig creates a MemoryConfig populated with default values.

type File deprecated

type File struct {
	// contains filtered or unexported fields
}

File is a file system based cache implementation.

TODO: V4 remove this

Deprecated: This implementation is no longer used.

func (*File) Add deprecated

func (f *File) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

Deprecated: This implementation is no longer used.

func (*File) CloseAsync deprecated

func (f *File) CloseAsync()

CloseAsync shuts down the cache.

Deprecated: This implementation is no longer used.

func (*File) Delete deprecated

func (f *File) Delete(key string) error

Delete attempts to remove a key.

Deprecated: This implementation is no longer used.

func (*File) Get deprecated

func (f *File) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

Deprecated: This implementation is no longer used.

func (*File) Set deprecated

func (f *File) Set(key string, value []byte) error

Set attempts to set the value of a key.

Deprecated: This implementation is no longer used.

func (*File) SetMulti deprecated

func (f *File) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

Deprecated: This implementation is no longer used.

func (*File) WaitForClose deprecated

func (f *File) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

Deprecated: This implementation is no longer used.

type FileConfig

type FileConfig struct {
	Directory string `json:"directory" yaml:"directory"`
}

FileConfig contains config fields for the File cache type.

func NewFileConfig

func NewFileConfig() FileConfig

NewFileConfig creates a FileConfig populated with default values.

type Memcached

type Memcached struct {
	// contains filtered or unexported fields
}

Memcached is a cache that connects to memcached servers.

func (*Memcached) Add

func (m *Memcached) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists or if the operation fails.

func (*Memcached) AddWithTTL

func (m *Memcached) AddWithTTL(key string, value []byte, ttl *time.Duration) error

AddWithTTL attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists or if the operation fails.

func (*Memcached) CloseAsync

func (m *Memcached) CloseAsync()

CloseAsync shuts down the cache.

func (*Memcached) Delete

func (m *Memcached) Delete(key string) error

Delete attempts to remove a key.

func (*Memcached) Get

func (m *Memcached) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist or if the operation failed.

func (*Memcached) Set

func (m *Memcached) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*Memcached) SetMulti

func (m *Memcached) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Memcached) SetMultiWithTTL

func (m *Memcached) SetMultiWithTTL(items map[string]types.CacheTTLItem) error

SetMultiWithTTL attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Memcached) SetWithTTL

func (m *Memcached) SetWithTTL(key string, value []byte, ttl *time.Duration) error

SetWithTTL attempts to set the value of a key.

func (*Memcached) WaitForClose

func (m *Memcached) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type MemcachedConfig

type MemcachedConfig struct {
	Addresses   []string `json:"addresses" yaml:"addresses"`
	Prefix      string   `json:"prefix" yaml:"prefix"`
	TTL         int32    `json:"ttl" yaml:"ttl"`
	Retries     int      `json:"retries" yaml:"retries"`
	RetryPeriod string   `json:"retry_period" yaml:"retry_period"`
}

MemcachedConfig is a config struct for a memcached connection.

func NewMemcachedConfig

func NewMemcachedConfig() MemcachedConfig

NewMemcachedConfig returns a MemcachedConfig with default values.

type Memory deprecated

type Memory struct {
	// contains filtered or unexported fields
}

Memory is a memory based cache implementation.

TODO: V4 remove this

Deprecated: This implementation is no longer used.

func (*Memory) Add deprecated

func (m *Memory) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

Deprecated: This implementation is no longer used.

func (*Memory) CloseAsync deprecated

func (m *Memory) CloseAsync()

CloseAsync shuts down the cache.

Deprecated: This implementation is no longer used.

func (*Memory) Delete deprecated

func (m *Memory) Delete(key string) error

Delete attempts to remove a key.

Deprecated: This implementation is no longer used.

func (*Memory) Get deprecated

func (m *Memory) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

Deprecated: This implementation is no longer used.

func (*Memory) Set deprecated

func (m *Memory) Set(key string, value []byte) error

Set attempts to set the value of a key.

Deprecated: This implementation is no longer used.

func (*Memory) SetMulti deprecated

func (m *Memory) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

Deprecated: This implementation is no longer used.

func (*Memory) WaitForClose deprecated

func (m *Memory) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

Deprecated: This implementation is no longer used.

type MemoryConfig

type MemoryConfig struct {
	TTL                int               `json:"ttl" yaml:"ttl"`
	CompactionInterval string            `json:"compaction_interval" yaml:"compaction_interval"`
	InitValues         map[string]string `json:"init_values" yaml:"init_values"`
	Shards             int               `json:"shards" yaml:"shards"`
}

MemoryConfig contains config fields for the Memory cache type.

func NewMemoryConfig

func NewMemoryConfig() MemoryConfig

NewMemoryConfig creates a MemoryConfig populated with default values.

type MongoDBConfig

type MongoDBConfig struct {
	client.Config `json:",inline" yaml:",inline"`
	KeyField      string `json:"key_field" yaml:"key_field"`
	ValueField    string `json:"value_field" yaml:"value_field"`
}

MongoDBConfig is a config struct for a mongo connection.

func NewMongoDBConfig

func NewMongoDBConfig() MongoDBConfig

NewMongoDBConfig returns a MongoDBConfig with default values.

type Multilevel

type Multilevel struct {
	// contains filtered or unexported fields
}

Multilevel is a file system based cache implementation.

func (*Multilevel) Add

func (l *Multilevel) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*Multilevel) AddWithTTL

func (l *Multilevel) AddWithTTL(key string, value []byte, ttl *time.Duration) error

AddWithTTL attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*Multilevel) CloseAsync

func (l *Multilevel) CloseAsync()

CloseAsync shuts down the cache.

func (*Multilevel) Delete

func (l *Multilevel) Delete(key string) error

Delete attempts to remove a key.

func (*Multilevel) Get

func (l *Multilevel) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

func (*Multilevel) Set

func (l *Multilevel) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*Multilevel) SetMulti

func (l *Multilevel) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Multilevel) SetMultiWithTTL

func (l *Multilevel) SetMultiWithTTL(items map[string]types.CacheTTLItem) error

SetMultiWithTTL attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Multilevel) SetWithTTL

func (l *Multilevel) SetWithTTL(key string, value []byte, ttl *time.Duration) error

SetWithTTL attempts to set the value of a key.

func (*Multilevel) WaitForClose

func (l *Multilevel) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type MultilevelConfig

type MultilevelConfig []string

MultilevelConfig contains config fields for the Multilevel cache type.

func NewMultilevelConfig

func NewMultilevelConfig() MultilevelConfig

NewMultilevelConfig creates a MultilevelConfig populated with default values.

type PluginConfigConstructor

type PluginConfigConstructor func() interface{}

PluginConfigConstructor is a func that returns a pointer to a new and fully populated configuration struct for a plugin type.

type PluginConfigSanitiser

type PluginConfigSanitiser func(conf interface{}) interface{}

PluginConfigSanitiser is a function that takes a configuration object for a plugin and returns a sanitised (minimal) version of it for printing in examples and plugin documentation.

This function is useful for when a plugins configuration struct is very large and complex, but can sometimes be expressed in a more concise way without losing the original intent.

type PluginConstructor

type PluginConstructor func(
	config interface{},
	manager types.Manager,
	logger log.Modular,
	metrics metrics.Type,
) (types.Cache, error)

PluginConstructor is a func that constructs a Benthos cache plugin. These are plugins that are specific to certain use cases, experimental, private or otherwise unfit for widespread general use. Any number of plugins can be specified when using Benthos as a framework.

The configuration object will be the result of the PluginConfigConstructor after overlaying the user configuration.

type Redis

type Redis struct {
	// contains filtered or unexported fields
}

Redis is a cache that connects to redis servers.

func (*Redis) Add

func (r *Redis) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists or if the operation fails.

func (*Redis) AddWithTTL

func (r *Redis) AddWithTTL(key string, value []byte, ttl *time.Duration) error

AddWithTTL attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists or if the operation fails.

func (*Redis) CloseAsync

func (r *Redis) CloseAsync()

CloseAsync shuts down the cache.

func (*Redis) Delete

func (r *Redis) Delete(key string) error

Delete attempts to remove a key.

func (*Redis) Get

func (r *Redis) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist or if the operation failed.

func (*Redis) Set

func (r *Redis) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*Redis) SetMulti

func (r *Redis) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Redis) SetMultiWithTTL

func (r *Redis) SetMultiWithTTL(items map[string]types.CacheTTLItem) error

SetMultiWithTTL attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Redis) SetWithTTL

func (r *Redis) SetWithTTL(key string, value []byte, ttl *time.Duration) error

SetWithTTL attempts to set the value of a key.

func (*Redis) WaitForClose

func (r *Redis) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type RedisConfig

type RedisConfig struct {
	bredis.Config `json:",inline" yaml:",inline"`
	Prefix        string `json:"prefix" yaml:"prefix"`
	Expiration    string `json:"expiration" yaml:"expiration"`
	Retries       int    `json:"retries" yaml:"retries"`
	RetryPeriod   string `json:"retry_period" yaml:"retry_period"`
}

RedisConfig is a config struct for a redis connection.

func NewRedisConfig

func NewRedisConfig() RedisConfig

NewRedisConfig returns a RedisConfig with default values.

type Ristretto

type Ristretto struct {
	// contains filtered or unexported fields
}

Ristretto is a memory based cache implementation.

func (*Ristretto) Add

func (r *Ristretto) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*Ristretto) AddWithTTL

func (r *Ristretto) AddWithTTL(key string, value []byte, ttl *time.Duration) error

AddWithTTL attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*Ristretto) CloseAsync

func (r *Ristretto) CloseAsync()

CloseAsync shuts down the cache.

func (*Ristretto) Delete

func (r *Ristretto) Delete(key string) error

Delete attempts to remove a key.

func (*Ristretto) Get

func (r *Ristretto) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

func (*Ristretto) Set

func (r *Ristretto) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*Ristretto) SetMulti

func (r *Ristretto) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Ristretto) SetMultiWithTTL

func (r *Ristretto) SetMultiWithTTL(items map[string]types.CacheTTLItem) error

SetMultiWithTTL attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Ristretto) SetWithTTL

func (r *Ristretto) SetWithTTL(key string, value []byte, ttl *time.Duration) error

SetWithTTL attempts to set the value of a key.

func (*Ristretto) WaitForClose

func (r *Ristretto) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type RistrettoConfig

type RistrettoConfig struct {
	TTL         string `json:"ttl" yaml:"ttl"`
	Retries     int    `json:"retries" yaml:"retries"`
	RetryPeriod string `json:"retry_period" yaml:"retry_period"`
}

RistrettoConfig contains config fields for the Ristretto cache type.

func NewRistrettoConfig

func NewRistrettoConfig() RistrettoConfig

NewRistrettoConfig creates a RistrettoConfig populated with default values.

type S3

type S3 struct {
	// contains filtered or unexported fields
}

S3 is a file system based cache implementation.

func (*S3) Add

func (s *S3) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*S3) CloseAsync

func (s *S3) CloseAsync()

CloseAsync shuts down the cache.

func (*S3) Delete

func (s *S3) Delete(key string) error

Delete attempts to remove a key.

func (*S3) Get

func (s *S3) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

func (*S3) Set

func (s *S3) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*S3) SetMulti

func (s *S3) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*S3) WaitForClose

func (s *S3) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type S3Config

type S3Config struct {
	sess.Config        `json:",inline" yaml:",inline"`
	Bucket             string `json:"bucket" yaml:"bucket"`
	ForcePathStyleURLs bool   `json:"force_path_style_urls" yaml:"force_path_style_urls"`
	ContentType        string `json:"content_type" yaml:"content_type"`
	Timeout            string `json:"timeout" yaml:"timeout"`
	Retries            int    `json:"retries" yaml:"retries"`
}

S3Config contains config fields for the S3 cache type.

func NewS3Config

func NewS3Config() S3Config

NewS3Config creates a S3Config populated with default values.

type TypeSpec

type TypeSpec struct {
	Summary     string
	Description string
	Footnotes   string

	FieldSpecs        docs.FieldSpecs
	Status            docs.Status
	SupportsPerKeyTTL bool
	Version           string
	// contains filtered or unexported fields
}

TypeSpec is a constructor and a usage description for each cache type.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL