cache

package
v3.47.0-rc2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 14, 2021 License: MIT Imports: 37 Imported by: 7

Documentation

Overview

Package cache implements the types.Cache interface for storing and retrieving key/value pairs from a range of storage strategies.

Index

Constants

View Source 
const (
	TypeAWSDynamoDB = "aws_dynamodb"
	TypeAWSS3       = "aws_s3"
	TypeDynamoDB    = "dynamodb"
	TypeFile        = "file"
	TypeMemcached   = "memcached"
	TypeMemory      = "memory"
	TypeMongoDB     = "mongodb"
	TypeMultilevel  = "multilevel"
	TypeRedis       = "redis"
	TypeRistretto   = "ristretto"
	TypeS3          = "s3"
)

String constants representing each cache type.

Variables

View Source 
var Constructors = map[string]TypeSpec{}

Constructors is a map of all cache types with their specs.

Functions

func Descriptions 

func Descriptions() string

Descriptions returns a formatted string of descriptions for each type.

func DocumentPlugin 

func DocumentPlugin(
	typeString, description string,
	configSanitiser PluginConfigSanitiser,
)

DocumentPlugin adds a description and an optional configuration sanitiser function to the definition of a registered plugin. This improves the documentation generated by PluginDescriptions.

func New 

func New(
	conf Config,
	mgr types.Manager,
	log log.Modular,
	stats metrics.Type,
) (types.Cache, error)

New creates a cache type based on an cache configuration.

func NewAWSDynamoDB added in v3.36.0 

func NewAWSDynamoDB(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewAWSDynamoDB creates a new DynamoDB cache type.

func NewAWSS3 added in v3.36.0 

func NewAWSS3(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewAWSS3 creates a new S3 cache type.

func NewDynamoDB 

func NewDynamoDB(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewDynamoDB creates a new DynamoDB cache type.

func NewFile 

func NewFile(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewFile creates a new File cache type.

func NewMemcached 

func NewMemcached(
	conf Config, mgr types.Manager, log log.Modular, stats metrics.Type,
) (types.Cache, error)

NewMemcached returns a Memcached processor.

func NewMemory 

func NewMemory(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewMemory creates a new Memory cache type.

func NewMultilevel added in v3.11.0 

func NewMultilevel(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewMultilevel creates a new Multilevel cache type.

func NewRedis 

func NewRedis(
	conf Config, mgr types.Manager, log log.Modular, stats metrics.Type,
) (types.Cache, error)

NewRedis returns a Redis processor.

func NewRistretto added in v3.24.0 

func NewRistretto(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewRistretto creates a new Ristretto cache type.

func NewS3 

func NewS3(conf Config, mgr types.Manager, log log.Modular, stats metrics.Type) (types.Cache, error)

NewS3 creates a new S3 cache type.

func PluginCount 

func PluginCount() int

PluginCount returns the number of registered plugins. This does NOT count the standard set of components.

func PluginDescriptions 

func PluginDescriptions() string

PluginDescriptions generates and returns a markdown formatted document listing each registered plugin and an example configuration for it.

func RegisterPlugin 

func RegisterPlugin(
	typeString string,
	configConstructor PluginConfigConstructor,
	constructor PluginConstructor,
)

RegisterPlugin registers a plugin by a unique name so that it can be constructed similar to regular caches. If configuration is not needed for this plugin then configConstructor can be nil. A constructor for the plugin itself must be provided.

func SanitiseConfig 

func SanitiseConfig(conf Config) (interface{}, error)

SanitiseConfig creates a sanitised version of a config.

func WalkConstructors added in v3.43.0 

func WalkConstructors(fn func(ConstructorFunc, docs.ComponentSpec))

WalkConstructors iterates each component constructor.

Types

type Config 

type Config struct {
	Label       string           `json:"label" yaml:"label"`
	Type        string           `json:"type" yaml:"type"`
	AWSDynamoDB DynamoDBConfig   `json:"aws_dynamodb" yaml:"aws_dynamodb"`
	AWSS3       S3Config         `json:"aws_s3" yaml:"aws_s3"`
	DynamoDB    DynamoDBConfig   `json:"dynamodb" yaml:"dynamodb"`
	File        FileConfig       `json:"file" yaml:"file"`
	Memcached   MemcachedConfig  `json:"memcached" yaml:"memcached"`
	Memory      MemoryConfig     `json:"memory" yaml:"memory"`
	MongoDB     MongoDBConfig    `json:"mongodb" yaml:"mongodb"`
	Multilevel  MultilevelConfig `json:"multilevel" yaml:"multilevel"`
	Plugin      interface{}      `json:"plugin,omitempty" yaml:"plugin,omitempty"`
	Redis       RedisConfig      `json:"redis" yaml:"redis"`
	Ristretto   RistrettoConfig  `json:"ristretto" yaml:"ristretto"`
	S3          S3Config         `json:"s3" yaml:"s3"`
}

Config is the all encompassing configuration struct for all cache types.

func NewConfig 

func NewConfig() Config

NewConfig returns a configuration struct fully populated with default values.

func (Config) Sanitised added in v3.24.0 

func (conf Config) Sanitised(removeDeprecated bool) (interface{}, error)

Sanitised returns a sanitised version of the config, meaning sections that aren't relevant to behaviour are removed. Also optionally removes deprecated fields.

func (*Config) UnmarshalYAML 

func (conf *Config) UnmarshalYAML(value *yaml.Node) error

UnmarshalYAML ensures that when parsing configs that are in a map or slice the default values are still applied.

type ConstructorFunc added in v3.43.0 

type ConstructorFunc func(Config, types.Manager, log.Modular, metrics.Type) (types.Cache, error)

ConstructorFunc is a func signature able to construct a cache.

func GetDeprecatedPlugin added in v3.43.0 

func GetDeprecatedPlugin(name string) (ConstructorFunc, bool)

GetDeprecatedPlugin returns a constructor for an old plugin if it exists.

type DynamoDB 

type DynamoDB struct {
	// contains filtered or unexported fields
}

DynamoDB is a DynamoDB based cache implementation.

func (*DynamoDB) Add 

func (d *DynamoDB) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*DynamoDB) CloseAsync 

func (d *DynamoDB) CloseAsync()

CloseAsync shuts down the cache.

func (*DynamoDB) Delete 

func (d *DynamoDB) Delete(key string) error

Delete attempts to remove a key.

func (*DynamoDB) Get 

func (d *DynamoDB) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

func (*DynamoDB) Set 

func (d *DynamoDB) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*DynamoDB) SetMulti 

func (d *DynamoDB) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, if any keys fail to be set an error is returned.

func (*DynamoDB) WaitForClose 

func (d *DynamoDB) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type DynamoDBConfig 

type DynamoDBConfig struct {
	ConsistentRead bool   `json:"consistent_read" yaml:"consistent_read"`
	DataKey        string `json:"data_key" yaml:"data_key"`
	HashKey        string `json:"hash_key" yaml:"hash_key"`
	Table          string `json:"table" yaml:"table"`
	TTL            string `json:"ttl" yaml:"ttl"`
	TTLKey         string `json:"ttl_key" yaml:"ttl_key"`
	retries.Config `json:",inline" yaml:",inline"`
	// contains filtered or unexported fields
}

DynamoDBConfig contains config fields for the DynamoDB cache type.

func NewDynamoDBConfig 

func NewDynamoDBConfig() DynamoDBConfig

NewDynamoDBConfig creates a MemoryConfig populated with default values.

type File 

type File struct {
	// contains filtered or unexported fields
}

File is a file system based cache implementation.

func (*File) Add 

func (f *File) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*File) CloseAsync 

func (f *File) CloseAsync()

CloseAsync shuts down the cache.

func (*File) Delete 

func (f *File) Delete(key string) error

Delete attempts to remove a key.

func (*File) Get 

func (f *File) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

func (*File) Set 

func (f *File) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*File) SetMulti 

func (f *File) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*File) WaitForClose 

func (f *File) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type FileConfig 

type FileConfig struct {
	Directory string `json:"directory" yaml:"directory"`
}

FileConfig contains config fields for the File cache type.

func NewFileConfig 

func NewFileConfig() FileConfig

NewFileConfig creates a FileConfig populated with default values.

type Memcached 

type Memcached struct {
	// contains filtered or unexported fields
}

Memcached is a cache that connects to memcached servers.

func (*Memcached) Add 

func (m *Memcached) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists or if the operation fails.

func (*Memcached) AddWithTTL added in v3.33.0 

func (m *Memcached) AddWithTTL(key string, value []byte, ttl *time.Duration) error

AddWithTTL attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists or if the operation fails.

func (*Memcached) CloseAsync 

func (m *Memcached) CloseAsync()

CloseAsync shuts down the cache.

func (*Memcached) Delete 

func (m *Memcached) Delete(key string) error

Delete attempts to remove a key.

func (*Memcached) Get 

func (m *Memcached) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist or if the operation failed.

func (*Memcached) Set 

func (m *Memcached) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*Memcached) SetMulti 

func (m *Memcached) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Memcached) SetMultiWithTTL added in v3.33.0 

func (m *Memcached) SetMultiWithTTL(items map[string]types.CacheTTLItem) error

SetMultiWithTTL attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Memcached) SetWithTTL added in v3.33.0 

func (m *Memcached) SetWithTTL(key string, value []byte, ttl *time.Duration) error

SetWithTTL attempts to set the value of a key.

func (*Memcached) WaitForClose 

func (m *Memcached) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type MemcachedConfig 

type MemcachedConfig struct {
	Addresses   []string `json:"addresses" yaml:"addresses"`
	Prefix      string   `json:"prefix" yaml:"prefix"`
	TTL         int32    `json:"ttl" yaml:"ttl"`
	Retries     int      `json:"retries" yaml:"retries"`
	RetryPeriod string   `json:"retry_period" yaml:"retry_period"`
}

MemcachedConfig is a config struct for a memcached connection.

func NewMemcachedConfig 

func NewMemcachedConfig() MemcachedConfig

NewMemcachedConfig returns a MemcachedConfig with default values.

type Memory 

type Memory struct {
	// contains filtered or unexported fields
}

Memory is a memory based cache implementation.

func (*Memory) Add 

func (m *Memory) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*Memory) CloseAsync 

func (m *Memory) CloseAsync()

CloseAsync shuts down the cache.

func (*Memory) Delete 

func (m *Memory) Delete(key string) error

Delete attempts to remove a key.

func (*Memory) Get 

func (m *Memory) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

func (*Memory) Set 

func (m *Memory) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*Memory) SetMulti 

func (m *Memory) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Memory) WaitForClose 

func (m *Memory) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type MemoryConfig 

type MemoryConfig struct {
	TTL                int               `json:"ttl" yaml:"ttl"`
	CompactionInterval string            `json:"compaction_interval" yaml:"compaction_interval"`
	InitValues         map[string]string `json:"init_values" yaml:"init_values"`
	Shards             int               `json:"shards" yaml:"shards"`
}

MemoryConfig contains config fields for the Memory cache type.

func NewMemoryConfig 

func NewMemoryConfig() MemoryConfig

NewMemoryConfig creates a MemoryConfig populated with default values.

type MongoDBConfig added in v3.43.0 

type MongoDBConfig struct {
	client.Config `json:",inline" yaml:",inline"`
	KeyField      string `json:"key_field" yaml:"key_field"`
	ValueField    string `json:"value_field" yaml:"value_field"`
}

MongoDBConfig is a config struct for a mongo connection.

func NewMongoDBConfig added in v3.43.0 

func NewMongoDBConfig() MongoDBConfig

NewMongoDBConfig returns a MongoDBConfig with default values.

type Multilevel added in v3.11.0 

type Multilevel struct {
	// contains filtered or unexported fields
}

Multilevel is a file system based cache implementation.

func (*Multilevel) Add added in v3.11.0 

func (l *Multilevel) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*Multilevel) AddWithTTL added in v3.33.0 

func (l *Multilevel) AddWithTTL(key string, value []byte, ttl *time.Duration) error

AddWithTTL attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*Multilevel) CloseAsync added in v3.11.0 

func (l *Multilevel) CloseAsync()

CloseAsync shuts down the cache.

func (*Multilevel) Delete added in v3.11.0 

func (l *Multilevel) Delete(key string) error

Delete attempts to remove a key.

func (*Multilevel) Get added in v3.11.0 

func (l *Multilevel) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

func (*Multilevel) Set added in v3.11.0 

func (l *Multilevel) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*Multilevel) SetMulti added in v3.11.0 

func (l *Multilevel) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Multilevel) SetMultiWithTTL added in v3.33.0 

func (l *Multilevel) SetMultiWithTTL(items map[string]types.CacheTTLItem) error

SetMultiWithTTL attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Multilevel) SetWithTTL added in v3.33.0 

func (l *Multilevel) SetWithTTL(key string, value []byte, ttl *time.Duration) error

SetWithTTL attempts to set the value of a key.

func (*Multilevel) WaitForClose added in v3.11.0 

func (l *Multilevel) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type MultilevelConfig added in v3.11.0 

type MultilevelConfig []string

MultilevelConfig contains config fields for the Multilevel cache type.

func NewMultilevelConfig added in v3.11.0 

func NewMultilevelConfig() MultilevelConfig

NewMultilevelConfig creates a MultilevelConfig populated with default values.

type PluginConfigConstructor 

type PluginConfigConstructor func() interface{}

PluginConfigConstructor is a func that returns a pointer to a new and fully populated configuration struct for a plugin type.

type PluginConfigSanitiser 

type PluginConfigSanitiser func(conf interface{}) interface{}

PluginConfigSanitiser is a function that takes a configuration object for a plugin and returns a sanitised (minimal) version of it for printing in examples and plugin documentation.

This function is useful for when a plugins configuration struct is very large and complex, but can sometimes be expressed in a more concise way without losing the original intent.

type PluginConstructor 

type PluginConstructor func(
	config interface{},
	manager types.Manager,
	logger log.Modular,
	metrics metrics.Type,
) (types.Cache, error)

PluginConstructor is a func that constructs a Benthos cache plugin. These are plugins that are specific to certain use cases, experimental, private or otherwise unfit for widespread general use. Any number of plugins can be specified when using Benthos as a framework.

The configuration object will be the result of the PluginConfigConstructor after overlaying the user configuration.

type Redis 

type Redis struct {
	// contains filtered or unexported fields
}

Redis is a cache that connects to redis servers.

func (*Redis) Add 

func (r *Redis) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists or if the operation fails.

func (*Redis) AddWithTTL added in v3.33.0 

func (r *Redis) AddWithTTL(key string, value []byte, ttl *time.Duration) error

AddWithTTL attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists or if the operation fails.

func (*Redis) CloseAsync 

func (r *Redis) CloseAsync()

CloseAsync shuts down the cache.

func (*Redis) Delete 

func (r *Redis) Delete(key string) error

Delete attempts to remove a key.

func (*Redis) Get 

func (r *Redis) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist or if the operation failed.

func (*Redis) Set 

func (r *Redis) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*Redis) SetMulti 

func (r *Redis) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Redis) SetMultiWithTTL added in v3.33.0 

func (r *Redis) SetMultiWithTTL(items map[string]types.CacheTTLItem) error

SetMultiWithTTL attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Redis) SetWithTTL added in v3.33.0 

func (r *Redis) SetWithTTL(key string, value []byte, ttl *time.Duration) error

SetWithTTL attempts to set the value of a key.

func (*Redis) WaitForClose 

func (r *Redis) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type RedisConfig 

type RedisConfig struct {
	bredis.Config `json:",inline" yaml:",inline"`
	Prefix        string `json:"prefix" yaml:"prefix"`
	Expiration    string `json:"expiration" yaml:"expiration"`
	Retries       int    `json:"retries" yaml:"retries"`
	RetryPeriod   string `json:"retry_period" yaml:"retry_period"`
}

RedisConfig is a config struct for a redis connection.

func NewRedisConfig 

func NewRedisConfig() RedisConfig

NewRedisConfig returns a RedisConfig with default values.

type Ristretto added in v3.24.0 

type Ristretto struct {
	// contains filtered or unexported fields
}

Ristretto is a memory based cache implementation.

func (*Ristretto) Add added in v3.24.0 

func (r *Ristretto) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*Ristretto) AddWithTTL added in v3.33.0 

func (r *Ristretto) AddWithTTL(key string, value []byte, ttl *time.Duration) error

AddWithTTL attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*Ristretto) CloseAsync added in v3.24.0 

func (r *Ristretto) CloseAsync()

CloseAsync shuts down the cache.

func (*Ristretto) Delete added in v3.24.0 

func (r *Ristretto) Delete(key string) error

Delete attempts to remove a key.

func (*Ristretto) Get added in v3.24.0 

func (r *Ristretto) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

func (*Ristretto) Set added in v3.24.0 

func (r *Ristretto) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*Ristretto) SetMulti added in v3.24.0 

func (r *Ristretto) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Ristretto) SetMultiWithTTL added in v3.33.0 

func (r *Ristretto) SetMultiWithTTL(items map[string]types.CacheTTLItem) error

SetMultiWithTTL attempts to set the value of multiple keys, returns an error if any keys fail.

func (*Ristretto) SetWithTTL added in v3.33.0 

func (r *Ristretto) SetWithTTL(key string, value []byte, ttl *time.Duration) error

SetWithTTL attempts to set the value of a key.

func (*Ristretto) WaitForClose added in v3.24.0 

func (r *Ristretto) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type RistrettoConfig added in v3.24.0 

type RistrettoConfig struct {
	TTL         string `json:"ttl" yaml:"ttl"`
	Retries     int    `json:"retries" yaml:"retries"`
	RetryPeriod string `json:"retry_period" yaml:"retry_period"`
}

RistrettoConfig contains config fields for the Ristretto cache type.

func NewRistrettoConfig added in v3.24.0 

func NewRistrettoConfig() RistrettoConfig

NewRistrettoConfig creates a RistrettoConfig populated with default values.

type S3 

type S3 struct {
	// contains filtered or unexported fields
}

S3 is a file system based cache implementation.

func (*S3) Add 

func (s *S3) Add(key string, value []byte) error

Add attempts to set the value of a key only if the key does not already exist and returns an error if the key already exists.

func (*S3) CloseAsync 

func (s *S3) CloseAsync()

CloseAsync shuts down the cache.

func (*S3) Delete 

func (s *S3) Delete(key string) error

Delete attempts to remove a key.

func (*S3) Get 

func (s *S3) Get(key string) ([]byte, error)

Get attempts to locate and return a cached value by its key, returns an error if the key does not exist.

func (*S3) Set 

func (s *S3) Set(key string, value []byte) error

Set attempts to set the value of a key.

func (*S3) SetMulti 

func (s *S3) SetMulti(items map[string][]byte) error

SetMulti attempts to set the value of multiple keys, returns an error if any keys fail.

func (*S3) WaitForClose 

func (s *S3) WaitForClose(timeout time.Duration) error

WaitForClose blocks until the cache has closed down.

type S3Config 

type S3Config struct {
	sess.Config        `json:",inline" yaml:",inline"`
	Bucket             string `json:"bucket" yaml:"bucket"`
	ForcePathStyleURLs bool   `json:"force_path_style_urls" yaml:"force_path_style_urls"`
	ContentType        string `json:"content_type" yaml:"content_type"`
	Timeout            string `json:"timeout" yaml:"timeout"`
	Retries            int    `json:"retries" yaml:"retries"`
}

S3Config contains config fields for the S3 cache type.

func NewS3Config 

func NewS3Config() S3Config

NewS3Config creates a S3Config populated with default values.

type TypeSpec 

type TypeSpec struct {
	Summary     string
	Description string
	Footnotes   string

	FieldSpecs        docs.FieldSpecs
	Status            docs.Status
	SupportsPerKeyTTL bool
	Version           string
	// contains filtered or unexported fields
}

TypeSpec is a constructor and a usage description for each cache type.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.