hypercache

package module
v0.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 4, 2023 License: MPL-2.0 Imports: 13 Imported by: 0

README

HyperCache

Go CodeQL

Synopsis

HyperCache is a thread-safe high-performance cache implementation in Go that supports multiple backends with an optional size limit, expiration, and eviction of items with custom algorithms alongside the defaults. It can be used as a standalone cache, distributed environents, or as a cache middleware for a service. It can implement a service interface to intercept and decorate the cache methods with middleware (default or custom). It is optimized for performance and flexibility, allowing to specify of the expiration and eviction intervals and providing and registering new eviction algorithms, stats collectors, and middleware(s). It ships with a default historigram stats collector and several eviction algorithms. However, you can develop and register your own if it implements the Eviction Algorithm interface.:

Features
  • Thread-safe
  • High-performance
  • Supports multiple, custom backends. Default backends are:
    1. In-memory
    2. Redis
  • Store items in the cache with a key and expiration duration
  • Retrieve items from the cache by their key
  • Delete items from the cache by their key
  • Clear the cache of all items
  • Evitc items in the background based on the cache capacity and items access leveraging several custom eviction algorithms
  • Expire items in the background based on their duration
  • Eviction Algorithm interface to implement custom eviction algorithms.
  • Stats collection with a default stats collector or a custom one that implements the StatsCollector interface.
  • Service interface implementation to allow intercepting cache methods and decorate them with custom or default middleware(s).

Installation

Install HyperCache:

go get -u github.com/hyp3rd/hypercache
performance

Running the benchmarks on a 2019 MacBook Pro with a 2.4 GHz 8-Core Intel Core i9 processor and 32 GB 2400 MHz DDR4 memory, the results are as follows on average, using a pretty busy machine:

make bench
cd tests/benchmark && go test -bench=. -benchmem -benchtime=4s . -timeout 30m
goos: darwin
goarch: amd64
pkg: github.com/hyp3rd/hypercache/tests/benchmark
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkHyperCache_Get-16                          37481116           115.7 ns/op         0 B/op          0 allocs/op
BenchmarkHyperCache_Get_ProactiveEviction-16        39486261           116.2 ns/op         0 B/op          0 allocs/op
BenchmarkHyperCache_List-16                         11299632           412.0 ns/op        85 B/op          1 allocs/op
BenchmarkHyperCache_Set-16                           2765406          1556 ns/op         248 B/op          4 allocs/op
BenchmarkHyperCache_Set_Proactive_Eviction-16        2947629          1700 ns/op         162 B/op          3 allocs/op
PASS
ok      github.com/hyp3rd/hypercache/tests/benchmark    30.031s
Examples

To run the examples, use the following command:

make run example=eviction  # or any other example

For a complete list of examples, refer to the examples directory.

API

The NewInMemoryWithDefaults function creates a new HyperCache instance with the defaults:

  1. The eviction interval is set to 10 minutes.
  2. The eviction algorithm is set to LRU.
  3. The expiration interval is set to 30 minutes.
  4. The capacity of the in-memory backend is set to 1000 items.

To create a new cache with a given capacity, use the New function as described below:

cache, err := hypercache.NewInMemoryWithDefaults(100)
if err != nil {
    // handle error
}

For fine-grained control over the cache configuration, use the New function, for instance:

config := hypercache.NewConfig[backend.InMemory]()
config.HyperCacheOptions = []hypercache.HyperCacheOption[backend.InMemory]{
    hypercache.WithEvictionInterval[backend.InMemory](time.Minute * 10),
    hypercache.WithEvictionAlgorithm[backend.InMemory]("cawolfu"),
}

config.InMemoryOptions = []backend.Option[backend.InMemory]{
    backend.WithCapacity(10),
}

// Create a new HyperCache with a capacity of 10
cache, err := hypercache.New(config)
if err != nil {
    fmt.Println(err)
    return
}

Refer to the config.go file for the full configuration options. For a comprehensive API overview, see the documentation.

Usage

Examples can be too broad for a readme, refer to the examples directory for a more comprehensive overview.

License

The code and documentation in this project are released under Mozilla Public License 2.0.

Author

I'm a surfer, a crypto trader, and a software architect with 15 years of experience designing highly available distributed production environments and developing cloud-native apps in public and private clouds. Feel free to hook me up on LinkedIn.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ApplyHyperCacheOptions added in v0.0.4 

func ApplyHyperCacheOptions[T backend.IBackendConstrain](cache *HyperCache[T], options ...Option[T])

ApplyHyperCacheOptions applies the given options to the given cache.

Types

type BackendManager added in v0.1.3 

type BackendManager struct {
	// contains filtered or unexported fields
}

BackendManager is a factory for creating HyperCache backend instances

func GetDefaultManager added in v0.1.3 

func GetDefaultManager() *BackendManager

GetDefaultManager returns the default BackendManager

func NewBackendManager added in v0.1.3 

func NewBackendManager() *BackendManager

NewBackendManager creates a new BackendManager

func (*BackendManager) RegisterBackend added in v0.1.3 

func (hcm *BackendManager) RegisterBackend(name string, constructor IBackendConstructor)

RegisterBackend registers a new backend constructor

type Config added in v0.0.4 

type Config[T backend.IBackendConstrain] struct {
	// BackendType is the type of the backend to use.
	BackendType string
	// InMemoryOptions is a slice of options that can be used to configure the `InMemory`.
	InMemoryOptions []backend.Option[backend.InMemory]
	// RedisOptions is a slice of options that can be used to configure the `Redis`.
	RedisOptions []backend.Option[backend.Redis]
	// HyperCacheOptions is a slice of options that can be used to configure `HyperCache`.
	HyperCacheOptions []Option[T]
}

Config is a struct that wraps all the configuration options to setup `HyperCache` and its backend.

func NewConfig added in v0.0.4 

func NewConfig[T backend.IBackendConstrain](backendType string) *Config[T]

NewConfig returns a new `Config` struct with default values:

  • `InMemoryOptions` is empty
  • `RedisOptions` is empty
  • `HyperCacheOptions` is set to: -- `WithExpirationInterval[T](30 * time.Minute)` -- `WithEvictionAlgorithm[T]("lfu")` -- `WithEvictionInterval[T](10 * time.Minute)`

Each of the above options can be overridden by passing a different option to the `NewConfig` function. It can be used to configure `HyperCache` and its backend and customize the behavior of the cache.

type HyperCache 

type HyperCache[T backend.IBackendConstrain] struct {

	// StatsCollector to collect cache statistics
	StatsCollector stats.ICollector
	// contains filtered or unexported fields
}

HyperCache is a cache that stores items with a key and expiration duration. It supports multiple backends and multiple eviction algorithms. The default in-memory implementation has a custom `ConcurrentMap` to store the items in the cache, The configuration is provided by the `Config` struct and can be customized by using the `With` functions. The cache has two loops that run in the background:

  • The expiration loop runs every `expirationInterval` and checks for expired items.
  • The eviction loop runs every `evictionInterval` and evicts items using the eviction algorithm.

The cache leverages two channels to signal the expiration and eviction loops to start:

  • The expirationTriggerCh channel is used to signal the expiration loop to start.
  • The evictCh channel is used to signal the eviction loop to start.

The cache also has a mutex that is used to protect the eviction algorithm from concurrent access. The stop channel is used to signal the expiration and eviction loops to stop. The evictCh channel is used to signal the eviction loop to start.

func New added in v0.0.5 

func New[T backend.IBackendConstrain](bm *BackendManager, config *Config[T]) (hyperCache *HyperCache[T], err error)

New initializes a new HyperCache with the given configuration. The default configuration is:

  • The eviction interval is set to 10 minutes.
  • The eviction algorithm is set to CAWOLFU.
  • The expiration interval is set to 30 minutes.
  • The stats collector is set to the HistogramStatsCollector stats collector.

func NewInMemoryWithDefaults added in v0.0.5 

func NewInMemoryWithDefaults(capacity int) (hyperCache *HyperCache[backend.InMemory], err error)

NewInMemoryWithDefaults initializes a new HyperCache with the default configuration. The default configuration is:

  • The eviction interval is set to 10 minutes.
  • The eviction algorithm is set to LRU.
  • The expiration interval is set to 30 minutes.
  • The capacity of the in-memory backend is set to 0 items (no limitations) unless specified.
  • The maximum cache size in bytes is set to 0 (no limitations).

func (*HyperCache[T]) Allocation added in v0.1.0 

func (hyperCache *HyperCache[T]) Allocation() int64

Allocation returns the size allocation in bytes of the current cache.

func (*HyperCache[T]) Capacity 

func (hyperCache *HyperCache[T]) Capacity() int

Capacity returns the capacity of the cache.

func (*HyperCache[T]) Clear 

func (hyperCache *HyperCache[T]) Clear(ctx context.Context) error

Clear removes all items from the cache.

func (*HyperCache[T]) Count added in v0.1.0 

func (hyperCache *HyperCache[T]) Count() int

Count returns the number of items in the cache.

func (*HyperCache[T]) Get 

func (hyperCache *HyperCache[T]) Get(key string) (value any, ok bool)

Get retrieves the item with the given key from the cache returning the value and a boolean indicating if the item was found.

func (*HyperCache[T]) GetMultiple 

func (hyperCache *HyperCache[T]) GetMultiple(ctx context.Context, keys ...string) (result map[string]any, failed map[string]error)

GetMultiple retrieves the items with the given keys from the cache.

func (*HyperCache[T]) GetOrSet 

func (hyperCache *HyperCache[T]) GetOrSet(ctx context.Context, key string, value any, expiration time.Duration) (any, error)

GetOrSet retrieves the item with the given key. If the item is not found, it adds the item to the cache with the given value and expiration duration. If the capacity of the cache is reached, leverage the eviction algorithm.

func (*HyperCache[T]) GetStats added in v0.0.4 

func (hyperCache *HyperCache[T]) GetStats() stats.Stats

GetStats returns the stats collected by the cache.

func (*HyperCache[T]) GetWithInfo added in v0.1.0 

func (hyperCache *HyperCache[T]) GetWithInfo(key string) (*types.Item, bool)

GetWithInfo retrieves the item with the given key from the cache returning the `Item` object and a boolean indicating if the item was found.

func (*HyperCache[T]) List 

func (hyperCache *HyperCache[T]) List(ctx context.Context, filters ...backend.IFilter) ([]*types.Item, error)

List lists the items in the cache that meet the specified criteria. It takes in a variadic number of any type as filters, it then checks the backend type, and calls the corresponding implementation of the List function for that backend, with the filters passed in as arguments

func (*HyperCache[T]) MaxCacheSize added in v0.1.0 

func (hyperCache *HyperCache[T]) MaxCacheSize() int64

MaxCacheSize returns the maximum size in bytes of the cache.

func (*HyperCache[T]) Remove 

func (hyperCache *HyperCache[T]) Remove(ctx context.Context, keys ...string)

Remove removes items with the given key from the cache. If an item is not found, it does nothing.

func (*HyperCache[T]) Set 

func (hyperCache *HyperCache[T]) Set(ctx context.Context, key string, value any, expiration time.Duration) error

Set adds an item to the cache with the given key and value. If an item with the same key already exists, it updates the value of the existing item. If the expiration duration is greater than zero, the item will expire after the specified duration. If the capacity of the cache is reached, the cache will leverage the eviction algorithm proactively if the evictionInterval is zero. If not, the background process will take care of the eviction.

func (*HyperCache[T]) SetCapacity 

func (hyperCache *HyperCache[T]) SetCapacity(ctx context.Context, capacity int)

SetCapacity sets the capacity of the cache. If the new capacity is smaller than the current number of items in the cache, it evicts the excess items from the cache.

func (*HyperCache[T]) Stop 

func (hyperCache *HyperCache[T]) Stop()

Stop function stops the expiration and eviction loops and closes the stop channel.

func (*HyperCache[T]) TriggerEviction 

func (hyperCache *HyperCache[T]) TriggerEviction()

TriggerEviction sends a signal to the eviction loop to start.

type IBackendConstructor added in v0.1.3 

type IBackendConstructor interface {
	Create(config interface{}) (interface{}, error)
}

IBackendConstructor is an interface for backend constructors

type InMemoryBackendConstructor added in v0.1.3 

type InMemoryBackendConstructor struct{}

InMemoryBackendConstructor is a backend constructor for InMemory

func (InMemoryBackendConstructor) Create added in v0.1.3 

func (ibc InMemoryBackendConstructor) Create(config interface{}) (interface{}, error)

Create creates a new InMemory backend

type JobFunc added in v0.1.3 

type JobFunc func() error

JobFunc is a function that can be enqueued in a worker pool.

type Middleware added in v0.0.4 

type Middleware func(Service) Service

Middleware describes a service middleware.

type Option 

type Option[T backend.IBackendConstrain] func(*HyperCache[T])

Option is a function type that can be used to configure the `HyperCache` struct.

func WithEvictionAlgorithm added in v0.0.4 

func WithEvictionAlgorithm[T backend.IBackendConstrain](name string) Option[T]

WithEvictionAlgorithm is an option that sets the eviction algorithm name field of the `HyperCache` struct. The eviction algorithm name determines which eviction algorithm will be used to evict items from the cache. The eviction algorithm name must be one of the following:

  • "LRU" (Least Recently Used) - Implemented in the `eviction/lru.go` file
  • "LFU" (Least Frequently Used) - Implemented in the `eviction/lfu.go` file
  • "CAWOLFU" (Cache-Aware Write-Optimized LFU) - Implemented in the `eviction/cawolfu.go` file
  • "FIFO" (First In First Out)
  • "RANDOM" (Random)
  • "CLOCK" (Clock) - Implemented in the `eviction/clock.go` file
  • "ARC" (Adaptive Replacement Cache) - Implemented in the `eviction/arc.go` file
  • "TTL" (Time To Live)
  • "LFUDA" (Least Frequently Used with Dynamic Aging)
  • "SLRU" (Segmented Least Recently Used)

func WithEvictionInterval 

func WithEvictionInterval[T backend.IBackendConstrain](evictionInterval time.Duration) Option[T]

WithEvictionInterval is an option that sets the eviction interval field of the `HyperCache` struct. The eviction interval determines how often the cache will run the eviction process to remove the least recently used items.

func WithExpirationInterval 

func WithExpirationInterval[T backend.IBackendConstrain](expirationInterval time.Duration) Option[T]

WithExpirationInterval is an option that sets the expiration interval field of the `HyperCache` struct. The expiration interval determines how often the cache will check for and remove expired items.

func WithMaxCacheSize added in v0.1.0 

func WithMaxCacheSize[T backend.IBackendConstrain](maxCacheSize int64) Option[T]

WithMaxCacheSize is an option that sets the maximum size of the cache. The maximum size of the cache is the maximum number of items that can be stored in the cache. If the maximum size of the cache is reached, the least recently used item will be evicted from the cache.

func WithMaxEvictionCount 

func WithMaxEvictionCount[T backend.IBackendConstrain](maxEvictionCount uint) Option[T]

WithMaxEvictionCount is an option that sets the max eviction count field of the `HyperCache` struct. The max eviction count determines the maximum number of items that can be removed during a single eviction run.

func WithStatsCollector 

func WithStatsCollector[T backend.IBackendConstrain](name string) Option[T]

WithStatsCollector is an option that sets the stats collector field of the `HyperCache` struct. The stats collector is used to collect statistics about the cache.

type RedisBackendConstructor added in v0.1.3 

type RedisBackendConstructor struct{}

RedisBackendConstructor is a backend constructor for Redis

func (RedisBackendConstructor) Create added in v0.1.3 

func (rbc RedisBackendConstructor) Create(config interface{}) (interface{}, error)

Create creates a new Redis backend

type Service added in v0.0.4 

type Service interface {
	// Get retrieves a value from the cache using the key
	Get(key string) (value interface{}, ok bool)
	// Set stores a value in the cache using the key and expiration duration
	Set(ctx context.Context, key string, value any, expiration time.Duration) error
	// GetOrSet retrieves a value from the cache using the key, if the key does not exist, it will set the value using the key and expiration duration
	GetOrSet(ctx context.Context, key string, value any, expiration time.Duration) (any, error)
	// GetWithInfo fetches from the cache using the key, and returns the `types.Item` and a boolean indicating if the key exists
	GetWithInfo(key string) (*types.Item, bool)
	// GetMultiple retrieves a list of values from the cache using the keys
	GetMultiple(ctx context.Context, keys ...string) (result map[string]any, failed map[string]error)
	// List returns a list of all items in the cache
	List(ctx context.Context, filters ...backend.IFilter) ([]*types.Item, error)
	// Remove removes a value from the cache using the key
	Remove(ctx context.Context, keys ...string)
	// Clear removes all values from the cache
	Clear(ctx context.Context) error
	// Capacity returns the capacity of the cache
	Capacity() int
	// Allocation returns the allocation in bytes of the current cache
	Allocation() int64
	// Count returns the number of items in the cache
	Count() int
	// TriggerEviction triggers the eviction of the cache
	TriggerEviction()
	// Stop stops the cache
	Stop()
	// GetStats returns the stats of the cache
	GetStats() stats.Stats
}

Service is the service interface for the HyperCache. It enables middleware to be added to the service.

func ApplyMiddleware added in v0.0.4 

func ApplyMiddleware(svc Service, mw ...Middleware) Service

ApplyMiddleware applies middlewares to a service.

type WorkerPool added in v0.1.3 

type WorkerPool struct {
	// contains filtered or unexported fields
}

WorkerPool is a pool of workers that can execute jobs concurrently.

func NewWorkerPool added in v0.1.3 

func NewWorkerPool(workers int) *WorkerPool

NewWorkerPool creates a new worker pool with the given number of workers.

func (*WorkerPool) Enqueue added in v0.1.3 

func (pool *WorkerPool) Enqueue(job JobFunc)

Enqueue adds a job to the worker pool.

func (*WorkerPool) Errors added in v0.1.3 

func (pool *WorkerPool) Errors() <-chan error

Errors returns a channel that can be used to receive errors from the worker pool.

func (*WorkerPool) Resize added in v0.1.3 

func (pool *WorkerPool) Resize(newSize int)

Resize resizes the worker pool.

func (*WorkerPool) Shutdown added in v0.1.3 

func (pool *WorkerPool) Shutdown()

Shutdown shuts down the worker pool. It waits for all jobs to finish.

Source Files

View all Source files

Directories

Path Synopsis
redis
v3
v4
examples
clear
eviction
get
list
redis
service
size
stats
libs
serializer

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.