hypercache

package module
v0.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 16, 2023 License: MPL-2.0 Imports: 9 Imported by: 0

README

HyperCache

Go CodeQL

Synopsis

HyperCache is a thread-safe high-performance cache implementation in Go that supports multiple backends with the expiration and eviction of items supporting custom algorithms alongside the defaults. It can be used as a standalone cache or as a cache middleware for a service. It can implement a service interface to intercept cache methods and decorate em with middleware (default or custom). It is optimized for performance and flexibility allowing to specify the expiration and eviction intervals, provide and register new eviction algorithms, stats collectors, middleware(s). It ships with a default historigram stats collector and several eviction algorithms, but you can develop and register your own as long as it implements the Eviction Algorithm interface.:

Features
  • Thread-safe
  • High-performance
  • Supports multiple backends, default backends are:
    1. In-memory
    2. Redis
  • Store items in the cache with a key and expiration duration
  • Retrieve items from the cache by their key
  • Delete items from the cache by their key
  • Clear the cache of all items
  • Evitc items in the background based on the cache capacity and items access leveraging several custom eviction algorithms
  • Expire items in the background based on their duration
  • Eviction Algorithm interface to implement custom eviction algorithms.
  • Stats collection with a default stats collector or a custom one that implements the StatsCollector interface.
  • Service interface implementation to allow intercepting cache methods and decorate them with custom or default middleware(s).

Installation

Install HyperCache:

go get -u github.com/hyp3rd/hypercache
performance

Running the benchmarks on a 2019 MacBook Pro with a 2.4 GHz 8-Core Intel Core i9 processor and 32 GB 2400 MHz DDR4 memory, the results are as follows on average, using a pretty busy machine:

make bench
cd tests/benchmark && go test -bench=. -benchmem -benchtime=4s . -timeout 30m
goos: darwin
goarch: amd64
pkg: github.com/hyp3rd/hypercache/tests/benchmark
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkHyperCache_Get-16                          38833602           123.9 ns/op         0 B/op          0 allocs/op
BenchmarkHyperCache_Get_ProactiveEviction-16        38079158           124.4 ns/op         0 B/op          0 allocs/op
BenchmarkHyperCache_Set-16                           4361000          1217 ns/op         203 B/op          3 allocs/op
BenchmarkHyperCache_Set_Proactive_Eviction-16        4343996          1128 ns/op          92 B/op          3 allocs/op
PASS
ok      github.com/hyp3rd/hypercache/tests/benchmark    23.723s
Examples

To run the examples, use the following command:

make run example=eviction  # or any other example

For a full list of examples, refer to the examples directory.

API

The NewInMemoryWithDefaults function creates a new HyperCache instance with the defaults:

  1. The eviction interval is set to 10 minutes.
  2. The eviction algorithm is set to LRU.
  3. The expiration interval is set to 30 minutes.
  4. The capacity of the in-memory backend is set to 1000 items.

To create a new cache with a given capacity, use the New function as described below:

cache, err := hypercache.NewInMemoryWithDefaults(100)
if err != nil {
    // handle error
}

For a fine grained control over the cache configuration, use the New function, for instance:

config := hypercache.NewConfig[backend.InMemory]()
config.HyperCacheOptions = []hypercache.HyperCacheOption[backend.InMemory]{
    hypercache.WithEvictionInterval[backend.InMemory](time.Minute * 10),
    hypercache.WithEvictionAlgorithm[backend.InMemory]("cawolfu"),
}

config.InMemoryOptions = []backend.Option[backend.InMemory]{
    backend.WithCapacity(10),
}

// Create a new HyperCache with a capacity of 10
cache, err := hypercache.New(config)
if err != nil {
    fmt.Println(err)
    return
}

For the full set of configuration options, refer to the config.go file.

Set

Set adds an item to the cache with the given key and value.

err := cache.Set("key", "value", time.Hour)
if err != nil {
    // handle error
}

The Set function takes a key, a value, and a duration as arguments. The key must be a non-empty string, the value can be of any type, and the duration specifies how long the item should stay in the cache before it expires.

Get

Get retrieves the item with the given key from the cache.

value, ok := cache.Get("key")
if !ok {
    // handle item not found
}

The Get function returns the value associated with the given key or an error if the key is not found or has expired.

Remove

Remove deletes items with the given key from the cache. If an item is not found, it does nothing.

err := cache.Remove("key", "key2", "key3")
if err != nil {
    // handle error
}

The Remove function takes a variadic number of keys as arguments and returns an error if any keys are not found.

For a comprehensive API overview, see the documentation.

Service interface for microservices implementation

The Service interface allows intercepting cache methods and decorate them with custom or default middleware(s).

var svc hypercache.Service
hyperCache, err := hypercache.NewInMemoryWithDefaults(10)

if err != nil {
    fmt.Println(err)
    return
}
// assign statsCollector of the backend to use it in middleware
statsCollector := hyperCache.StatsCollector
svc = hyperCache

if err != nil {
    fmt.Println(err)
    return
}

// Example of using zap logger from uber
logger, _ := zap.NewProduction()

sugar := logger.Sugar()
defer sugar.Sync()
defer logger.Sync()

// apply middleware in the same order as you want to execute them
svc = hypercache.ApplyMiddleware(svc,
    // middleware.YourMiddleware,
    func(next hypercache.Service) hypercache.Service {
        return middleware.NewLoggingMiddleware(next, sugar)
    },
    func(next hypercache.Service) hypercache.Service {
        return middleware.NewStatsCollectorMiddleware(next, statsCollector)
    },
)

err = svc.Set("key string", "value any", 0)
if err != nil {
    fmt.Println(err)
    return
}
key, ok := svc.Get("key string")
if !ok {
    fmt.Println("key not found")
    return
}
fmt.Println(key)

Usage

Here is an example of using the HyperCache package. For a more comprehensive overview, see the examples directory.

package main

import (
    "fmt"
    "log"
    "time"

    "github.com/hyp3rd/hypercache"
)

func main() {
    // Create a new HyperCache with a capacity of 10
    cache, err := hypercache.NewInMemoryWithDefaults(10)
    if err != nil {
        fmt.Println(err)
        return
    }
    // Stop the cache when the program exits
    defer cache.Stop()

    log.Println("adding items to the cache")
    // Add 10 items to the cache
    for i := 0; i < 10; i++ {
        key := fmt.Sprintf("key%d", i)
        val := fmt.Sprintf("val%d", i)

        err = cache.Set(key, val, time.Minute)

        if err != nil {
            fmt.Printf("unexpected error: %v\n", err)
            return
        }
    }

    log.Println("fetching items from the cache using the `GetMultiple` method, key11 does not exist")
    // Retrieve the specific of items from the cache
    items, errs := cache.GetMultiple("key1", "key7", "key9", "key11")

    // Print the errors if any
    for k, e := range errs {
        log.Printf("error fetching item %s: %s\n", k, e)
    }

    // Print the items
    for k, v := range items {
        fmt.Println(k, v)
    }

    log.Println("fetching items from the cache using the `GetOrSet` method")
    // Retrieve a specific of item from the cache
    // If the item is not found, set it and return the value
    val, err := cache.GetOrSet("key11", "val11", time.Minute)
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Println(val)

    log.Println("fetching items from the cache using the simple `Get` method")
    item, ok := cache.Get("key7")
    if !ok {
        fmt.Println("item not found")
        return
    }
    fmt.Println(item)
}

License

The code and documentation in this project are released under Mozilla Public License 2.0.

Author

I'm a surfer, a crypto trader, and a software architect with 15 years of experience designing highly available distributed production environments and developing cloud-native apps in public and private clouds. Feel free to hook me up on LinkedIn.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ApplyHyperCacheOptions added in v0.0.4 

func ApplyHyperCacheOptions[T backend.IBackendConstrain](cache *HyperCache[T], options ...Option[T])

ApplyHyperCacheOptions applies the given options to the given cache.

Types

type Config added in v0.0.4 

type Config[T backend.IBackendConstrain] struct {
	// InMemoryOptions is a slice of options that can be used to configure the `InMemory`.
	InMemoryOptions []backend.Option[backend.InMemory]
	// RedisOptions is a slice of options that can be used to configure the `RedisBackend`.
	RedisOptions []backend.Option[backend.RedisBackend]
	// HyperCacheOptions is a slice of options that can be used to configure `HyperCache`.
	HyperCacheOptions []Option[T]
}

Config is a struct that wraps all the configuration options to setup `HyperCache` and its backend.

func NewConfig added in v0.0.4 

func NewConfig[T backend.IBackendConstrain]() *Config[T]

NewConfig returns a new `Config` struct with default values:

  • `InMemoryOptions` is empty
  • `RedisOptions` is empty
  • `HyperCacheOptions` is set to: -- `WithExpirationInterval[T](30 * time.Minute)` -- `WithEvictionAlgorithm[T]("lfu")` -- `WithEvictionInterval[T](10 * time.Minute)`

Each of the above options can be overridden by passing a different option to the `NewConfig` function. It can be used to configure `HyperCache` and its backend and customize the behavior of the cache.

type HyperCache 

type HyperCache[T backend.IBackendConstrain] struct {

	// StatsCollector to collect cache statistics
	StatsCollector stats.ICollector
	// contains filtered or unexported fields
}

HyperCache is a cache that stores items with a key and expiration duration. It supports multiple backends and multiple eviction algorithms. The default in-memory implementation has a custom `ConcurrentMap` to store the items in the cache, The configuration is provided by the `Config` struct and can be customized by using the `With` functions. The cache has two loops that run in the background:

  • The expiration loop runs every `expirationInterval` and checks for expired items.
  • The eviction loop runs every `evictionInterval` and evicts items using the eviction algorithm.

The cache leverages two channels to signal the expiration and eviction loops to start:

  • The expirationTriggerCh channel is used to signal the expiration loop to start.
  • The evictCh channel is used to signal the eviction loop to start.

The cache also has a mutex that is used to protect the eviction algorithm from concurrent access. The stop channel is used to signal the expiration and eviction loops to stop. The evictCh channel is used to signal the eviction loop to start.

func New added in v0.0.5 

func New[T backend.IBackendConstrain](config *Config[T]) (hyperCache *HyperCache[T], err error)

New initializes a new HyperCache with the given configuration. The default configuration is:

  • The eviction interval is set to 10 minutes.
  • The eviction algorithm is set to CAWOLFU.
  • The expiration interval is set to 30 minutes.
  • The stats collector is set to the HistogramStatsCollector stats collector.

func NewInMemoryWithDefaults added in v0.0.5 

func NewInMemoryWithDefaults(capacity int) (hyperCache *HyperCache[backend.InMemory], err error)

NewInMemoryWithDefaults initializes a new HyperCache with the default configuration. The default configuration is:

  • The eviction interval is set to 10 minutes.
  • The eviction algorithm is set to LRU.
  • The expiration interval is set to 30 minutes.
  • The capacity of the in-memory backend is set to 1000 items.

func (*HyperCache[T]) Capacity 

func (hyperCache *HyperCache[T]) Capacity() int

Capacity returns the capacity of the cache.

func (*HyperCache[T]) Clear 

func (hyperCache *HyperCache[T]) Clear() error

Clear removes all items from the cache.

func (*HyperCache[T]) Get 

func (hyperCache *HyperCache[T]) Get(key string) (value any, ok bool)

Get retrieves the item with the given key from the cache.

func (*HyperCache[T]) GetMultiple 

func (hyperCache *HyperCache[T]) GetMultiple(keys ...string) (result map[string]any, failed map[string]error)

GetMultiple retrieves the items with the given keys from the cache.

func (*HyperCache[T]) GetOrSet 

func (hyperCache *HyperCache[T]) GetOrSet(key string, value any, expiration time.Duration) (any, error)

GetOrSet retrieves the item with the given key. If the item is not found, it adds the item to the cache with the given value and expiration duration. If the capacity of the cache is reached, leverage the eviction algorithm.

func (*HyperCache[T]) GetStats added in v0.0.4 

func (hyperCache *HyperCache[T]) GetStats() stats.Stats

GetStats returns the stats collected by the cache.

func (*HyperCache[T]) List 

func (hyperCache *HyperCache[T]) List(filters ...any) ([]*models.Item, error)

List lists the items in the cache that meet the specified criteria. It takes in a variadic number of any type as filters, it then checks the backend type, and calls the corresponding implementation of the List function for that backend, with the filters passed in as arguments

func (*HyperCache[T]) Remove 

func (hyperCache *HyperCache[T]) Remove(keys ...string)

Remove removes items with the given key from the cache. If an item is not found, it does nothing.

func (*HyperCache[T]) Set 

func (hyperCache *HyperCache[T]) Set(key string, value any, expiration time.Duration) error

Set adds an item to the cache with the given key and value. If an item with the same key already exists, it updates the value of the existing item. If the expiration duration is greater than zero, the item will expire after the specified duration. If the capacity of the cache is reached, the cache will leverage the eviction algorithm proactively if the evictionInterval is zero. If not, the background process will take care of the eviction.

func (*HyperCache[T]) SetCapacity 

func (hyperCache *HyperCache[T]) SetCapacity(capacity int)

SetCapacity sets the capacity of the cache. If the new capacity is smaller than the current number of items in the cache, it evicts the excess items from the cache.

func (*HyperCache[T]) SetMultiple added in v0.0.4 

func (hyperCache *HyperCache[T]) SetMultiple(items map[string]any, expiration time.Duration) error

func (*HyperCache[T]) Size 

func (hyperCache *HyperCache[T]) Size() int

Size returns the number of items in the cache.

func (*HyperCache[T]) Stop 

func (hyperCache *HyperCache[T]) Stop()

Stop function stops the expiration and eviction loops and closes the stop channel.

func (*HyperCache[T]) TriggerEviction 

func (hyperCache *HyperCache[T]) TriggerEviction()

TriggerEviction sends a signal to the eviction loop to start.

type Middleware added in v0.0.4 

type Middleware func(Service) Service

Middleware describes a service middleware.

type Option 

type Option[T backend.IBackendConstrain] func(*HyperCache[T])

Option is a function type that can be used to configure the `HyperCache` struct.

func WithEvictionAlgorithm added in v0.0.4 

func WithEvictionAlgorithm[T backend.IBackendConstrain](name string) Option[T]

WithEvictionAlgorithm is an option that sets the eviction algorithm name field of the `HyperCache` struct. The eviction algorithm name determines which eviction algorithm will be used to evict items from the cache. The eviction algorithm name must be one of the following:

  • "LRU" (Least Recently Used) - Implemented in the `eviction/lru.go` file
  • "LFU" (Least Frequently Used) - Implemented in the `eviction/lfu.go` file
  • "CAWOLFU" (Cache-Aware Write-Optimized LFU) - Implemented in the `eviction/cawolfu.go` file
  • "FIFO" (First In First Out)
  • "RANDOM" (Random)
  • "CLOCK" (Clock) - Implemented in the `eviction/clock.go` file
  • "ARC" (Adaptive Replacement Cache) - Implemented in the `eviction/arc.go` file
  • "TTL" (Time To Live)
  • "LFUDA" (Least Frequently Used with Dynamic Aging)
  • "SLRU" (Segmented Least Recently Used)

func WithEvictionInterval 

func WithEvictionInterval[T backend.IBackendConstrain](evictionInterval time.Duration) Option[T]

WithEvictionInterval is an option that sets the eviction interval field of the `HyperCache` struct. The eviction interval determines how often the cache will run the eviction process to remove the least recently used items.

func WithExpirationInterval 

func WithExpirationInterval[T backend.IBackendConstrain](expirationInterval time.Duration) Option[T]

WithExpirationInterval is an option that sets the expiration interval field of the `HyperCache` struct. The expiration interval determines how often the cache will check for and remove expired items.

func WithMaxEvictionCount 

func WithMaxEvictionCount[T backend.IBackendConstrain](maxEvictionCount uint) Option[T]

WithMaxEvictionCount is an option that sets the max eviction count field of the `HyperCache` struct. The max eviction count determines the maximum number of items that can be removed during a single eviction run.

func WithStatsCollector 

func WithStatsCollector[T backend.IBackendConstrain](name string) Option[T]

WithStatsCollector is an option that sets the stats collector field of the `HyperCache` struct. The stats collector is used to collect statistics about the cache.

type Service added in v0.0.4 

type Service interface {
	// Get retrieves a value from the cache using the key
	Get(key string) (value interface{}, ok bool)
	// Set stores a value in the cache using the key and expiration duration
	Set(key string, value any, expiration time.Duration) error
	// GetOrSet retrieves a value from the cache using the key, if the key does not exist, it will set the value using the key and expiration duration
	GetOrSet(key string, value any, expiration time.Duration) (any, error)
	// GetMultiple retrieves a list of values from the cache using the keys
	GetMultiple(keys ...string) (result map[string]any, failed map[string]error)
	// List returns a list of all items in the cache
	List(filters ...any) ([]*models.Item, error)
	// Remove removes a value from the cache using the key
	Remove(keys ...string)
	// Clear removes all values from the cache
	Clear() error
	// Capacity returns the capacity of the cache
	Capacity() int
	// Size returns the number of items in the cache
	Size() int
	// TriggerEviction triggers the eviction of the cache
	TriggerEviction()
	// Stop stops the cache
	Stop()
	// GetStats returns the stats of the cache
	GetStats() stats.Stats
}

Service is the service interface for the HyperCache. It enables middleware to be added to the service.

func ApplyMiddleware added in v0.0.4 

func ApplyMiddleware(svc Service, mw ...Middleware) Service

ApplyMiddleware applies middlewares to a service.

Source Files

View all Source files

Directories

Path Synopsis
redis
examples
clear
eviction
get
list
redis
service
stats
libs
serializer

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.