cache

README

Mango Cache

Partial implementations of Guava Cache in Go.

Supported cache replacement policies:

• LRU
• Segmented LRU (default)
• TinyLFU (experimental)

The TinyLFU implementation is inspired by Caffeine by Ben Manes and go-tinylfu by Damian Gryski.

Download

go get -u github.com/goburrow/cache

Example

package main

import (
	"fmt"
	"math/rand"
	"time"

	"github.com/goburrow/cache"
)

func main() {
	load := func(k cache.Key) (cache.Value, error) {
		time.Sleep(100 * time.Millisecond) // Slow task
		return fmt.Sprintf("%d", k), nil
	}
	// Create a loading cache
	c := cache.NewLoadingCache(load,
		cache.WithMaximumSize(100),                 // Limit number of entries in the cache.
		cache.WithExpireAfterAccess(1*time.Minute), // Expire entries after 1 minute since last accessed.
		cache.WithRefreshAfterWrite(2*time.Minute), // Expire entries after 2 minutes since last created.
	)

	getTicker := time.Tick(100 * time.Millisecond)
	reportTicker := time.Tick(5 * time.Second)
	for {
		select {
		case <-getTicker:
			_, _ = c.Get(rand.Intn(200))
		case <-reportTicker:
			st := cache.Stats{}
			c.Stats(&st)
			fmt.Printf("%+v\n", st)
		}
	}
}

Performance

See traces and benchmark

Documentation

Overview

Package cache provides partial implementations of Guava Cache, including support for LRU, Segmented LRU and TinyLFU.

Index

• type Cache
  • func New(options ...Option) Cache
• type Func
• type Hash
• type Key
• type LoaderFunc
• type LoadingCache
  • func NewLoadingCache(loader LoaderFunc, options ...Option) LoadingCache
• type Option
  • func WithExpireAfterAccess(d time.Duration) Option
  • func WithExpireAfterWrite(d time.Duration) Option
  • func WithMaximumSize(size int) Option
  • func WithPolicy(name string) Option
  • func WithRefreshAfterWrite(d time.Duration) Option
  • func WithReloader(reloader Reloader) Option
  • func WithRemovalListener(onRemoval Func) Option
  • func WithStatsCounter(st StatsCounter) Option
• type Reloader
• type Stats
  • func (s *Stats) AverageLoadPenalty() time.Duration
  • func (s *Stats) HitRate() float64
  • func (s *Stats) LoadErrorRate() float64
  • func (s *Stats) MissRate() float64
  • func (s *Stats) RequestCount() uint64
  • func (s *Stats) String() string
• type StatsCounter
• type Value

Types

type Cache

type Cache interface {
	// GetIfPresent returns value associated with Key or (nil, false)
	// if there is no cached value for Key.
	GetIfPresent(Key) (Value, bool)

	// Get all keys.
	GetAllKeys() []interface{}

	// Get all values.
	GetAllValues() []interface{}

	// Get all entries.
	GetAll() map[interface{}]interface{}

	// Put associates value with Key. If a value is already associated
	// with Key, the old one will be replaced with Value.
	Put(Key, Value)

	// Invalidate discards cached value of the given Key.
	Invalidate(Key)

	// InvalidateAll discards all entries.
	InvalidateAll()

	// Size of cache
	Size() int

	// Stats copies cache statistics to given Stats pointer.
	Stats(*Stats)

	// Close implements io.Closer for cleaning up all resources.
	// Users must ensure the cache is not being used before closing or
	// after closed.
	Close() error
}

Cache is a key-value cache which entries are added and stayed in the cache until either are evicted or manually invalidated.

func New

func New(options ...Option) Cache

New returns a local in-memory Cache.

type Func

type Func func(Key, Value)

Func is a generic callback for entry events in the cache.

type Hash

type Hash interface {
	Sum64() uint64
}

Hash is an interface implemented by cache keys to override default hash function.

type Key

type Key interface{}

Key is any value which is comparable. See http://golang.org/ref/spec#Comparison_operators for details.

type LoaderFunc

type LoaderFunc func(Key) (Value, error)

LoaderFunc retrieves the value corresponding to given Key.

type LoadingCache

type LoadingCache interface {
	Cache

	// Get returns value associated with Key or call underlying LoaderFunc
	// to load value if it is not present.
	Get(Key) (Value, error)

	// Refresh loads new value for Key. If the Key already existed, the previous value
	// will continue to be returned by Get while the new value is loading.
	// If Key does not exist, this function will block until the value is loaded.
	Refresh(Key)
}

LoadingCache is a cache with values are loaded automatically and stored in the cache until either evicted or manually invalidated.

func NewLoadingCache

func NewLoadingCache(loader LoaderFunc, options ...Option) LoadingCache

NewLoadingCache returns a new LoadingCache with given loader function and cache options.

type Option

type Option func(c *localCache)

Option add options for default Cache.

func WithExpireAfterAccess

func WithExpireAfterAccess(d time.Duration) Option

WithExpireAfterAccess returns an option to expire a cache entry after the given duration without being accessed.

func WithExpireAfterWrite

func WithExpireAfterWrite(d time.Duration) Option

WithExpireAfterWrite returns an option to expire a cache entry after the given duration from creation.

func WithMaximumSize

func WithMaximumSize(size int) Option

WithMaximumSize returns an Option which sets maximum size for the cache. Any non-positive numbers is considered as unlimited.

func WithPolicy

func WithPolicy(name string) Option

WithPolicy returns an option which sets cache policy associated to the given name. Supported policies are: lru, slru, tinylfu.

func WithRefreshAfterWrite

func WithRefreshAfterWrite(d time.Duration) Option

WithRefreshAfterWrite returns an option to refresh a cache entry after the given duration. This option is only applicable for LoadingCache.

func WithReloader

func WithReloader(reloader Reloader) Option

WithReloader returns an option which sets reloader for a loading cache. By default, each asynchronous reload is run in a go routine. This option is only applicable for LoadingCache.

func WithRemovalListener

func WithRemovalListener(onRemoval Func) Option

WithRemovalListener returns an Option to set cache to call onRemoval for each entry evicted from the cache.

func WithStatsCounter

func WithStatsCounter(st StatsCounter) Option

WithStatsCounter returns an option which overrides default cache stats counter.

type Reloader

type Reloader interface {
	// Reload should reload the value asynchronously.
	// Application must call setFunc to set new value or error.
	Reload(key Key, oldValue Value, setFunc func(Value, error))
	// Close shuts down all running tasks. Currently, the error returned is not being used.
	Close() error
}

Reloader specifies how cache loader is run to refresh value for the given Key. If Reloader is not set, cache values are refreshed in a new go routine.

type Stats

type Stats struct {
	HitCount         uint64
	MissCount        uint64
	LoadSuccessCount uint64
	LoadErrorCount   uint64
	TotalLoadTime    time.Duration
	EvictionCount    uint64
}

Stats is statistics about performance of a cache.

func (*Stats) AverageLoadPenalty

func (s *Stats) AverageLoadPenalty() time.Duration

AverageLoadPenalty returns the average time spent loading new values.

func (*Stats) HitRate

func (s *Stats) HitRate() float64

HitRate returns the ratio of cache requests which were hits.

func (*Stats) LoadErrorRate

func (s *Stats) LoadErrorRate() float64

LoadErrorRate returns the ratio of cache loading attempts which returned errors.

func (*Stats) MissRate

func (s *Stats) MissRate() float64

MissRate returns the ratio of cache requests which were misses.

func (*Stats) RequestCount

func (s *Stats) RequestCount() uint64

RequestCount returns a total of HitCount and MissCount.

func (*Stats) String

func (s *Stats) String() string

String returns a string representation of this statistics.

type StatsCounter

type StatsCounter interface {
	// RecordHits records cache hits.
	RecordHits(count uint64)

	// RecordMisses records cache misses.
	RecordMisses(count uint64)

	// RecordLoadSuccess records successful load of a new entry.
	RecordLoadSuccess(loadTime time.Duration)

	// RecordLoadError records failed load of a new entry.
	RecordLoadError(loadTime time.Duration)

	// RecordEviction records eviction of an entry from the cache.
	RecordEviction()

	// Snapshot writes snapshot of this counter values to the given Stats pointer.
	Snapshot(*Stats)
}

StatsCounter accumulates statistics of a cache.

type Value

type Value interface{}

Value is any value.