ttlcache

README

TTLCache - an in-memory cache with expiration

TTLCache is a simple key/value cache in golang with the following functions:

1. Expiration of items based on time, or custom function
2. Loader function to retrieve missing keys can be provided. Additional Get calls on the same key block while fetching is in progress (groupcache style).
3. Individual expiring time or global expiring time, you can choose
4. Auto-Extending expiration on Get -or- DNS style TTL, see SkipTTLExtensionOnHit(bool)
5. Can trigger callback on key expiration
6. Cleanup resources by calling Close() at end of lifecycle.
7. Thread-safe with comprehensive testing suite. This code is in production at bol.com on critical systems.

Note (issue #25): by default, due to historic reasons, the TTL will be reset on each cache hit and you need to explicitly configure the cache to use a TTL that will not get extended.

Usage

go get github.com/ReneKroon/ttlcache/v2

You can copy it as a full standalone demo program. The first snippet is basic usage, where the second exploits more options in the cache.

Basic:

package main

import (
	"fmt"
	"time"

	"github.com/ReneKroon/ttlcache/v2"
)

var notFound = ttlcache.ErrNotFound

func main() {
	var cache ttlcache.SimpleCache = ttlcache.NewCache()

	cache.SetTTL(time.Duration(10 * time.Second))
	cache.Set("MyKey", "MyValue")
	cache.Set("MyNumber", 1000)

	if val, err := cache.Get("MyKey"); err != notFound {
		fmt.Printf("Got it: %s\n", val)
	}

	cache.Remove("MyNumber")
	cache.Purge()
	cache.Close()
}

Advanced:

package main

import (
	"fmt"
	"time"

	"github.com/ReneKroon/ttlcache/v2"
)

var (
	notFound = ttlcache.ErrNotFound
	isClosed = ttlcache.ErrClosed
)

func main() {
	newItemCallback := func(key string, value interface{}) {
		fmt.Printf("New key(%s) added\n", key)
	}
	checkExpirationCallback := func(key string, value interface{}) bool {
		if key == "key1" {
			// if the key equals "key1", the value
			// will not be allowed to expire
			return false
		}
		// all other values are allowed to expire
		return true
	}

	expirationCallback := func(key string, reason ttlcache.EvictionReason, value interface{}) {
		fmt.Printf("This key(%s) has expired because of %s\n", key, reason)
	}

	loaderFunction := func(key string) (data interface{}, ttl time.Duration, err error) {
		ttl = time.Second * 300
		data, err = getFromNetwork(key)

		return data, ttl, err
	}

	cache := ttlcache.NewCache()
	cache.SetTTL(time.Duration(10 * time.Second))
	cache.SetExpirationReasonCallback(expirationCallback)
	cache.SetLoaderFunction(loaderFunction)
	cache.SetNewItemCallback(newItemCallback)
	cache.SetCheckExpirationCallback(checkExpirationCallback)
	cache.SetCacheSizeLimit(2)

	cache.Set("key", "value")
	cache.SetWithTTL("keyWithTTL", "value", 10*time.Second)

	if value, exists := cache.Get("key"); exists == nil {
		fmt.Printf("Got value: %v\n", value)
	}
	count := cache.Count()
	if result := cache.Remove("keyNNN"); result == notFound {
		fmt.Printf("Not found, %d items left\n", count)
	}

	cache.Set("key6", "value")
	cache.Set("key7", "value")
	metrics := cache.GetMetrics()
	fmt.Printf("Total inserted: %d\n", metrics.Inserted)

	cache.Close()

}

func getFromNetwork(key string) (string, error) {
	time.Sleep(time.Millisecond * 30)
	return "value", nil
}

TTLCache - Some design considerations

1. The complexity of the current cache is already quite high. Therefore not all requests can be implemented in a straight-forward manner.
2. The locking should be done only in the exported functions and startExpirationProcessing of the Cache struct. Else data races can occur or recursive locks are needed, which are both unwanted.
3. I prefer correct functionality over fast tests. It's ok for new tests to take seconds to proof something.

Original Project

TTLCache was forked from wunderlist/ttlcache to add extra functions not avaiable in the original scope. The main differences are:

1. A item can store any kind of object, previously, only strings could be saved
2. Optionally, you can add callbacks too: check if a value should expire, be notified if a value expires, and be notified when new values are added to the cache
3. The expiration can be either global or per item
4. Items can exist without expiration time (time.Zero)
5. Expirations and callbacks are realtime. Don't have a pooling time to check anymore, now it's done with a heap.
6. A cache count limiter

Documentation

Index

• Constants
• type Cache
  • func NewCache[K comparable, V any]() *Cache[K, V]
  • func (cache *Cache[K, V]) Close() error
  • func (cache *Cache[K, V]) Count() int
  • func (cache *Cache[K, V]) Get(key K) (V, error)
  • func (cache *Cache[K, V]) GetByLoader(key K, customLoaderFunction LoaderFunction[K, V]) (V, error)
  • func (cache *Cache[K, V]) GetByLoaderWithTtl(key K, customLoaderFunction LoaderFunction[K, V]) (V, time.Duration, error)
  • func (cache *Cache[K, V]) GetItems() map[K]V
  • func (cache *Cache[K, V]) GetKeys() []K
  • func (cache *Cache[K, V]) GetMetrics() Metrics
  • func (cache *Cache[K, V]) GetWithTTL(key K) (V, time.Duration, error)
  • func (cache *Cache[K, V]) Purge() error
  • func (cache *Cache[K, V]) Remove(key K) error
  • func (cache *Cache[K, V]) Set(key K, value V) error
  • func (cache *Cache[K, V]) SetCacheSizeLimit(limit int)
  • func (cache *Cache[K, V]) SetCheckExpirationCallback(callback CheckExpireCallback[K, V])
  • func (cache *Cache[K, V]) SetExpirationCallback(callback ExpireCallback[K, V])
  • func (cache *Cache[K, V]) SetExpirationReasonCallback(callback ExpireReasonCallback[K, V])
  • func (cache *Cache[K, V]) SetLoaderFunction(loader LoaderFunction[K, V])
  • func (cache *Cache[K, V]) SetNewItemCallback(callback ExpireCallback[K, V])
  • func (cache *Cache[K, V]) SetTTL(ttl time.Duration) error
  • func (cache *Cache[K, V]) SetWithTTL(key K, value V, ttl time.Duration) error
  • func (cache *Cache[K, V]) SkipTTLExtensionOnHit(value bool)
  • func (cache *Cache[K, V]) Touch(key K) error
• type CheckExpireCallback
• type EvictionReason
  • func EvictionReasonString(s string) (EvictionReason, error)
  • func EvictionReasonValues() []EvictionReason
  • func (i EvictionReason) IsAEvictionReason() bool
  • func (i EvictionReason) String() string
• type ExpireCallback
• type ExpireReasonCallback
• type LoaderFunction
• type Metrics
• type SimpleCache

Constants

const (
	// ErrClosed is raised when operating on a cache where Close() has
	// already been called.
	ErrClosed = constError("cache already closed")
	// ErrNotFound indicates that the requested key is not present
	// in the cache
	ErrNotFound = constError("key not found")
)

const (
	// ItemNotExpire Will avoid the item being expired by TTL, but can
	// still be exired by callback etc.
	ItemNotExpire time.Duration = -1
	// ItemExpireWithGlobalTTL will use the global TTL when set.
	ItemExpireWithGlobalTTL time.Duration = 0
)

Types

type Cache

type Cache[K comparable, V any] struct {
	// contains filtered or unexported fields
}

Cache is a synchronized map of items that can auto-expire once stale.

func NewCache

func NewCache[K comparable, V any]() *Cache[K, V]

NewCache creates a new instance of the Cache type.

func (*Cache[K, V]) Close

func (cache *Cache[K, V]) Close() error

Close calls Purge after stopping the goroutine that does ttl checking, for a clean shutdown. The cache is no longer cleaning up after the first call to Close, repeated calls are safe and return ErrClosed.

func (*Cache[K, V]) Count

func (cache *Cache[K, V]) Count() int

Count returns the number of items in the cache. Returns zero when the cache has been closed.

func (*Cache[K, V]) Get

func (cache *Cache[K, V]) Get(key K) (V, error)

Get is a thread-safe way to lookup items. Every lookup, also touches the item, hence extending its life.

func (*Cache[K, V]) GetByLoader

func (cache *Cache[K, V]) GetByLoader(key K, customLoaderFunction LoaderFunction[K, V]) (V, error)

GetByLoader can take a per key loader function (i.e. to propagate context).

func (*Cache[K, V]) GetByLoaderWithTtl

func (cache *Cache[K, V]) GetByLoaderWithTtl(key K, customLoaderFunction LoaderFunction[K, V]) (V, time.Duration, error)

GetByLoaderWithTtl can take a per key loader function (i.e. to propagate context)

func (*Cache[K, V]) GetItems

func (cache *Cache[K, V]) GetItems() map[K]V

GetItems returns a copy of all items in the cache. Returns nil when the cache has been closed.

func (*Cache[K, V]) GetKeys

func (cache *Cache[K, V]) GetKeys() []K

GetKeys returns all keys of items in the cache. Returns nil when the cache has been closed.

func (*Cache[K, V]) GetMetrics

func (cache *Cache[K, V]) GetMetrics() Metrics

GetMetrics exposes the metrics of the cache. This is a snapshot copy of the metrics.

func (*Cache[K, V]) GetWithTTL

func (cache *Cache[K, V]) GetWithTTL(key K) (V, time.Duration, error)

GetWithTTL has exactly the same behaviour as Get but also returns the remaining TTL for a specific item at the moment its retrieved.

func (*Cache[K, V]) Purge

func (cache *Cache[K, V]) Purge() error

Purge will remove all entries.

func (*Cache[K, V]) Remove

func (cache *Cache[K, V]) Remove(key K) error

Remove removes an item from the cache if it exists, triggers expiration callback when set. Can return ErrNotFound if the entry was not present.

func (*Cache[K, V]) Set

func (cache *Cache[K, V]) Set(key K, value V) error

Set is a thread-safe way to add new items to the map.

func (*Cache[K, V]) SetCacheSizeLimit

func (cache *Cache[K, V]) SetCacheSizeLimit(limit int)

SetCacheSizeLimit sets a limit to the amount of cached items. If a new item is getting cached, the closes item to being timed out will be replaced. Set to 0 to turn off

func (*Cache[K, V]) SetCheckExpirationCallback

func (cache *Cache[K, V]) SetCheckExpirationCallback(callback CheckExpireCallback[K, V])

SetCheckExpirationCallback sets a callback that will be called when an item is about to expire in order to allow external code to decide whether the item expires or remains for another TTL cycle

func (*Cache[K, V]) SetExpirationCallback

func (cache *Cache[K, V]) SetExpirationCallback(callback ExpireCallback[K, V])

SetExpirationCallback sets a callback that will be called when an item expires.

func (*Cache[K, V]) SetExpirationReasonCallback

func (cache *Cache[K, V]) SetExpirationReasonCallback(callback ExpireReasonCallback[K, V])

SetExpirationReasonCallback sets a callback that will be called when an item expires, includes reason of expiry.

func (*Cache[K, V]) SetLoaderFunction

func (cache *Cache[K, V]) SetLoaderFunction(loader LoaderFunction[K, V])

SetLoaderFunction allows you to set a function to retrieve cache misses. The signature matches that of the Get function. Additional Get calls on the same key block while fetching is in progress (groupcache style).

func (*Cache[K, V]) SetNewItemCallback

func (cache *Cache[K, V]) SetNewItemCallback(callback ExpireCallback[K, V])

SetNewItemCallback sets a callback that will be called when a new item is added to the cache.

func (*Cache[K, V]) SetTTL

func (cache *Cache[K, V]) SetTTL(ttl time.Duration) error

SetTTL sets the global TTL value for items in the cache, which can be overridden at the item level.

func (*Cache[K, V]) SetWithTTL

func (cache *Cache[K, V]) SetWithTTL(key K, value V, ttl time.Duration) error

SetWithTTL is a thread-safe way to add new items to the map with individual ttl.

func (*Cache[K, V]) SkipTTLExtensionOnHit

func (cache *Cache[K, V]) SkipTTLExtensionOnHit(value bool)

SkipTTLExtensionOnHit allows the user to change the cache behaviour. When this flag is set to true it will no longer extend TTL of items when they are retrieved using Get, or when their expiration condition is evaluated using SetCheckExpirationCallback.

func (*Cache[K, V]) Touch

func (cache *Cache[K, V]) Touch(key K) error

Touch resets the TTL of the key when it exists, returns ErrNotFound if the key is not present.

type CheckExpireCallback

type CheckExpireCallback[K comparable, V any] func(key K, value V) bool

CheckExpireCallback is used as a callback for an external check on item expiration.

type EvictionReason

type EvictionReason int

EvictionReason is an enum that explains why an item was evicted.

const (
	// Removed : explicitly removed from cache via API call.
	Removed EvictionReason = iota
	// EvictedSize : evicted due to exceeding the cache size.
	EvictedSize
	// Expired : the time to live is zero and therefore the item is
	// removed.
	Expired
	// Closed : the cache was closed.
	Closed
)

func EvictionReasonString

func EvictionReasonString(s string) (EvictionReason, error)

EvictionReasonString retrieves an enum value from the enum constants string name. Throws an error if the param is not part of the enum.

func EvictionReasonValues

func EvictionReasonValues() []EvictionReason

EvictionReasonValues returns all values of the enum

func (EvictionReason) IsAEvictionReason

func (i EvictionReason) IsAEvictionReason() bool

IsAEvictionReason returns "true" if the value is listed in the enum definition. "false" otherwise

func (EvictionReason) String

func (i EvictionReason) String() string

type ExpireCallback

type ExpireCallback[K comparable, V any] func(key K, value V)

ExpireCallback is used as a callback on item expiration or when notifying of an item new to the cache. Note that ExpireReasonCallback will be the successor of this function in the next major release.

type ExpireReasonCallback

type ExpireReasonCallback[K comparable, V any] func(key K, reason EvictionReason, value V)

ExpireReasonCallback is used as a callback on item expiration with extra information why the item expired.

type LoaderFunction

type LoaderFunction[K comparable, V any] func(key K) (value V, ttl time.Duration, err error)

LoaderFunction can be supplied to retrieve an item where a cache miss occurs. Supply an item specific ttl or Duration.Zero

type Metrics

type Metrics struct {
	// succesful inserts
	Inserted int64
	// retrieval attempts
	Retrievals int64
	// all get calls that were in the cache (excludes loader invocations)
	Hits int64
	// entries not in cache (includes loader invocations)
	Misses int64
	// items removed from the cache in any way
	Evicted int64
}

Metrics contains common cache metrics so you can calculate hit and miss rates.

type SimpleCache

type SimpleCache[K comparable, V any] interface {
	Get(key K) (V, error)
	GetWithTTL(key K) (V, time.Duration, error)
	Set(key K, value V) error
	SetTTL(ttl time.Duration) error
	SetWithTTL(key K, value V, ttl time.Duration) error
	Remove(key K) error
	Close() error
	Purge() error
}

SimpleCache interface enables a quick-start. Interface for basic usage.