README
¶
BigCache 
Fast, concurrent, evicting in-memory cache written to keep big number of entries without impact on performance. BigCache keeps entries on heap but omits GC for them. To achieve that operations on bytes arrays take place, therefore entries (de)serialization in front of the cache will be needed in most use cases.
Usage
Simple initialization
import "github.com/allegro/bigcache"
cache, _ := bigcache.NewBigCache(bigcache.DefaultConfig(10 * time.Minute))
cache.Set("my-unique-key", []byte("value"))
entry, _ := cache.Get("my-unique-key")
fmt.Println(string(entry))
Custom initialization
When cache load can be predicted in advance then it is better to use custom initialization because additional memory allocation can be avoided in that way.
import (
"log"
"github.com/allegro/bigcache"
)
config := bigcache.Config {
// number of shards (must be a power of 2)
Shards: 1024,
// time after which entry can be evicted
LifeWindow: 10 * time.Minute,
// rps * lifeWindow, used only in initial memory allocation
MaxEntriesInWindow: 1000 * 10 * 60,
// max entry size in bytes, used only in initial memory allocation
MaxEntrySize: 500,
// prints information about additional memory allocation
Verbose: true,
// cache will not allocate more memory than this limit, value in MB
// if value is reached then the oldest entries can be overridden for the new ones
// 0 value means no size limit
HardMaxCacheSize: 8192,
// callback fired when the oldest entry is removed because of its expiration time or no space left
// for the new entry, or because delete was called. A bitmask representing the reason will be returned.
// Default value is nil which means no callback and it prevents from unwrapping the oldest entry.
OnRemove: nil,
// OnRemoveWithReason is a callback fired when the oldest entry is removed because of its expiration time or no space left
// for the new entry, or because delete was called. A constant representing the reason will be passed through.
// Default value is nil which means no callback and it prevents from unwrapping the oldest entry.
// Ignored if OnRemove is specified.
OnRemoveWithReason: nil,
}
cache, initErr := bigcache.NewBigCache(config)
if initErr != nil {
log.Fatal(initErr)
}
cache.Set("my-unique-key", []byte("value"))
if entry, err := cache.Get("my-unique-key"); err == nil {
fmt.Println(string(entry))
}
Benchmarks
Three caches were compared: bigcache, freecache and map. Benchmark tests were made using an i7-6700K with 32GB of RAM on Windows 10.
Writes and reads
cd caches_bench; go test -bench=. -benchtime=10s ./... -timeout 30m
BenchmarkMapSet-8 3000000 569 ns/op 202 B/op 3 allocs/op
BenchmarkConcurrentMapSet-8 1000000 1592 ns/op 347 B/op 8 allocs/op
BenchmarkFreeCacheSet-8 3000000 775 ns/op 355 B/op 2 allocs/op
BenchmarkBigCacheSet-8 3000000 640 ns/op 303 B/op 2 allocs/op
BenchmarkMapGet-8 5000000 407 ns/op 24 B/op 1 allocs/op
BenchmarkConcurrentMapGet-8 3000000 558 ns/op 24 B/op 2 allocs/op
BenchmarkFreeCacheGet-8 2000000 682 ns/op 136 B/op 2 allocs/op
BenchmarkBigCacheGet-8 3000000 512 ns/op 152 B/op 4 allocs/op
BenchmarkBigCacheSetParallel-8 10000000 225 ns/op 313 B/op 3 allocs/op
BenchmarkFreeCacheSetParallel-8 10000000 218 ns/op 341 B/op 3 allocs/op
BenchmarkConcurrentMapSetParallel-8 5000000 318 ns/op 200 B/op 6 allocs/op
BenchmarkBigCacheGetParallel-8 20000000 178 ns/op 152 B/op 4 allocs/op
BenchmarkFreeCacheGetParallel-8 20000000 295 ns/op 136 B/op 3 allocs/op
BenchmarkConcurrentMapGetParallel-8 10000000 237 ns/op 24 B/op 2 allocs/op
Writes and reads in bigcache are faster than in freecache. Writes to map are the slowest.
GC pause time
cd caches_bench; go run caches_gc_overhead_comparison.go
Number of entries: 20000000
GC pause for bigcache: 5.8658ms
GC pause for freecache: 32.4341ms
GC pause for map: 52.9661ms
Test shows how long are the GC pauses for caches filled with 20mln of entries. Bigcache and freecache have very similar GC pause time. It is clear that both reduce GC overhead in contrast to map which GC pause time took more than 10 seconds.
How it works
BigCache relies on optimization presented in 1.5 version of Go (issue-9477).
This optimization states that if map without pointers in keys and values is used then GC will omit its content.
Therefore BigCache uses map[uint64]uint32 where keys are hashed and values are offsets of entries.
Entries are kept in bytes array, to omit GC again. Bytes array size can grow to gigabytes without impact on performance because GC will only see single pointer to it.
Bigcache vs Freecache
Both caches provide the same core features but they reduce GC overhead in different ways.
Bigcache relies on map[uint64]uint32, freecache implements its own mapping built on
slices to reduce number of pointers.
Results from benchmark tests are presented above. One of the advantage of bigcache over freecache is that you don’t need to know the size of the cache in advance, because when bigcache is full, it can allocate additional memory for new entries instead of overwriting existing ones as freecache does currently. However hard max size in bigcache also can be set, check HardMaxCacheSize.
HTTP Server
This package also includes an easily deployable HTTP implementation of BigCache, which can be found in the server package.
More
Bigcache genesis is described in allegro.tech blog post: writing a very fast cache service in Go
License
BigCache is released under the Apache 2.0 license (see LICENSE)
Documentation
¶
Index ¶
- Constants
- Variables
- func DefaultLogger() *log.Logger
- type BigCache
- func (c *BigCache) Capacity() int
- func (c *BigCache) Close() error
- func (c *BigCache) Delete(key string) error
- func (c *BigCache) Get(key string) ([]byte, error)
- func (c *BigCache) Iterator() *EntryInfoIterator
- func (c *BigCache) Len() int
- func (c *BigCache) Reset() error
- func (c *BigCache) Set(key string, entry []byte) error
- func (c *BigCache) Stats() Stats
- type Config
- type EntryInfo
- type EntryInfoIterator
- type Hasher
- type Logger
- type RemoveReason
- type Stats
Constants ¶
const ErrCannotRetrieveEntry = iteratorError("Could not retrieve entry from cache")
ErrCannotRetrieveEntry is reported when entry cannot be retrieved from underlying
const ErrInvalidIteratorState = iteratorError("Iterator is in invalid state. Use SetNext() to move to next position")
ErrInvalidIteratorState is reported when iterator is in invalid state
Variables ¶
var ErrEntryNotFound = errors.New("Entry not found")
ErrEntryNotFound is an error type struct which is returned when entry was not found for provided key
Functions ¶
func DefaultLogger ¶ added in v1.1.0
DefaultLogger returns a `Logger` implementation backed by stdlib's log
Types ¶
type BigCache ¶
type BigCache struct {
// contains filtered or unexported fields
}
BigCache is fast, concurrent, evicting cache created to keep big number of entries without impact on performance. It keeps entries on heap but omits GC for them. To achieve that, operations take place on byte arrays, therefore entries (de)serialization in front of the cache will be needed in most use cases.
func NewBigCache ¶
NewBigCache initialize new instance of BigCache
func (*BigCache) Close ¶ added in v1.2.0
Close is used to signal a shutdown of the cache when you are done with it. This allows the cleaning goroutines to exit and ensures references are not kept to the cache preventing GC of the entire cache.
func (*BigCache) Get ¶
Get reads entry for the key. It returns an ErrEntryNotFound when no entry exists for the given key.
func (*BigCache) Iterator ¶
func (c *BigCache) Iterator() *EntryInfoIterator
Iterator returns iterator function to iterate over EntryInfo's from whole cache.
type Config ¶
type Config struct {
// Number of cache shards, value must be a power of two
Shards int
// Time after which entry can be evicted
LifeWindow time.Duration
// Interval between removing expired entries (clean up).
// If set to <= 0 then no action is performed. Setting to < 1 second is counterproductive — bigcache has a one second resolution.
CleanWindow time.Duration
// Max number of entries in life window. Used only to calculate initial size for cache shards.
// When proper value is set then additional memory allocation does not occur.
MaxEntriesInWindow int
// Max size of entry in bytes. Used only to calculate initial size for cache shards.
MaxEntrySize int
// Verbose mode prints information about new memory allocation
Verbose bool
// Hasher used to map between string keys and unsigned 64bit integers, by default fnv64 hashing is used.
Hasher Hasher
// HardMaxCacheSize is a limit for cache size in MB. Cache will not allocate more memory than this limit.
// It can protect application from consuming all available memory on machine, therefore from running OOM Killer.
// Default value is 0 which means unlimited size. When the limit is higher than 0 and reached then
// the oldest entries are overridden for the new ones.
HardMaxCacheSize int
// OnRemove is a callback fired when the oldest entry is removed because of its expiration time or no space left
// for the new entry, or because delete was called.
// Default value is nil which means no callback and it prevents from unwrapping the oldest entry.
OnRemove func(key string, entry []byte)
// OnRemoveWithReason is a callback fired when the oldest entry is removed because of its expiration time or no space left
// for the new entry, or because delete was called. A constant representing the reason will be passed through.
// Default value is nil which means no callback and it prevents from unwrapping the oldest entry.
// Ignored if OnRemove is specified.
OnRemoveWithReason func(key string, entry []byte, reason RemoveReason)
// Logger is a logging interface and used in combination with `Verbose`
// Defaults to `DefaultLogger()`
Logger Logger
// contains filtered or unexported fields
}
Config for BigCache
func DefaultConfig ¶
DefaultConfig initializes config with default values. When load for BigCache can be predicted in advance then it is better to use custom config.
func (Config) OnRemoveFilterSet ¶ added in v1.2.0
func (c Config) OnRemoveFilterSet(reasons ...RemoveReason) Config
OnRemoveFilterSet sets which remove reasons will trigger a call to OnRemoveWithReason. Filtering out reasons prevents bigcache from unwrapping them, which saves cpu.
type EntryInfo ¶
type EntryInfo struct {
// contains filtered or unexported fields
}
EntryInfo holds informations about entry in the cache
type EntryInfoIterator ¶
type EntryInfoIterator struct {
// contains filtered or unexported fields
}
EntryInfoIterator allows to iterate over entries in the cache
func (*EntryInfoIterator) SetNext ¶
func (it *EntryInfoIterator) SetNext() bool
SetNext moves to next element and returns true if it exists.
func (*EntryInfoIterator) Value ¶
func (it *EntryInfoIterator) Value() (EntryInfo, error)
Value returns current value from the iterator
type Hasher ¶
Hasher is responsible for generating unsigned, 64 bit hash of provided string. Hasher should minimize collisions (generating same hash for different strings) and while performance is also important fast functions are preferable (i.e. you can use FarmHash family).
type Logger ¶ added in v1.1.0
type Logger interface {
Printf(format string, v ...interface{})
}
Logger is invoked when `Config.Verbose=true`
type RemoveReason ¶ added in v1.2.0
type RemoveReason uint32
RemoveReason is a value used to signal to the user why a particular key was removed in the OnRemove callback.
const ( // Expired means the key is past its LifeWindow. Expired RemoveReason = iota // NoSpace means the key is the oldest and the cache size was at its maximum when Set was called, or the // entry exceeded the maximum shard size. NoSpace // Deleted means Delete was called and this key was removed as a result. Deleted )
type Stats ¶ added in v1.1.0
type Stats struct {
// Hits is a number of successfully found keys
Hits int64 `json:"hits"`
// Misses is a number of not found keys
Misses int64 `json:"misses"`
// DelHits is a number of successfully deleted keys
DelHits int64 `json:"delete_hits"`
// DelMisses is a number of not deleted keys
DelMisses int64 `json:"delete_misses"`
// Collisions is a number of happened key-collisions
Collisions int64 `json:"collisions"`
}
Stats stores cache statistics
Source Files
Directories