wfcache

README

Waterfall Cache

wfcache is a multi-layered cache with waterfall hit propagation and built-in storage adapters for DynamoDB, Redis, BigCache (in-memory)

This project is under active development. Use at your own risk.

wfcache is effective for read-heavy workloads and it can be used both as a side-cache or a read-through/write-through cache.

Built-in Storage Adapters

Package	Description	Eviction strategy
DynamoDB	DynamoDB	TTL
Redis	Redis	TTL/LRU
BigCache	Performant on heap storage with minimal GC	TTL (enforced on add)
GoLRU	In-memory storage with approximated LRU similar to Redis	TTL/LRU
Basic	Basic in-memory storage (not recommended)	TTL (enforced on get)

Installation

To retrieve wfcache, run:

$ go get github.com/juliaqiuxy/wfcache

To update to the latest version, run:

$ go get -u github.com/juliaqiuxy/wfcache

Usage

import "github.com/juliaqiuxy/wfcache"

c, err := wfcache.Create(
  basicAdapter.Create(5 * time.Minute),
  bigCacheAdapter.Create(2 * time.Hour),
  dynamodbAdapter.Create(dynamodbClient, "my-cache-table", 24 * time.Hour),
)

items, err := c.BatchGet(keys)
if err == wfcache.ErrPartiallyFulfilled {
  fmt.Println("Somethings are missing")
}

Usage with hooks

You can configure wfcache to notify you when each storage operation starts and finishes. This is useful when you want to do performance logging, tracing etc.

import (
  "context"
  "github.com/juliaqiuxy/wfcache"
  "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
)

func onStartStorageOp(ctx context.Context, opName string) interface{} {
  span, _ := tracer.StartSpanFromContext(ctx, opName)
  return span
}

func onFinishStorageOp(span interface{}) {
  span.(ddtrace.Span).Finish()
}

wfcache.CreateWithHooks(
  onStartStorageOp,
  onFinishStorageOp,
  basicAdapter.Create(5 * time.Minute),
  bigCacheAdapter.Create(2 * time.Hour),
  dynamodbAdapter.Create(dynamodbClient, "my-cache-table", 24 * time.Hour),
)

How it works

The following steps outline how reads from wfcache work:

• When getting a value, wfcache tries to read it from the first storage layer (e.g. BigCache).
• If the storage layer is not populated with the requested key-value pair (cache miss), transparent to the application, wfcache notes the missing key and moves on to the next layer. This continues until all configured storage options are exhausted.
• When there is a cache hit, wfcache then primes each storage layer with a previously reported cache miss to make the data available for any subsequent reads.
• wfcache returns the key-value pair back to the application

If you want to use wfcache as read-through cache, you can implement a custom adapter for your source database and configure it as the last storage layer. In this setup, a cache miss only ever happens in intermediate storage layers (which are then primed as your source storage resolves values) but wfcache would always yield data.

When mutating wfcache, key-value pairs are written and removed from all storage layers. To mutate a specific storage layer in isolation, you can keep a refernece to it. However, this is not recommended as the interface is subject to change.

Cache eviction

wfcache leaves it up to each storage layer to implement their eviction strategy. Built-in adapters use a combination of Time-to-Live (TTL) and Least Recently Used (LRU) algorithm to decide which items to evict.

Also note that the built-in Basic storage is not meant for production use as the TTL enforcement only happens if and when a "stale" item is requested form the storage layer.

Implementing Custom Adapters

For use cases where:

• you require a stroge adapter which is not included in wfcache, or
• you want to use wfcache as a read-through/write-through cache

it is trivial to extend wfcache by implementing the following adapter interface:

type Storage interface {
  Get(ctx context.Context, key string) *CacheItem
  BatchGet(ctx context.Context, keys []string) []*CacheItem
  Set(ctx context.Context, key string, value []byte) error
  BatchSet(ctx context.Context, pairs map[string][]byte) error
  Del(ctx context.Context, key string) error
}

Documentation

Index

• Variables
• type Cache
  • func Create(maker StorageMaker, otherMakers ...StorageMaker) (*Cache, error)
  • func CreateWithHooks(sop StartStorageOp, fop FinishStorageOp, maker StorageMaker, ...) (*Cache, error)
  • func (c *Cache) BatchGet(keys []string) ([]*CacheItem, error)
  • func (c *Cache) BatchGetWithContext(ctx context.Context, keys []string) ([]*CacheItem, error)
  • func (c *Cache) BatchSet(pairs map[string]interface{}) error
  • func (c *Cache) BatchSetWithContext(ctx context.Context, pairs map[string]interface{}) error
  • func (c *Cache) Del(key string) error
  • func (c *Cache) DelWithContext(ctx context.Context, key string) error
  • func (c *Cache) Get(key string) (*CacheItem, error)
  • func (c *Cache) GetWithContext(ctx context.Context, key string) (*CacheItem, error)
  • func (c *Cache) Set(key string, value interface{}) error
  • func (c *Cache) SetWithContext(ctx context.Context, key string, value interface{}) error
• type CacheItem
• type FinishStorageOp
• type Future
  • func Promise(f func() (interface{}, error)) Future
• type StartStorageOp
• type Storage
• type StorageMaker

Variables

var (
	ErrNotFulfilled       = errors.New("look up not fulfilled")
	ErrPartiallyFulfilled = errors.New("look up only partially fulfilled")
)

Types

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

func Create

func Create(maker StorageMaker, otherMakers ...StorageMaker) (*Cache, error)

func CreateWithHooks

func CreateWithHooks(sop StartStorageOp, fop FinishStorageOp, maker StorageMaker, otherMakers ...StorageMaker) (*Cache, error)

func (*Cache) BatchGet

func (c *Cache) BatchGet(keys []string) ([]*CacheItem, error)

func (*Cache) BatchGetWithContext

func (c *Cache) BatchGetWithContext(ctx context.Context, keys []string) ([]*CacheItem, error)

func (*Cache) BatchSet

func (c *Cache) BatchSet(pairs map[string]interface{}) error

func (*Cache) BatchSetWithContext

func (c *Cache) BatchSetWithContext(ctx context.Context, pairs map[string]interface{}) error

func (*Cache) Del

func (c *Cache) Del(key string) error

func (*Cache) DelWithContext

func (c *Cache) DelWithContext(ctx context.Context, key string) error

func (*Cache) Get

func (c *Cache) Get(key string) (*CacheItem, error)

func (*Cache) GetWithContext

func (c *Cache) GetWithContext(ctx context.Context, key string) (*CacheItem, error)

func (*Cache) Set

func (c *Cache) Set(key string, value interface{}) error

func (*Cache) SetWithContext

func (c *Cache) SetWithContext(ctx context.Context, key string, value interface{}) error

type CacheItem

type CacheItem struct {
	Key       string `json:"key"`
	Value     []byte `json:"value"`
	ExpiresAt int64  `json:"expiresAt"`
}

type FinishStorageOp

type FinishStorageOp func(interface{})

type Future

type Future interface {
	Await() (interface{}, error)
	AwaitWithContext(context.Context) (interface{}, error)
}

func Promise

func Promise(f func() (interface{}, error)) Future

type StartStorageOp

type StartStorageOp func(ctx context.Context, opName string) interface{}

type Storage

type Storage interface {
	TimeToLive() time.Duration

	Get(ctx context.Context, key string) *CacheItem
	BatchGet(ctx context.Context, keys []string) []*CacheItem
	Set(ctx context.Context, key string, value []byte) error
	BatchSet(ctx context.Context, pairs map[string][]byte) error
	Del(ctx context.Context, key string) error
}

type StorageMaker

type StorageMaker func() (Storage, error)