cache

package module
v1.0.0-RC Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 26, 2023 License: MIT Imports: 10 Imported by: 0

README

Redis Cache

Redis Cache is a library for caching any data structure in Redis. Redis Cache is meant to be used with the official Redis Go client and works by unmarshalling and marshaling data structures from/to bytes automatically. By default, Redis Cache will use msgpack to marshal/unmarshal data, but you can customize the behavior by providing your own Marshaller and Unmarshaller using the Serialization option with the NewCache function.

Requirements

  • Go 1.19+
  • Redis 6+

Getting Redis Cache

go get github.com/jkratz55/redis-cache

Usage

Under the hood Redis Cache was designed to be used with go-redis. However, it can work with any type that implements the RedisClient interface.

type RedisClient interface {
    Get(ctx context.Context, key string) *redis.StringCmd
    MGet(ctx context.Context, keys ...string) *redis.SliceCmd
    Set(ctx context.Context, key string, val any, ttl time.Duration) *redis.StatusCmd
    SetNX(ctx context.Context, key string, value interface{}, expiration time.Duration) *redis.BoolCmd
    SetXX(ctx context.Context, key string, value interface{}, expiration time.Duration) *redis.BoolCmd
    Del(ctx context.Context, keys ...string) *redis.IntCmd
    Watch(ctx context.Context, fn func(*redis.Tx) error, keys ...string) error
    Scan(ctx context.Context, cursor uint64, match string, count int64) *redis.ScanCmd
    FlushDB(ctx context.Context) *redis.StatusCmd
    FlushDBAsync(ctx context.Context) *redis.StatusCmd
}

This means that the Cache type can work with the following types in the go-redis client library.

  • redis.Client
  • redis.ClusterClient
  • redis.Ring

The following example shows the basic usage of the Redis Cache library.

package main

import (
	"context"
	"fmt"

	"github.com/redis/go-redis/v9"

	rcache "github.com/jkratz55/redis-cache"
)

type Person struct {
	FirstName string
	LastName  string
	Age       int
}

func main() {
	client := redis.NewClient(&redis.Options{
		Addr: "localhost:6379",
	})

	cache := rcache.NewCache(client)

	if err := cache.Set(context.Background(), "person", Person{
		FirstName: "Biily",
		LastName:  "Bob",
		Age:       45,
	}); err != nil {
		panic("ohhhhh snap!")
	}

	var p Person
	if err := cache.Get(context.Background(), "person", &p); err != nil {
		panic("ohhhhh snap")
	}
	fmt.Printf("%v\n", p)

	if err := cache.Delete(context.Background(), "person"); err != nil {
		panic("ohhh snap!")
	}

	if err := cache.Get(context.Background(), "person", &p); err != rcache.ErrKeyNotFound {
		panic("ohhhhh snap, this key should be gone!")
	}
}

If you wanted to use json instead of msgpack you could have customized the Cache like the example below.

marshaller := func(v any) ([]byte, error) {
    return json.Marshal(v)
}
unmarshaller := func(data []byte, v any) error {
    return json.Unmarshal(data, v)
}
rdb := rcache.NewCache(client, rcache.Serialization(marshaller, unmarshaller))

Because of limitations in GO's implementation of generics MGet is a function instead of a method on the Cache type. The MGet function accepts the Cache type as an argument to leverage the same marshaller and unmarshaller.

This library also supports atomic updates of existing keys by using the Upsert and UpsertTTL functions. If the key was modified while the upsert is in progress it will return RetryableError signaling the operation can be retried and the UpsertCallback can decide how to handle merging the changes.

Compression

In some cases compressing values stored in Redis can have tremendous benefits, particularly when storing large volumes of data, large values per key, or both. Compression reduces the size of the cache, significantly decreases bandwidth and latency but at the cost of additional CPU consumption.

This library can be configured to automatically compress and decompress values using the Compression option when calling NewCache. It accepts a Codec and out of the box gzip, flate, and lz4 are supported. However, you are free to compress data any way you please by implementing the Codec interface.

package main

import (
	"context"
	"fmt"

	"github.com/redis/go-redis/v9"

	rcache "github.com/jkratz55/redis-cache"
)

type Person struct {
	FirstName string
	LastName  string
	Age       int
}

func main() {
	client := redis.NewClient(&redis.Options{
		Addr: "localhost:6379",
	})

	c := rcache.NewCache(client, rcache.Compression(rcache.NewCodec(rcache.GZip)))

	if err := c.Set(context.Background(), "person", Person{
		FirstName: "Biily",
		LastName:  "Bob",
		Age:       45,
	}); err != nil {
		panic("ohhhhh snap!")
	}

	var p Person
	if err := c.Get(context.Background(), "person", &p); err != nil {
		panic("ohhhhh snap")
	}
	fmt.Printf("%v\n", p)

	if err := c.Delete(context.Background(), "person"); err != nil {
		panic("ohhh snap!")
	}

	if err := c.Get(context.Background(), "person", &p); err != rcache.ErrKeyNotFound {
		panic("ohhhhh snap, this key should be gone!")
	}
}

In the above example NewCodec is used which creates a default configured flate, gzip, or lz4 codec. If you wish to configure the behavior of the codecs you can create them yourself and rather than using NewCodec.

package main

import (
	"compress/flate"
	"context"
	"fmt"

	"github.com/redis/go-redis/v9"

	rcache "github.com/jkratz55/redis-cache"
	"github.com/jkratz55/redis-cache/compression/gzip"
)

type Person struct {
	FirstName string
	LastName  string
	Age       int
}

func main() {
	client := redis.NewClient(&redis.Options{
		Addr: "localhost:6379",
	})

	c := rcache.NewCache(client, rcache.Compression(gzip.NewCodec(flate.BestSpeed)))

	if err := c.Set(context.Background(), "person", Person{
		FirstName: "Biily",
		LastName:  "Bob",
		Age:       45,
	}); err != nil {
		panic("ohhhhh snap!")
	}

	var p Person
	if err := c.Get(context.Background(), "person", &p); err != nil {
		panic("ohhhhh snap")
	}
	fmt.Printf("%v\n", p)

	if err := c.Delete(context.Background(), "person"); err != nil {
		panic("ohhh snap!")
	}

	if err := c.Get(context.Background(), "person", &p); err != rcache.ErrKeyNotFound {
		panic("ohhhhh snap, this key should be gone!")
	}
}

In the above example gzip codec is used but configured for best speed rather than the default best compression.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrKeyNotFound is an error value that signals the key requested does not
	// exist in the cache.
	ErrKeyNotFound = errors.New("key not found")
)

Functions

func IsRetryable added in v0.5.0 

func IsRetryable(err error) bool

IsRetryable accepts an error and returns a boolean indicating if the operation that generated the error is retryable.

func Upsert added in v0.5.0 

func Upsert[T any](ctx context.Context, c *Cache, key string, val T, cb UpsertCallback[T]) error

Upsert retrieves the existing value for a given key and invokes the UpsertCallback. The UpsertCallback function is responsible for determining the value to be stored. The value returned from the UpsertCallback is what is set in Redis.

Upsert allows for atomic updates of existing records, or simply inserting new entries when the key doesn't exist.

Redis uses an optimistic locking model. If the key changes during the transaction Redis will fail the transaction and return an error. However, these errors are retryable. To determine if the error is retryable use the IsRetryable function with the returned error. 

cb := rcache.UpsertCallback[Person](func(found bool, oldValue Person, newValue Person) Person {
	fmt.Println(found)
	fmt.Println(oldValue)
	fmt.Println(newValue)
	return newValue
})
retries := 3
for i := 0; i < retries; i++ {
	err := rcache.Upsert[Person](context.Background(), c, "BillyBob", p, cb)
	if rcache.IsRetryable(err) {
		continue
	}
	// do something useful ...
	break
}

func UpsertTTL added in v0.5.0 

func UpsertTTL[T any](ctx context.Context, c *Cache, key string, val T, cb UpsertCallback[T], ttl time.Duration) error

UpsertTTL retrieves the existing value for a given key and invokes the UpsertCallback. The UpsertCallback function is responsible for determining the value to be stored. The value returned from the UpsertCallback is what is set in Redis.

Upsert allows for atomic updates of existing records, or simply inserting new entries when the key doesn't exist.

Redis uses an optimistic locking model. If the key changes during the transaction Redis will fail the transaction and return an error. However, these errors are retryable. To determine if the error is retryable use the IsRetryable function with the returned error. 

cb := rcache.UpsertCallback[Person](func(found bool, oldValue Person, newValue Person) Person {
	fmt.Println(found)
	fmt.Println(oldValue)
	fmt.Println(newValue)
	return newValue
})
retries := 3
for i := 0; i < retries; i++ {
	err := rcache.UpsertTTL[Person](context.Background(), c, "BillyBob", p, cb, time.Minute * 1)
	if rcache.IsRetryable(err) {
		continue
	}
	// do something useful ...
	break
}

Types

type Cache 

type Cache struct {
	// contains filtered or unexported fields
}

Cache is a simple type that provides basic caching functionality: store, retrieve, and delete. It is backed by Redis and supports storing entries with a TTL.

The zero-value is not usable, and this type should be instantiated using the NewCache function.

func NewCache 

func NewCache(redis RedisClient, opts ...Option) *Cache

NewCache creates and initializes a new Cache instance.

By default, msgpack is used for marshalling and unmarshalling the entry values. The behavior of Cache can be configured by passing Options.

func (*Cache) Delete 

func (c *Cache) Delete(ctx context.Context, keys ...string) error

Delete removes entries from the cache for a given set of keys.

func (*Cache) Flush added in v0.6.0 

func (c *Cache) Flush(ctx context.Context)

Flush flushes the cache deleting all keys/entries.

func (*Cache) FlushAsync added in v0.6.0 

func (c *Cache) FlushAsync(ctx context.Context)

FlushAsync flushes the cache deleting all keys/entries asynchronously. Only keys that were present when FLUSH ASYNC command was received by Redis will be deleted. Any keys created during asynchronous flush will be unaffected.

func (*Cache) Get 

func (c *Cache) Get(ctx context.Context, key string, v any) error

Get retrieves an entry from the Cache for the given key, and if found will unmarshall the value into the provided type.

If the key does not exist ErrKeyNotFound will be returned as the error value. A non-nil error value will be returned if the operation on the backing Redis fails, or if the value cannot be unmarshalled into the target type.

func (*Cache) Keys added in v0.6.0 

func (c *Cache) Keys(ctx context.Context) ([]string, error)

Keys retrieves all the keys in Redis/Cache

func (*Cache) Set 

func (c *Cache) Set(ctx context.Context, key string, v any) error

Set adds an entry into the cache, or overwrites an entry if the key already existed. The entry is set without an expiration.

func (*Cache) SetIfAbsent added in v0.3.0 

func (c *Cache) SetIfAbsent(ctx context.Context, key string, v any, ttl time.Duration) (bool, error)

SetIfAbsent adds an entry into the cache only if the key doesn't already exist. The entry is set with the provided TTL and automatically removed from the cache once the TTL is expired.

func (*Cache) SetIfPresent added in v0.3.0 

func (c *Cache) SetIfPresent(ctx context.Context, key string, v any, ttl time.Duration) (bool, error)

SetIfPresent updates an entry into the cache if they key already exists in the cache. The entry is set with the provided TTL and automatically removed from the cache once the TTL is expired.

func (*Cache) SetTTL added in v0.3.0 

func (c *Cache) SetTTL(ctx context.Context, key string, v any, ttl time.Duration) error

SetTTL adds an entry into the cache, or overwrites an entry if the key already existed. The entry is set with the provided TTL and automatically removed from the cache once the TTL is expired.

type Codec added in v1.0.0 

type Codec interface {
	Flate(data []byte) ([]byte, error)
	Deflate(data []byte) ([]byte, error)
}

Codec is an interface type that defines the behavior for compressing and decompressing data.

func NewCodec added in v1.0.0 

func NewCodec(ct CodecType) Codec

NewCodec creates a default codec for the given CodecType. By default, the codecs favor compression over performance where applicable.

An invalid or unsupported CodecType will result in a panic.

type CodecType added in v1.0.0 

type CodecType uint8
const (
	None  CodecType = 0
	Flate CodecType = 1
	GZip  CodecType = 2
	LZ4   CodecType = 3
)

type Marshaller 

type Marshaller func(v any) ([]byte, error)

Marshaller is a function type that marshals the value of a cache entry for storage.

func DefaultMarshaller 

func DefaultMarshaller() Marshaller

DefaultMarshaller returns a Marshaller using msgpack to marshall values.

type MultiResult added in v0.2.0 

type MultiResult[T any] map[string]T

MultiResult is a type representing returning multiple entries from the Cache.

func MGet added in v0.2.0 

func MGet[R any](ctx context.Context, c *Cache, keys ...string) (MultiResult[R], error)

MGet uses the provided Cache to retrieve multiple keys from Redis and returns a MultiResult.

func (MultiResult[T]) Get added in v0.2.0 

func (mr MultiResult[T]) Get(key string) (T, bool)

Get returns the value and a boolean indicating if the key exists. If the key doesn't exist the value will be the default zero value.

func (MultiResult[T]) IsEmpty added in v0.2.0 

func (mr MultiResult[T]) IsEmpty() bool

IsEmpty returns a boolean indicating if the results are empty.

func (MultiResult[T]) Keys added in v0.2.0 

func (mr MultiResult[T]) Keys() []string

Keys returns all the keys found.

func (MultiResult[T]) Values added in v0.2.0 

func (mr MultiResult[T]) Values() []T

Values returns all the values found.

type Option 

type Option func(c *Cache)

Option allows for the Cache behavior/configuration to be customized.

func Compression added in v1.0.0 

func Compression(codec Codec) Option

Compression allows for the values to be flated and deflated to conserve bandwidth and memory at the cost of higher CPU time.

func Serialization 

func Serialization(mar Marshaller, unmar Unmarshaller) Option

Serialization allows for the marshalling and unmarshalling behavior to be customized for the Cache.

A valid Marshaller and Unmarshaller must be provided. Providing nil for either will immediately panic.

type RedisClient added in v0.3.0 

type RedisClient interface {
	Get(ctx context.Context, key string) *redis.StringCmd
	MGet(ctx context.Context, keys ...string) *redis.SliceCmd
	Set(ctx context.Context, key string, val any, ttl time.Duration) *redis.StatusCmd
	SetNX(ctx context.Context, key string, value interface{}, expiration time.Duration) *redis.BoolCmd
	SetXX(ctx context.Context, key string, value interface{}, expiration time.Duration) *redis.BoolCmd
	Del(ctx context.Context, keys ...string) *redis.IntCmd
	Watch(ctx context.Context, fn func(*redis.Tx) error, keys ...string) error
	Scan(ctx context.Context, cursor uint64, match string, count int64) *redis.ScanCmd
	FlushDB(ctx context.Context) *redis.StatusCmd
	FlushDBAsync(ctx context.Context) *redis.StatusCmd
}

RedisClient is an interface type that defines the Redis functionality this package requires to use Redis as a cache.

type RetryableError added in v0.5.0 

type RetryableError struct {
	// contains filtered or unexported fields
}

func (RetryableError) Error added in v0.5.0 

func (e RetryableError) Error() string

func (RetryableError) IsRetryable added in v0.5.0 

func (e RetryableError) IsRetryable() bool

type Unmarshaller 

type Unmarshaller func(b []byte, v any) error

Unmarshaller is a function type that unmarshalls the value retrieved from the cache into the target type.

func DefaultUnmarshaller 

func DefaultUnmarshaller() Unmarshaller

DefaultUnmarshaller returns a Unmarshaller using msgpack to unmarshall values.

type UpsertCallback added in v0.5.0 

type UpsertCallback[T any] func(found bool, oldValue T, newValue T) T

UpsertCallback is a callback function that is invoked by Upsert. An UpsertCallback is passed if a key was found, the old value (or zero-value if the key wasn't found) and the new value. An UpsertCallback is responsible for determining what value should be set for a given key in the cache. The value returned from UpsertCallback is the value set.

Source Files

View all Source files

Directories

Path Synopsis
compression
flate
gzip
lz4
examples
compression
simple

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.