cachego

README

📜 cachego

cachego 是一个拥有高性能分段锁机制的轻量级内存缓存，拥有懒清理和哨兵清理两种清理机制，可以应用于所有的 GoLang 应用程序中。

目前已经在多个线上服务中运行良好，也抵御过最高 17w/s qps 的冲击，可以稳定使用！

我正在开发 v0.3.x 版本，这将在 API 以及功能上达到全新的使用体验，敬请期待，也期待大家的建议！！！

Read me in English.

🕹 功能特性

• 以键值对形式缓存数据，极简的 API 设计风格
• 引入 option function 模式，可定制化各种操作的过程
• 使用粒度更细的分段锁机制进行设计，具有非常高的并发性能
• 支持懒清理机制，每一次访问的时候判断是否过期
• 支持哨兵清理机制，每隔一定的时间间隔进行清理
• 自带 singleflight 机制，减少缓存穿透的伤害
• ....

更多功能请参考 _examples。架构设计请参考 arch.md 文档。

历史版本的特性请查看 HISTORY.md。未来版本的新特性和计划请查看 FUTURE.md。

🚀 安装方式

$ go get -u github.com/FishGoddess/cachego

💡 参考案例

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/FishGoddess/cachego"
	"github.com/FishGoddess/cachego/pkg/task"
)

func main() {
	// Create a cache for use.
	// We use option function to customize the creation of cache.
	// WithAutoGC means it will do gc automatically.
	cache := cachego.NewCache(cachego.WithAutoGC(10 * time.Minute))

	// Set a new entry to cache.
	// Both of them are set a key-value with no ttl.
	//cache.Set("key", 666, cachego.WithSetNoTTL())
	cache.Set("key", 666)

	// Get returns the value of this key.
	v, err := cache.Get("key")
	fmt.Println(v, err) // Output: 666 <nil>

	// If you pass a not existed key to of method, nil and errNotFound will be returned.
	v, err = cache.Get("not existed key")
	if cachego.IsNotFound(err) {
		fmt.Println(v, err) // Output: <nil> cachego: key not found
	}

	// SetWithTTL sets an entry with expired time.
	// See more information in example of ttl.
	cache.Set("ttlKey", 123, cachego.WithOpTTL(10*time.Second))

	// Also, you can get value from cache first, then load it to cache if missed.
	// onMissed is usually used to get data from db or somewhere, so you can refresh the value in cache.
	// Notice ctx in onMissed is passed by Get option.
	onMissed := func(ctx context.Context) (data interface{}, err error) {
		return "newValue", nil
	}

	v, err = cache.Get("newKey", cachego.WithOpOnMissed(onMissed), cachego.WithOpTTL(3*time.Second))
	fmt.Println(v, err) // Output: newValue <nil>

	// We provide a way to set data to cache automatically, so you can access some hottest data extremely fast.
	// See pkg/task/Task.
	t := task.Task{
		Before: func(ctx context.Context) {
			cache.Set("before", "value")
		},
		Fn: func(ctx context.Context) {
			cache.Set("fn", "value")
		},
		After: func(ctx context.Context) {
			cache.Set("after", "value")
		},
	}

	// Run this task automatically every second.
	go t.Run(context.Background(), time.Second)
	time.Sleep(5 * time.Second)
}

更多使用案例请查看 _examples 目录。

🔥 性能测试

测试文件：_examples/performance_test.go

$ go test -v ./_examples/performance_test.go

总缓存数据为 100w 条，并发数为 10w，循环测试写入和读取次数为 50 次

测试环境：R7-5800X CPU @ 3.8GHZ GHZ，32 GB RAM

测试	读取消耗时间 (越小越好)	写入消耗时间 (越小越好)	混合操作消耗时间 (越小越好)
cachego	945ms	942ms	941ms
go-cache	965ms	3251ms	4390ms
freeCache	935ms	994ms	1012ms
ECache	931ms	1068ms	1071ms

可以看出，由于使用了分段锁机制，读写性能在并发下依然非常高，但是分段锁会多一次定位的操作，如果加锁的消耗小于定位的消耗，那分段锁就不占优势。 这也是为什么 cachego 在写入性能上比 go-cache 强一大截，但是读取性能却没强多少的原因。后续会着重优化读取性能！

👥 贡献者

• cristiane：提供 hash 算法的优化建议
• hzy15610046011：提供架构设计文档和图片
• chen661：提供 segmentSize 设置选项的参数限制想法

如果您觉得 cachego 缺少您需要的功能，请不要犹豫，马上参与进来，发起一个 issue。

最后，我想感谢 JetBrains 公司的 free JetBrains Open Source license(s)，因为 cachego 是用该计划下的 Idea / GoLand 完成开发的。

Documentation

Overview

Package cachego provides an easy way to use foundation for your caching operations.

1. The basic usage:

// Create a cache for use.
// We use option function to customize the creation of cache.
// WithAutoGC means it will do gc automatically.
cache := cachego.NewCache(cachego.WithAutoGC(10 * time.Minute))

// Set a new entry to cache.
// Both of them are set a key-value with no ttl.
//cache.Set("key", 666, cachego.WithSetNoTTL())
cache.Set("key", 666)

// Get returns the value of this key.
v, err := cache.Get("key")
fmt.Println(v, err) // Output: 666 <nil>

// If you pass a not existed key to of method, nil and errNotFound will be returned.
v, err = cache.Get("not existed key")
if cachego.IsNotFound(err) {
	fmt.Println(v, err) // Output: <nil> cachego: key not found
}

// SetWithTTL sets an entry with expired time.
// See more information in example of ttl.
cache.Set("ttlKey", 123, cachego.WithOpTTL(10*time.Second))

// Also, you can get value from cache first, then load it to cache if missed.
// onMissed is usually used to get data from db or somewhere, so you can refresh the value in cache.
// Notice ctx in onMissed is passed by Get option.
onMissed := func(ctx context.Context) (data interface{}, err error) {
	return "newValue", nil
}

v, err = cache.Get("newKey", cachego.WithOpOnMissed(onMissed), cachego.WithOpTTL(3*time.Second))
fmt.Println(v, err) // Output: newValue <nil>

// We provide a way to set data to cache automatically, so you can access some hottest data extremely fast.
// See pkg/task/Task.
t := task.Task{
	Before: func(ctx context.Context) {
		cache.Set("before", "value")
	},
	Fn: func(ctx context.Context) {
		cache.Set("fn", "value")
	},
	After: func(ctx context.Context) {
		cache.Set("after", "value")
	},
}

// Run this task automatically every second.
go t.Run(context.Background(), time.Second)
time.Sleep(5 * time.Second)

2. The ttl usage:

// Create a cache and set an entry to cache.
cache := cachego.NewCache()
cache.Set("key", "value", cachego.WithOpTTL(3*time.Second))

// Check if the key is alive.
value, err := cache.Get("key")
fmt.Println(value, err) // Output: value <nil>

// Wait for 5 seconds and check again.
// Now the key is gone.
time.Sleep(5 * time.Second)
value, err = cache.Get("key")
fmt.Println(value, err) // Output: <nil> cachego: key not found

// However, the key is still in cache, and you should remove it by Delete() or DeleteAll().
// So, we provide an automatic way to remove those who are dead. See more information in example of gc.
cache.AutoGC(10 * time.Minute)

3. The gc usage:

// Create a cache and set an entry to cache.
cache := cachego.NewCache()
cache.Set("key", "value", cachego.WithOpTTL(1*time.Second))

value, err := cache.Get("key")
fmt.Println(value, err) // Output: value <nil>

// Wait for 2 seconds and check the key.
time.Sleep(2 * time.Second)

// We can see this key is gone, and we can't get it anymore.
value, err = cache.Get("key")
fmt.Println(value, err) // Output: <nil> cachego: key not found

// However, the key still stores in cache and occupies the space.
size := cache.Size()
fmt.Println(size) // Output: 1

// We should call GC() to clean up these dead entries.
// Notice that this method will take some CPU time to finish this task.
cache.GC()
size = cache.Size()
fmt.Println(size) // Output: 0

// Also, we provide an automatic way to do this job at fixed duration.
// It returns a channel which can be used to stop this automatic job.
// If you want to stop it, just send an true or false to the chan!
stopAutoGc := cache.AutoGC(10 * time.Minute)
stopAutoGc <- struct{}{}

4. The option usage:

// We use option function to customize the creation of cache.
// You can just new one without options.
cache := cachego.NewCache()
cache.Set("key", "value")

// You can set it to a cache with automatic gc if you want
//  Try WithAutoGC.
cache = cachego.NewCache(cachego.WithAutoGC(10 * time.Minute))

// Also, you can add more than one option to cache.
cache = cachego.NewCache(cachego.WithAutoGC(10*time.Minute), cachego.WithMapSize(64), cachego.WithSegmentSize(4096))

// Remember, some operations have their options, here is one example:
cache.Get("key", cachego.WithOpOnMissed(func(ctx context.Context) (data interface{}, err error) {
	return "value", nil
}))

5. The singleflight usage:

// In default, cachego enables single-flight mode in get operations.
// Just use WithOpOnMissed option to enjoy the flight of data.
cache := cachego.NewCache()

var wg sync.WaitGroup
for i := 0; i < 10; i++ {
	wg.Add(1)
	go func() {
		defer wg.Done()

		cache.Get("key1", cachego.WithOpOnMissed(func(ctx context.Context) (data interface{}, err error) {
			time.Sleep(30 * time.Millisecond) // Assume I/O costs 30ms
			fmt.Println("key1: single-flight")
			return 123, nil
		}))
	}()
}
wg.Wait()

// If you want to disable single-flight mode in some Get operations, try this:
wg = sync.WaitGroup{}
for i := 0; i < 10; i++ {
	wg.Add(1)
	go func() {
		defer wg.Done()

		cache.Get("key2", cachego.WithOpOnMissed(func(ctx context.Context) (data interface{}, err error) {
			time.Sleep(30 * time.Millisecond) // Assume I/O costs 30ms
			fmt.Println("key2: multi-flight")
			return 456, nil
		}), cachego.WithOpDisableSingleflight())
	}()
}
wg.Wait()

// Of course, we all know single-flight mode will decrease the success rate of loading data.
// So you can disable it globally if you need.
cache = cachego.NewCache(cachego.WithDisableSingleflight())

wg = sync.WaitGroup{}
for i := 0; i < 10; i++ {
	wg.Add(1)
	go func() {
		defer wg.Done()

		cache.Get("key3", cachego.WithOpOnMissed(func(ctx context.Context) (data interface{}, err error) {
			time.Sleep(30 * time.Millisecond) // Assume I/O costs 30ms
			fmt.Println("key3: multi-flight")
			return 666, nil
		}))
	}()
}
wg.Wait()

6. The task usage:

// We provide a task for you to do some loops.
t := task.Task{
	Before: func(ctx context.Context) {
		fmt.Println("Before...")
	},
	Fn: func(ctx context.Context) {
		fmt.Println("Fn...")
	},
	After: func(ctx context.Context) {
		fmt.Println("After...")
	},
}

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

// Run runs a task which is usually called in a new goroutine.
// go t.Run(ctx, time.Second)
t.Run(ctx, time.Second)

// You can use it to update your cache. Try this:
cache := cachego.NewCache()

t = task.Task{
	Before: func(ctx context.Context) {
		cache.Set("key", "before")
	},
	Fn: func(ctx context.Context) {
		cache.Set("key", strconv.FormatInt(rand.Int63n(100), 10))
	},
	After: func(ctx context.Context) {
		cache.Set("key", "after")
	},
}

ctx, cancel = context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()

go t.Run(ctx, time.Second)

// Simulate user requests
for i := 0; i < 22; i++ {
	fmt.Println(cache.Get("key"))
	time.Sleep(500 * time.Millisecond)
}

Index

• Constants
• func IsNotFound(err error) bool
• type Cache
  • func NewCache(opts ...Option) *Cache
  • func (c *Cache) AutoGC(duration time.Duration) chan<- struct{}
  • func (c *Cache) AutoSet()
  • func (c *Cache) Delete(key string)
  • func (c *Cache) DeleteAll()
  • func (c *Cache) GC()
  • func (c *Cache) Get(key string, opts ...OpOption) (interface{}, error)
  • func (c *Cache) Set(key string, value interface{}, opts ...OpOption)
  • func (c *Cache) Size() int
• type OpOption
  • func WithOpContext(ctx context.Context) OpOption
  • func WithOpDisableSingleflight() OpOption
  • func WithOpNoTTL() OpOption
  • func WithOpOnMissed(onMissed func(ctx context.Context) (data interface{}, err error)) OpOption
  • func WithOpTTL(ttl time.Duration) OpOption
• type Option
  • func WithAutoGC(d time.Duration) Option
  • func WithDisableSingleflight() Option
  • func WithMapSize(mapSize uint) Option
  • func WithSegmentSize(segmentSize uint) Option

Constants

const Version = "v0.3.3-alpha"

Version is the version string representation of cachego.

Functions

func IsNotFound

func IsNotFound(err error) bool

IsNotFound returns if this error is key not found.

Types

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Cache is a struct of cache.

func NewCache

func NewCache(opts ...Option) *Cache

NewCache returns a new Cache holder for use.

func (*Cache) AutoGC

func (c *Cache) AutoGC(duration time.Duration) chan<- struct{}

AutoGC starts a goroutine to execute GC() at fixed duration. It returns a channel which can be used to stop this goroutine.

func (*Cache) AutoSet

func (c *Cache) AutoSet()

AutoSet starts a goroutine to execute Set() at fixed duration. It returns a channel which can be used to stop this goroutine. See AutoSetOption. Deprecated: Use pkg/task/Task instead.

func (*Cache) Delete

func (c *Cache) Delete(key string)

Delete removes the value of key. If this key is not existed, nothing will happen.

func (*Cache) DeleteAll

func (c *Cache) DeleteAll()

DeleteAll removes all keys in cache. Notice that this method is weak-consistency.

func (*Cache) GC

func (c *Cache) GC()

GC removes dead entries in cache. Notice that this method is weak-consistency, and it doesn't guarantee 100% removed.

func (*Cache) Get

func (c *Cache) Get(key string, opts ...OpOption) (interface{}, error)

Get fetches value of key from cache first, and returns it if ok. Returns an NotFoundErr if this key is not found, and you can use IsNotFound to judge if this error is not found. Also, you can specify a function which will be called if missed, so you can load this entry to cache again. See OpOption.

func (*Cache) Set

func (c *Cache) Set(key string, value interface{}, opts ...OpOption)

Set sets key and value to cache. In default, this entry will not expire, so if you want it to expire, see SetOption.

func (*Cache) Size

func (c *Cache) Size() int

Size returns the size of cache. Notice that this method is weak-consistency.

type OpOption

type OpOption func(conf *opConfig)

OpOption is a function which initializes opConfig.

func WithOpContext

func WithOpContext(ctx context.Context) OpOption

WithOpContext sets context to ctx.

func WithOpDisableSingleflight

func WithOpDisableSingleflight() OpOption

WithOpDisableSingleflight sets the single-flight mode to false.

func WithOpNoTTL

func WithOpNoTTL() OpOption

WithOpNoTTL sets the ttl of missed key to no ttl.

func WithOpOnMissed

func WithOpOnMissed(onMissed func(ctx context.Context) (data interface{}, err error)) OpOption

WithOpOnMissed sets onMissed to Get operation.

func WithOpTTL

func WithOpTTL(ttl time.Duration) OpOption

WithOpTTL sets the ttl of missed key if loaded to ttl.

type Option

type Option func(conf *config)

Option is a function which initializes config.

func WithAutoGC

func WithAutoGC(d time.Duration) Option

WithAutoGC is an option turning on automatically gc.

func WithDisableSingleflight

func WithDisableSingleflight() Option

WithDisableSingleflight is an option disabling single-flight mode of cache.

func WithMapSize

func WithMapSize(mapSize uint) Option

WithMapSize is an option setting initializing map size of cache.

func WithSegmentSize

func WithSegmentSize(segmentSize uint) Option

WithSegmentSize is an option setting initializing segment size of cache. segmentSize must be the pow of 2 (such as 64) or the segments may be uneven.