gcache

README

gcache

缓存框架对比：https://cloud.tencent.com/developer/article/1967978

实现特性：

• 实现基于HTTP+protobuf的分布式缓存节点通信机制
• 使用一致性哈希算法解决Key路由和缓存雪崩问题
• 使用SingleFlight算法防止缓存击穿问题
• 实现缓存空值机制，解决缓存穿透问题
• 实现LRU缓存淘汰机制，避免内存无限增长
• 实现TTL机制，基于ZSet的惰性删除
• 实现基于ETCD的服务注册和发现，解决需要手动处理集群变化问题
• 实现远程HotKey的本地缓存机制，避免HotKey频繁网络请求带来的性能问题

待实现特性：

• 基于TCP的自定义协议通信伙伴节点通信，降低网络通信成本
• 实现近似LRU（可以参考freecache和redis的淘汰机制）
• 实现TinyLFU https://blog.csdn.net/l_dongyang/article/details/108583476

Documentation

Index

• type ByteView
  • func NewByteView(b []byte, expire time.Time) ByteView
  • func (v ByteView) ByteSlice() []byte
  • func (v ByteView) Expire() time.Time
  • func (v ByteView) Len() int
  • func (v ByteView) String() string
• type Getter
• type GetterFunc
  • func (f GetterFunc) Get(key string) (ByteView, error)
• type Group
  • func GetGroup(name string) *Group
  • func NewGroup(name string, cacheBytes int, getter Getter) *Group
  • func (g *Group) Get(key string) (ByteView, error)
  • func (g *Group) RegisterPeers(peers PeerPicker)
  • func (g *Group) Remove(key string) error
  • func (g *Group) SetEmptyWhenError(duration time.Duration)
  • func (g *Group) SetHotCache(cacheBytes int)
• type HTTPPool
  • func NewHTTPPool(self string) *HTTPPool
  • func (p *HTTPPool) GetAll() []PeerGetter
  • func (p *HTTPPool) Log(format string, v ...any)
  • func (p *HTTPPool) PickPeer(key string) (PeerGetter, bool)
  • func (p *HTTPPool) ServeHTTP(w http.ResponseWriter, r *http.Request)
  • func (p *HTTPPool) Set(peers ...string)
  • func (p *HTTPPool) SetETCDRegistry(ctx context.Context, etcdAddrs ...string) error
• type PeerGetter
• type PeerPicker

Types

type ByteView

type ByteView struct {
	// contains filtered or unexported fields
}

ByteView 一个不可变的字节数组视图

func NewByteView

func NewByteView(b []byte, expire time.Time) ByteView

func (ByteView) ByteSlice

func (v ByteView) ByteSlice() []byte

func (ByteView) Expire

func (v ByteView) Expire() time.Time

func (ByteView) Len

func (v ByteView) Len() int

func (ByteView) String

func (v ByteView) String() string

type Getter

type Getter interface {
	Get(key string) (ByteView, error)
}

Getter 用于加载数据

type GetterFunc

type GetterFunc func(key string) (ByteView, error)

func (GetterFunc) Get

func (f GetterFunc) Get(key string) (ByteView, error)

type Group

type Group struct {
	// contains filtered or unexported fields
}

Group 一个缓存命名空间

func GetGroup

func GetGroup(name string) *Group

GetGroup 从全局缓存获取Group

func NewGroup

func NewGroup(name string, cacheBytes int, getter Getter) *Group

NewGroup 创建一个Group

func (*Group) Get

func (g *Group) Get(key string) (ByteView, error)

Get 从缓存获取key对应的value

func (*Group) RegisterPeers

func (g *Group) RegisterPeers(peers PeerPicker)

RegisterPeers 注册获取远程节点请求客户端的PeerPicker

func (*Group) Remove

func (g *Group) Remove(key string) error

Remove 从缓存删除key

func (*Group) SetEmptyWhenError

func (g *Group) SetEmptyWhenError(duration time.Duration)

SetEmptyWhenError 当getter返回error时设置空值，缓解缓存穿透问题 为0表示该机制不生效

func (*Group) SetHotCache

func (g *Group) SetHotCache(cacheBytes int)

SetHotCache 设置远程节点Hot Key-Value的缓存，避免频繁请求远程节点

type HTTPPool

type HTTPPool struct {
	// contains filtered or unexported fields
}

HTTPPool 实现了伙伴节点

func NewHTTPPool

func NewHTTPPool(self string) *HTTPPool

NewHTTPPool 创建一个HTTPPool

func (*HTTPPool) GetAll

func (p *HTTPPool) GetAll() []PeerGetter

GetAll 获取的远程节点客户端

func (*HTTPPool) Log

func (p *HTTPPool) Log(format string, v ...any)

func (*HTTPPool) PickPeer

func (p *HTTPPool) PickPeer(key string) (PeerGetter, bool)

PickPeer 根据键获取对应的远程节点客户端

func (*HTTPPool) ServeHTTP

func (p *HTTPPool) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP 处理所有http请求

func (*HTTPPool) Set

func (p *HTTPPool) Set(peers ...string)

Set 更新同伴节点

func (*HTTPPool) SetETCDRegistry

func (p *HTTPPool) SetETCDRegistry(ctx context.Context, etcdAddrs ...string) error

SetETCDRegistry 设置etcd名字服务

type PeerGetter

type PeerGetter interface {
	Get(in *pb.Request, out *pb.Response) error
	Remove(in *pb.Request) error
}

PeerGetter 远程客户端，根据group和key获取缓存

type PeerPicker

type PeerPicker interface {
	PickPeer(key string) (PeerGetter, bool)
	GetAll() []PeerGetter
}

PeerPicker 用于获取远程节点的请求客户端