README
¶
gofw-cache
It is a distributed caching framework based on a simple implementation of Go, and is only a beginner's project. 中文
TODO
- Only data of type key-value strings is supported
- LRU cache cull policy
- LFU cache cull policy
- Single cache
- Consistent hash algorithm
- Distributed cache
- Realize GET/SET/DELETE three operations
- Cache expiration handling
- INI configuration information
- Cluster dynamic expansion and shrinkage
- Data persistence
- Deploy Docker image Document introduction
- Support cache grouping with cacheName as the group name Document introduction
API DOCUMENT
PROCESS
curl http://[ip:port]/_cache/[cacheName]/[_set or _get or _del]/[key]/[value]
[ip:port] --> Host IP and port
[cacheName] --> Cache name (default is fwcache)
[_set or _get or _del] --> operation
[key] --> key
[value] --> value
SET
curl http://localhost:8000/_cache/fwcache/_set/test/test/[ttl]
ttl：Expiration time, the default unit is seconds, optional
It is to find the data key through consistent hash to the corresponding host, and then the data key-value is stored in the host.
GET
curl http://localhost:8000/_cache/fwcache/_get/test
It is to find the data key through consistent hash to the corresponding host, and then read the data key-value from the host.
DELETE
curl http://localhost:8000/_cache/fwcache/_del/test
It is to find the data key through the consistent hash to the corresponding host, and then delete the data key-value from the host.
EXAMPLE
Multiple cluster deployment
Run cacheServerCluster.go in the cmd directory
package main
import (
"gitee.com/wudongdongfw/gofw-cache/internal/cache"
"gitee.com/wudongdongfw/gofw-cache/internal/conf"
"gitee.com/wudongdongfw/gofw-cache/internal/remote"
"log"
"net/http"
"strings"
)
// Cluster deployment
func main() {
// init cache config
cacheName, capacity := conf.InitCacheConfig(conf.CACHE_CLUSTER)
c := cache.NewCache(cacheName, capacity)
// init http config
selfAddr, clusterAddr := conf.InitHttpConfig(conf.HTTP_CLUSTER)
nodes := remote.NewHttpPool(selfAddr)
nodes.SetClusterAddr(clusterAddr.([]string)...)
// register nodes
c.RegisterNodes(nodes)
log.Println("gofw-cache is runnning...", selfAddr)
log.Fatal(http.ListenAndServe(":"+strings.Split(selfAddr, ":")[1], nodes))
}
The configuration information is imported through config/cluster.ini
[cache]
# cache name
name = fwcache
# cache capacity
capacity = 10
[http]
# deployment ip:port
selfAddr = 127.0.0.1:8000
# cluster ip:port
clusterAddr = 127.0.0.1:8000,127.0.0.1:8001,127.0.0.1:8002,127.0.0.1:8003
Configure the cache name (id), capacity, cluster address, and native address.
Start the 4 hosts, respectively selfAddr respectively modified into 127.0.0.1:8000,127.0.0.1:8001,127.0.0.1:8002, 127.0.0.1:8003. We can choose any host as the API host. For example, execute:
curl http://localhost:8000/_cache/fwcache/_set/test4/test
# http://localhost:8000 result:
2021/02/08 11:34:48 gofw-cache is runnning... localhost:8000
2021/02/08 11:34:48 Receive remote: GET /_cache/fwcache/_set/test4/test
2021/02/08 11:34:48 key:[test4] in the node:[http://localhost:8003]
2021/02/08 11:34:48 set key value success
# http://localhost:8003 result:
2021/02/08 11:34:48 gofw-cache is runnning... localhost:8003
2021/02/08 11:34:48 Receive remote: GET /_cache/fwcache/_set/test4/test
2021/02/08 11:34:48 set key value success
It is obvious that the 8000 port host is located to the 8003 port host through the consistent hashing algorithm, and the cache is added to the 8003 port host.
Access port 8002 host again find Key is the value of "test4"：
curl http://localhost:8002/_cache/fwcache/_get/test4
# http://localhost:8002 result:
2021/02/08 11:38:22 gofw-cache is runnning... localhost:8002
2021/02/08 13:00:05 Receive remote: GET /_cache/fwcache/_get/test4
2021/02/08 13:00:05 key:[test4] in the node:[http://localhost:8003]
2021/02/08 13:00:05 get key value success
# http://localhost:8003 result:
2021/02/08 11:38:10 gofw-cache is runnning... localhost:8003
2021/02/08 11:41:41 Receive remote: GET /_cache/fwcache/_get/test4
2021/02/08 11:41:41 set key value success
Finally delete "test4 "data through port 8001:
curl http://localhost:8001/_cache/fwcache/_del/test4
# http://localhost:8001 result:
2021/02/08 11:38:31 gofw-cache is runnning... localhost:8001
2021/02/08 13:07:31 Receive remote: GET /_cache/fwcache/_del/test4
2021/02/08 13:07:31 key:[test4] in the node:[http://localhost:8003]
2021/02/08 13:07:31 delete key value success
# http://localhost:8003 result:
2021/02/08 13:07:31 Receive remote: GET /_cache/fwcache/_del/test4
2021/02/08 13:07:31 delete key value success
Single cluster deployment
Run cacheServerSingle.go in the cmd directory
Default configuration file:
[cache]
# 缓存名
name = fwcache
# 缓存最大容量(条数)
capacity = 10
[http]
# 部署ip:host
selfAddr = 127.0.0.1:5252
# 集群ip:host
clusterAddr = 127.0.0.1:5252
TEST
Performance tests, test files：
example/gofwcache_test.go
$ go build -v ./example/gofwcache_test.go
The total cache data is 10,000 pieces, the number of concurrency is 1,000, and the number of loop test writes and reads is 100 times
| Test | Write elapsed time | Read elapsed time | Read and write operations consume time |
|---|---|---|---|
| gofw-cache | 8273ms | 1035ms | 1989ms |
PERSISTENCE
By default, cache persistence is carried out at zero every day. The save directory is: './db/xxx. FDB ', with the timestamp as the file name.
Format:
{"fwcache": xxxxxx}
{"wddcache": xxxxxx}
xxxxxx is the serialized value of
map[string]* list.element
Directories