README
¶
Rotom
📃Introduction
Welcome to rotom, an embedded high-performance key-value in-memory database written in Go, has many built-in data types, support for persistence and data recovery.
Currently features:
- Built-in data types
string,map,set,list,zset, andbitmap. - Second level ttl supported for each key-value pair.
- Based on GigaCache, which is managing GB-level data, saving 50% memory compared to
stdmap, with better performance and reduced GC overhead. - Internal encoding/decoding lib that more effective than
protobuf. - Persistent log support, and can recover database from logs.
If you want to know more technical details, check out doc.
🚚Usage
Before using, please install rotom into your project first:
go get github.com/xgzlucario/rotom
And install the gofakeit library for generating some random data:
go get github.com/brianvoe/gofakeit/v6
Run the sample program:
package main
import (
"fmt"
"time"
"github.com/brianvoe/gofakeit/v6"
"github.com/xgzlucario/rotom"
)
func main() {
db, err := rotom.Open(rotom.DefaultOptions)
if err != nil {
panic(err)
}
defer db.Close()
// Set
for i := 0; i < 10000; i++ {
phone := gofakeit.Phone()
user := []byte(gofakeit.Username())
// Set bytes
db.Set(phone, user)
// Or set with ttl
db.SetEx(phone, user, time.Minute)
// Or set with deadline
db.SetTx(phone, user, time.Now().Add(time.Minute).UnixNano())
}
// Get
key := gofakeit.Phone()
user, ttl, ok := db.Get(key)
// ...
}
🚀Performance
Rotom has super multi-threading performance. The following is the bench test data.
Test Environment
goos: linux
goarch: amd64
pkg: github.com/xgzlucario/GigaCache
cpu: 13th Gen Intel(R) Core(TM) i5-13600KF
Benchmark
========== Set ==========
size: 100*10000 enties
desc: key 10 bytes, value 10 bytes
cost: 1.120466957s
qps: 892467.60
50th: 990 ns
90th: 1107 ns
99th: 1724 ns
========== SetEx ==========
size: 100*10000 enties
desc: key 10 bytes, value 10 bytes, ttl 1min
cost: 1.125406833s
qps: 888551.77
50th: 986 ns
90th: 1092 ns
99th: 2110 ns
========== Get ==========
size: 100*10000 enties
desc: key 10 bytes, value 10 bytes
cost: 255.023737ms
qps: 3920845.99
50th: 212 ns
90th: 267 ns
99th: 532 ns
========== Get 8 parallel ==========
size: 100*10000 enties
desc: key 10 bytes, value 10 bytes
cost: 138.089874ms
qps: 7240728.26
50th: 173 ns
90th: 261 ns
99th: 501 ns
========== RPush ==========
size: 100*10000 enties
desc: value 10 bytes
cost: 1.033844185s
qps: 967248.36
50th: 936 ns
90th: 1011 ns
99th: 1609 ns
========== HSet ==========
size: 100*10000 enties
desc: field 10 bytes, value 10 bytes
cost: 1.182939928s
qps: 845337.51
50th: 987 ns
90th: 1117 ns
99th: 1850 ns
========== HGet ==========
size: 100*10000 enties
desc: field 10 bytes, value 10 bytes
cost: 292.57105ms
qps: 3417634.25
50th: 220 ns
90th: 306 ns
99th: 575 ns
========== BitSet ==========
size: 100*10000 enties
desc: offset uint32
cost: 916.172391ms
qps: 1091477.02
50th: 833 ns
90th: 880 ns
99th: 1219 ns
========== ZSet ==========
size: 100*10000 enties
desc: field 10 bytes, incr float64
cost: 1.321257444s
qps: 756843.75
50th: 1121 ns
90th: 1248 ns
99th: 2025 ns
Documentation
¶
Overview ¶
Package rotom provides an in-memory key-value database.
Index ¶
- Variables
- type BitMap
- type Cmd
- type DB
- func (db *DB) BitAnd(dst string, src ...string) error
- func (db *DB) BitArray(key string) ([]uint32, error)
- func (db *DB) BitCount(key string) (uint64, error)
- func (db *DB) BitFlip(key string, start, end uint32) error
- func (db *DB) BitOr(dst string, src ...string) error
- func (db *DB) BitSet(key string, val bool, offsets ...uint32) (int, error)
- func (db *DB) BitTest(key string, offset uint32) (bool, error)
- func (db *DB) BitXor(dst string, src ...string) error
- func (db *DB) Close() error
- func (db *DB) GC()
- func (db *DB) Get(key string) ([]byte, int64, error)
- func (db *DB) GetOptions() Options
- func (db *DB) HGet(key, field string) ([]byte, error)
- func (db *DB) HKeys(key string) ([]string, error)
- func (db *DB) HLen(key string) (int, error)
- func (db *DB) HRemove(key string, fields ...string) (n int, err error)
- func (db *DB) HSet(key, field string, val []byte) error
- func (db *DB) Incr(key string, incr int64) (n int64, err error)
- func (db *DB) LIndex(key string, i int) (string, error)
- func (db *DB) LLen(key string) (int, error)
- func (db *DB) LPop(key string) (string, error)
- func (db *DB) LPush(key string, items ...string) error
- func (db *DB) LRange(key string, start, end int, f func(string) (stop bool)) error
- func (db *DB) LSet(key string, index int, item string) (bool, error)
- func (db *DB) Len() int
- func (db *DB) RPop(key string) (string, error)
- func (db *DB) RPush(key string, items ...string) error
- func (db *DB) Remove(keys ...string) (n int)
- func (db *DB) SAdd(key string, items ...string) (int, error)
- func (db *DB) SCard(key string) (int, error)
- func (db *DB) SDiff(dst string, src ...string) error
- func (db *DB) SHas(key string, items ...string) (bool, error)
- func (db *DB) SInter(dst string, src ...string) error
- func (db *DB) SMembers(key string) ([]string, error)
- func (db *DB) SRemove(key string, items ...string) error
- func (db *DB) SUnion(dst string, src ...string) error
- func (db *DB) Scan(f func([]byte, []byte, int64) bool)
- func (db *DB) Set(key string, val []byte)
- func (db *DB) SetEx(key string, val []byte, ttl time.Duration)
- func (db *DB) SetTTL(key string, ts int64) bool
- func (db *DB) SetTx(key string, val []byte, ts int64)
- func (db *DB) Shrink() error
- func (db *DB) Sync() error
- func (db *DB) ZAdd(zset, key string, score float64) error
- func (db *DB) ZCard(zset string) (int, error)
- func (db *DB) ZGet(zset, key string) (float64, error)
- func (db *DB) ZIncr(zset, key string, incr float64) (float64, error)
- func (db *DB) ZIter(zset string, f func(string, float64) bool) error
- func (db *DB) ZRemove(zset string, key string) error
- type List
- type Map
- type Operation
- type Options
- type Set
- type String
- type SyncPolicy
- type Type
- type ZSet
Constants ¶
This section is empty.
Variables ¶
View Source
var ( ErrKeyNotFound = errors.New("key not found") ErrFieldNotFound = errors.New("field not found") ErrTypeAssert = errors.New("type assert error") ErrOutOfBounds = errors.New("index out of bounds") ErrWrongType = errors.New("wrong data type") ErrWrongBitValue = errors.New("wrong bit value") ErrUnSupportDataType = errors.New("unsupport data type") ErrUnknownOperationType = errors.New("unknown operation type") ErrNotString = errors.New("value is not string") ErrNotNumberic = errors.New("value is not numberic") ErrInvalidArgs = errors.New("invalid args") ErrInvalidResponse = errors.New("invalid response") ErrDatabaseClosed = errors.New("database closed") ErrUnSupportOperation = errors.New("unsupport operation") ErrIndexOutOfRange = errors.New("index out of range") ErrEmptyList = errors.New("list is empty") ErrDatabaseIsUsing = errors.New("database is using") ErrShrinkRunning = errors.New("shrink is running") )
View Source
var ( DefaultOptions = Options{ DirPath: "rotom", ShardCount: 1024, SyncPolicy: EverySecond, ShrinkCronExpr: "0 0 0/1 * * ?", RunSkipLoadError: true, } )
Functions ¶
This section is empty.
Types ¶
type DB ¶ added in v1.7.3
type DB struct {
// contains filtered or unexported fields
}
DB represents a rotom database.
func (*DB) GC ¶ added in v1.7.3
func (db *DB) GC()
GC triggers the garbage collection to evict expired kv datas.
type Options ¶ added in v1.7.3
type Options struct {
// Dir path if the db storage path.
DirPath string
// ShardCount is the shard numbers for underlying hashmap.
ShardCount uint32
// SyncPolicy is whether to synchronize writes to disk.
// Setting `Sync` is required for durability of a single write operation, but also results in slower writes.
// Setting `EverySecond` is much faster, but less durable as it relies on the OS to flush the writes to disk.
SyncPolicy SyncPolicy
// ShrinkCronExpr sauto shrink will be triggered when cron expr is satisfied.
// cron expression follows the standard cron expression.
// e.g. "0 0 * * *" means merge at 00:00:00 every day.
// Setting empty string "" will disable auto shrink.
ShrinkCronExpr string
// Skip error when loading db file when startup.
RunSkipLoadError bool
}
Options represents the configuration for rotom.
type SyncPolicy ¶ added in v1.7.3
type SyncPolicy byte
SyncPolicy represents how often data is synced to disk.
const ( EverySecond SyncPolicy = iota Sync )
Source Files
Directories
Click to show internal directories.
Click to hide internal directories.