kvlog

README

Overview

kvlog is a key-value store with history, so that it can store multiple values for the same key over time. It's intended to store data that only changes relatively slowly (e.g. tens or hundreds of values per key), not data that changes rapidly (like in a time series database, with thousands or millions of data points per key).

Conceptually it changes from the simple key-value store (k,v) tuple to include a timestamp component i.e. we have a (k,ts,v) tuple, indexed on both k and ts.

Thus a kvlog can perform traditional key-value operations i.e.

• set key=value
• get key

plus additional operations that include a time component e.g.

• get key at this time
• get all values of key (ordered by time, after this time, etc.)
• set key=value at time T

Status

Experimental. API is unstable.

Copyright and Licence

Copyright 2021 Gavin Carr gavin@openfusion.net.

This project is licensed under the terms of the MIT licence.

Documentation

Index

• type Iterator
  • func (it *Iterator) Close()
  • func (it *Iterator) Err() error
  • func (it *Iterator) Next() *KVLog
• type KDB
  • func NewKDB(ctx context.Context) (*KDB, error)
  • func NewKDBOptions(ctx context.Context, opts KDBOptions) (*KDB, error)
  • func (kdb *KDB) Disconnect()
  • func (kdb *KDB) Get(key string) (string, error)
  • func (kdb *KDB) GetAt(key string, ts int64) (string, error)
  • func (kdb *KDB) GetIterator(key string) (*Iterator, error)
  • func (kdb *KDB) Set(key, val string) error
• type KDBOptions
• type KVLog
• type Value

Types

type Iterator

type Iterator struct {
	// contains filtered or unexported fields
}

func (*Iterator) Close

func (it *Iterator) Close()

Close marks the iterator as closed. Next() should not be called again after the iterator has been closed.

func (*Iterator) Err

func (it *Iterator) Err() error

Err returns the most recent error received from the iterator

func (*Iterator) Next

func (it *Iterator) Next() *KVLog

Next returns the next KVLog record from the iterator, or nil if no records remain, or an error occurred (which will be available via Iterator.Err()).

type KDB

type KDB struct {
	// contains filtered or unexported fields
}

func NewKDB

func NewKDB(ctx context.Context) (*KDB, error)

NewKDB creates a new connection to the kvlog database using the ctx context and default options, and returns *KDB. The caller is responsible for doing a Disconnect(ctx) on *KDB.client when completed.

func NewKDBOptions

func NewKDBOptions(ctx context.Context, opts KDBOptions) (*KDB, error)

NewKDBOptions creates a new connection to the kvlog database using the ctx context and the given options, and returns *KDB. The caller is responsible for calling KDB.Disconnect when completed.

func (*KDB) Disconnect

func (kdb *KDB) Disconnect()

func (*KDB) Get

func (kdb *KDB) Get(key string) (string, error)

Get fetches the latest value for key

func (*KDB) GetAt

func (kdb *KDB) GetAt(key string, ts int64) (string, error)

GetAt fetches the first value for key after ts

func (*KDB) GetIterator

func (kdb *KDB) GetIterator(key string) (*Iterator, error)

GetIterator returns an Interator to fetch successive KVLog records, in reverse timestamp order (i.e. latest first). The caller is responsible for calling Close() on the returned iterator once finished.

func (*KDB) Set

func (kdb *KDB) Set(key, val string) error

Set sets the current value for key to val (if not already val)

type KDBOptions

type KDBOptions struct {
	URI    string
	DBName string
}

First-pass implementation: mongodb

type KVLog

type KVLog struct {
	Key string `bson:"k"`
	TS  int64  `bson:"ts"`
	Val string `bson:"v"`
	Vid string `bson:"vid"`
}

type Value

type Value struct {
	ID  string `bson:"_id"`
	Val string `bson:"v"`
}