edb

package module
v0.3.12 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 6, 2024 License: MIT Imports: 25 Imported by: 2

README

Go Embedded Database — in-process document database using Bolt (Badger soon)

Go reference Go Report Card

Why?

A single server in 202x can handle almost any load almost any small-to-mid-sized company has. This means you can massively save on development time, costs and complexity by going back from the cloud to a dedicated unicorn server.

This won't be the right choice every time; there's availability and security boundaries to consider, too. However, a lot of companies could benefit from faster time-to-market and smaller development teams, and it is very often the right choice for startups in particular.

To realize those benefits, you need an efficient database:

  • Very fast data access means that, instead of optimizing a tricky SQL, you can just loop over your data with code. Imagine the savings.
  • Serialized writes simplify statistics & similar, eliminate a whole class of errors, and greatly simplify the code.

To access data fast, you want that data to already be a part of your process memory. In-process key-value datastores do that quickly an efficiently. We're using Bolt for now, but its maintenance story leaves a lot to be desired, so we'll be switching to Badger soon.

We use msgpack for encoding structs currently. More options are possible, but not available right now.

Usage

Install:

go get github.com/andreyvit/edb

Saves ordinary structs in the database, the first field is the primary key (use another struct for a composite key) and must be marked as msgpack:"-" to avoid storing it as part of the value:

type Post struct {
    ID        string    `msgpack:"-"`
    Time      time.Time `msgpack:"tm"`
    Author    string    `msgpack:"a"`
    Content   string    `msgpack:"c"`
    Published bool      `msgpack:"pub"`
}

Define schema:

var (
    mySchema = edb.NewSchema(edb.SchemaOpts{})
    postsTable = edb.AddTable[Post](mySchema, "posts", 1, nil, nil, nil)
)

Those nils are: indexer func, migration func, a list of indices, all optional. Let's add a couple of indices:

var (
    postsTable = edb.AddTable(mySchema, "posts", 1, func (post *Post, ib *edb.IndexBuilder) {
        if post.Author != "" {
            ib.Add(postsByAuthor, post.Author)
        }
        if post.Published {
            ib.Add(publishedPostsByTime, post.Time)
        }
    }, nil, []*edb.Index{
        postsByAuthor,
        publishedPostsByTime,
    })
    postsByAuthor = AddIndex[string]("by_author")
    publishedPostsByTime = AddIndex[time.Time]("published_by_time")
)

Open a db:

db := must(edb.Open(filePath, mySchema, edb.Options{}))

Save a post:

post := &Post{
    ID:        "123", // use UUID generator here, or Snowflake IDs, or something
    Time:      time.Now(),
    Author:    "alice",
    Content:   "This is my first post.",
    Published: true,
}
ensure(db.Tx(true, func(tx *db.Tx) error {
    edb.Put(tx, post) // no error possible, all errors are only returned when committing a tx
    return nil
}))

All operations must be performed inside a transaction. Those can be read-only (db.Tx(false, ...)) or mutable (db.Tx(true, ...)). We'll assume you have a transaction going in the code below.

Find a post by ID:

post := edb.Get[Post](tx, "123")
if post == nil {
    log.Printf("not found")
} else {
    log.Printf("post = %v", *post)
}

Find posts by author name:

for c := edb.ExactIndexScan[Post](tx, postsByAuthor, "alice"); c.Next(); {
    post := c.Row()
    log.Printf("found: %v", *post)
}

ExactIndexScan is a helper that combines IndexScan with ExactScan option. To find posts by time, scanning backwards, we'll have to use these lower-level tools:

for c := edb.IndexScan[Post](tx, postsByAuthor, edb.ExactScan("alice").Reversed()); c.Next(); {
    post := c.Row()
    log.Printf("found: %v", *post)
}

You can scan the entire table:

for c := edb.TableScan[Post](tx, edb.FullScan()); c.Next(); {
    post := c.Row()
    log.Printf("found: %v", *post)
}

Use edb.All to obtain a slice of all rows from a cursor (noting, of course, that this might use unbounded memory, so looping is preferrable whenever possible):

allPosts := edb.All(edb.ExactIndexScan[Post](tx, postsByAuthor, "alice"))

Migrate schema versions by adding a migrator func; the number you pass in to edb.AddTable is the latest schema version number, and your func is responsible for migrating older versions:

var (
    postsTable = edb.AddTable(mySchema, "posts", 2, ..., func(tx *edb.Tx, post *Post, oldVer uint64) {
        if oldVer < 2 {
            post.Author = strings.ToLower(posts.Author)
        }
    }, ...)
)

The migrator will be invoked when loading a post. Schema version is stored per row, and you don't need to migrate all rows immediately; it's okay to migrate them as they get saved. We'll add more options for schema migrations later.

These examples use the following two error handling helpers:

func must[T any](v T, err error) T {
    if err != nil {
        panic(err)
    }
    return v
}

func ensure(err error) {
    if err != nil {
        panic(err)
    }
}

Technical Details

Buckets

We rely on scoped namespaces for keys called buckets. Bolt supports them natively. A flat database like Badger will simulate buckets via key prefixes. We use nested buckets in Bolt (tablename/data and tablename/i_indexname), but only for conveninece; flat buckets are fine.

Key Encoding

Keys are encoded using a tuple encoding. This concatenates all values together, and then appends a reversed variable-length encoding of lengths of each component except for the last one, and then the number of components. For single-component keys, the overhead is a single byte (1) at the end.

Value Encoding

Value = value header, encoded data, encoded index keys.

Value header:_

  1. Flags (uvarint).
  2. Schema version (uvarint).
  3. Data size (uvarint).
  4. Index size (uvarint).

Encoded data is just msgpack encoding of the struct.

Encoded index keys record the keys contributed by this row. If index computation changes in the future, we still need to know which index keys to delete when updating the row, so we store all index keys added by the row. Format:

  1. Number of entries (uvarint).
  2. For each entry: index ordinal (uvarint), key length (uvarint), key bytes.

Table States

We store a meta document per table, called “table state”. This document holds info on which indexes are defined for the table, and assigns an index ordinal (a unique positive integer) to each one. Ordinals are never reused as indexes are removed and added, so are safe to store inside values.

We should probably move to a single per-database state document.

Contributing

Contributions are welcome. There's a lot to do here still. Tests and documention will be much appreciated, too.

Auto-testing via modd (go install github.com/cortesi/modd/cmd/modd@latest):

modd

MIT license

Copyright (c) 2023 Andrey Tarantsov. Published under the terms of the MIT license.

Documentation

Overview

Package edb implements a document database on top of a key-value store (in this case, on top of Bolt).

We implement:

1. Tables, collection of arbitrary documents marshaled from the given struct.

2. Indices, allowing quick lookup of table data by custom values.

3. Maps, exposing raw key-value buckets with string keys and untyped values.

4. Singleton Keys, allowing to store a typed value for a given key within a map (say, a “config” value in a “globals” map).

Technical Details

**Buckets.** We rely on scoped namespaces for keys called buckets. Bolt supports them natively. A flat database like Redis could simulate buckets via key prefixes. We use nested buckets in Bolt, but only for conveninece; flat buckets are fine.

**Index ordinal** We assign a unique positive integer ordinal to each index. These values are never reused, even if an index is removed.

**Table states** We store a meta document per table, called “table state”. This document holds the information about which indexes are defined for the table, and assigns a (an ordinal) to each one. Ordinals are never reused as indexes are removed and added.

## Binary encoding

**Key encoding**. Keys are encoded using a _tuple encoding_.

**Value**: value header, then encoded data, then encoded index key records.

**Value header**: 1. Flags (uvarint). 2. Schema version (uvarint). 3. Data size (uvarint). 3. Index size (uvarint).

**Value data**: msgpack of the row struct.

**Index key records** (inside a value) record the keys contributed by this row. If index computation changes in the future, we still need to know which index keys to delete when updating the row, so we store all index keys. Format: 1. Number of entries (uvarint). 2. For each entry: index ordinal (uvarint), key length (uvarint), key bytes.

Index

Constants

View Source 
const (
	DumpTableHeaders = DumpFlags(1 << iota)
	DumpRows
	DumpStats
	DumpIndices
	DumpIndexRows

	DumpAll = DumpFlags(0xFFFFFFFFFFFFFFFF)
)
View Source 
const (
	MsgPack encodingMethod = iota
	JSON
)
View Source 
const (
	ScanMethodFull = ScanMethod(iota)
	ScanMethodExact
	ScanMethodRange
	ScanMethodExactIndexWithIDRange
)
View Source 
const (
	SuppressContentWhenLogging = tableOpt(1)
)

Variables

View Source 
var (
	ReaderCount atomic.Int64
	WriterCount atomic.Int64
)
View Source 
var Break = errors.New("break")

Functions

func All 

func All[Row any](c Cursor[Row]) []*Row

func AllKeys 

func AllKeys[Key any](c RawCursor) []Key

func AllLimited added in v0.1.6 

func AllLimited[Row any](c Cursor[Row], limit int) []*Row

func AllRawKeys 

func AllRawKeys(c RawCursor) [][]byte

func AllTableRows 

func AllTableRows[Row any](txh Txish) []*Row

func AllUntypedKeys 

func AllUntypedKeys(c RawCursor) []any

func Count added in v0.1.6 

func Count(c RawCursor) int

func CountAll 

func CountAll(txh Txish, tbl *Table) int

func DeleteAll 

func DeleteAll(c RawCursor) int

func DeleteByKey 

func DeleteByKey[Row any](txh Txish, key any) bool

func DeleteRow 

func DeleteRow[Row any](txh Txish, row *Row) bool

func Exists 

func Exists[Row any](txh Txish, key any) bool

func Filter added in v0.1.1 

func Filter[Row any](c Cursor[Row], f func(*Row) bool) []*Row

func First added in v0.1.1 

func First[Row any](c Cursor[Row]) *Row

func Foo added in v0.3.9 

func Foo() int64

func Get 

func Get[Row any](txh Txish, key any) *Row

func GetByKeyRaw added in v0.1.6 

func GetByKeyRaw[Row any](txh Txish, keyRaw []byte) *Row

func Lookup 

func Lookup[Row any](txh Txish, idx *Index, indexKey any) *Row

func LookupExists added in v0.2.0 

func LookupExists(txh Txish, idx *Index, indexKey any) bool

func LookupKey 

func LookupKey[Key any](txh Txish, idx *Index, indexKey any) (Key, bool)

func Memo 

func Memo[T any](txish Txish, key string, f func() (T, error)) (T, error)

func Proto 

func Proto[T any]() any

func Put 

func Put(txh Txish, rows ...any) bool

func Reload 

func Reload[Row any](txh Txish, row *Row) *Row

func SGet 

func SGet[T any](txh Txish, sk *SKey, v *T) bool

func SGetRaw 

func SGetRaw(txh Txish, sk *SKey) []byte

func SPut 

func SPut[T any](txh Txish, sk *SKey, v *T)

func SPutRaw 

func SPutRaw(txh Txish, sk *SKey, raw []byte)

func Select added in v0.1.1 

func Select[Row any](c Cursor[Row], f func(*Row) bool) *Row

Types

type Change added in v0.2.0 

type Change struct {
	// contains filtered or unexported fields
}

func (*Change) HasKey added in v0.2.0 

func (chg *Change) HasKey() bool

func (*Change) HasOldRow added in v0.2.0 

func (chg *Change) HasOldRow() bool

func (*Change) HasRow added in v0.2.0 

func (chg *Change) HasRow() bool

func (*Change) Key added in v0.2.0 

func (chg *Change) Key() any

func (*Change) KeyVal added in v0.2.0 

func (chg *Change) KeyVal() reflect.Value

func (*Change) OldRow added in v0.2.0 

func (chg *Change) OldRow() any

func (*Change) OldRowVal added in v0.2.0 

func (chg *Change) OldRowVal() reflect.Value

func (*Change) Op added in v0.2.0 

func (chg *Change) Op() Op

func (*Change) RawKey added in v0.2.0 

func (chg *Change) RawKey() []byte

func (*Change) Row added in v0.2.0 

func (chg *Change) Row() any

func (*Change) RowVal added in v0.2.0 

func (chg *Change) RowVal() reflect.Value

func (*Change) Table added in v0.2.0 

func (chg *Change) Table() *Table

type ChangeFlags added in v0.2.0 

type ChangeFlags uint64
const (
	ChangeFlagNotify ChangeFlags = 1 << iota
	ChangeFlagIncludeKey
	ChangeFlagIncludeRow
	ChangeFlagIncludeOldRow
	ChangeFlagIncludeMutableRow
)

func (ChangeFlags) Contains added in v0.2.0 

func (v ChangeFlags) Contains(f ChangeFlags) bool

func (ChangeFlags) ContainsAny added in v0.2.0 

func (v ChangeFlags) ContainsAny(f ChangeFlags) bool

type Cursor 

type Cursor[Row any] struct {
	RawCursor
}

func ExactIndexScan 

func ExactIndexScan[Row any](txh Txish, idx *Index, indexValue any) Cursor[Row]

func ExactTableScan added in v0.1.6 

func ExactTableScan[Row any](txh Txish, value any) Cursor[Row]

func FullIndexScan 

func FullIndexScan[Row any](txh Txish, idx *Index) Cursor[Row]

func FullReverseTableScan added in v0.1.4 

func FullReverseTableScan[Row any](txh Txish) Cursor[Row]

func FullTableScan added in v0.1.1 

func FullTableScan[Row any](txh Txish) Cursor[Row]

func IndexScan 

func IndexScan[Row any](txh Txish, idx *Index, opt ScanOptions) Cursor[Row]

func PrefixIndexScan 

func PrefixIndexScan[Row any](txh Txish, idx *Index, els int, indexValue any) Cursor[Row]

func RangeIndexScan added in v0.1.6 

func RangeIndexScan[Row any](txh Txish, idx *Index, lowerValue, upperValue any, lowerInc, upperInc bool) Cursor[Row]

func RangeTableScan added in v0.1.6 

func RangeTableScan[Row any](txh Txish, lowerValue, upperValue any, lowerInc, upperInc bool) Cursor[Row]

func ReverseExactIndexScan added in v0.1.5 

func ReverseExactIndexScan[Row any](txh Txish, idx *Index, indexValue any) Cursor[Row]

func ReversePrefixIndexScan added in v0.1.5 

func ReversePrefixIndexScan[Row any](txh Txish, idx *Index, els int, indexValue any) Cursor[Row]

func ReverseRangeIndexScan added in v0.1.6 

func ReverseRangeIndexScan[Row any](txh Txish, idx *Index, lowerValue, upperValue any, lowerInc, upperInc bool) Cursor[Row]

func ReverseRangeTableScan added in v0.1.6 

func ReverseRangeTableScan[Row any](txh Txish, lowerValue, upperValue any, lowerInc, upperInc bool) Cursor[Row]

func TableScan 

func TableScan[Row any](txh Txish, opt ScanOptions) Cursor[Row]

func (Cursor[Row]) Key 

func (c Cursor[Row]) Key() any

func (Cursor[Row]) Meta 

func (c Cursor[Row]) Meta() ValueMeta

func (Cursor[Row]) Next 

func (c Cursor[Row]) Next() bool

func (Cursor[Row]) Raw added in v0.1.6 

func (c Cursor[Row]) Raw() RawCursor

func (Cursor[Row]) Row 

func (c Cursor[Row]) Row() *Row

func (Cursor[Row]) RowVal 

func (c Cursor[Row]) RowVal() (reflect.Value, ValueMeta)

type DB 

type DB struct {
	ReaderCount        atomic.Int64
	WriterCount        atomic.Int64
	PendingWriterCount atomic.Int64
	ReadCount          atomic.Uint64
	WriteCount         atomic.Uint64
	// contains filtered or unexported fields
}

func Open 

func Open(path string, schema *Schema, opt Options) (*DB, error)

func (*DB) BeginRead 

func (db *DB) BeginRead() *Tx

func (*DB) BeginUpdate 

func (db *DB) BeginUpdate() *Tx

func (*DB) Bolt 

func (db *DB) Bolt() *bbolt.DB

func (*DB) Close 

func (db *DB) Close()

Close is safe to call multiple times, but not concurrently.

func (*DB) DescribeOpenTxns added in v0.1.6 

func (db *DB) DescribeOpenTxns() string

func (*DB) IsClosed added in v0.3.12 

func (db *DB) IsClosed() bool

func (*DB) Read 

func (db *DB) Read(f func(tx *Tx))

func (*DB) ReadErr 

func (db *DB) ReadErr(f func(tx *Tx) error) error

func (*DB) Size added in v0.1.6 

func (db *DB) Size() int64

func (*DB) Tx 

func (db *DB) Tx(writable bool, f func(tx *Tx) error) error

Tx currently implements Check-Mutate phases for writable transactions:

Phase 1, Check: before any modifications are made. Runs inside bdb.Batch. Returning an error won't cause the entire batch to fail.

Phase 2, Mutate: from the first mutation. Runs inside bdb.Batch. The entire transaction will be retried in case of error.

TODO: split Mutate phase into Mutate and Post-Mutate phases:

Phase 1, Check (initial phase): inside bdb.Batch, error does not fail batch.

Phase 2, Mutate (initiated by any mutation call): inside bdb.Batch, error fails batch.

Phase 3, Read (initiated by explicit call like Commit): mutations committed, outside bdb.Batch, a new read-only tx is opened on demand.

Check-Mutate-Read would allow to avoid holding the batch during rendering.

A read-only transaction would be a natural extension of Check-Mutate-Read with Check and Mutate phases skipped.

func (*DB) Write 

func (db *DB) Write(f func(tx *Tx))

type DataError 

type DataError struct {
	Data []byte
	Off  int
	Err  error
	Msg  string
}

func (*DataError) Error 

func (e *DataError) Error() string

func (*DataError) Unwrap 

func (e *DataError) Unwrap() error

type DumpFlags 

type DumpFlags uint64

func (DumpFlags) Contains 

func (f DumpFlags) Contains(v DumpFlags) bool

type FlatMarshaler 

type FlatMarshaler interface {
	MarshalFlat(buf []byte) []byte
}

type FlatMarshallable 

type FlatMarshallable interface {
	FlatMarshaler
	FlatUnmarshaler
}

type FlatUnmarshaler 

type FlatUnmarshaler interface {
	UnmarshalFlat(buf []byte) error
}

type Index 

type Index struct {
	// contains filtered or unexported fields
}

func AddIndex 

func AddIndex[T any](name string, opts ...any) *Index

func (*Index) DecodeIndexKeyVal added in v0.1.6 

func (idx *Index) DecodeIndexKeyVal(tup tuple) reflect.Value

func (*Index) DecodeIndexKeyValInto added in v0.2.0 

func (idx *Index) DecodeIndexKeyValInto(keyVal reflect.Value, tup tuple)

func (*Index) FullName 

func (idx *Index) FullName() string

func (*Index) ParseNakedIndexKey added in v0.2.2 

func (idx *Index) ParseNakedIndexKey(s string) (any, error)

func (*Index) ParseNakedIndexKeyVal added in v0.2.2 

func (idx *Index) ParseNakedIndexKeyVal(s string) (reflect.Value, error)

func (*Index) ShortName 

func (idx *Index) ShortName() string

func (*Index) Table added in v0.1.6 

func (idx *Index) Table() *Table

func (*Index) Unique 

func (idx *Index) Unique() *Index

type IndexBuilder 

type IndexBuilder struct {
	// contains filtered or unexported fields
}

func (*IndexBuilder) Add 

func (b *IndexBuilder) Add(idx *Index, value any) *IndexRow

type IndexOpt added in v0.2.0 

type IndexOpt int
const (
	IndexOptSkipInitialFill IndexOpt = iota
	IndexOptDebugScans
)

type IndexRow 

type IndexRow struct {
	IndexOrd uint64
	Index    *Index
	KeyRaw   []byte
	KeyBuf   []byte
	ValueRaw []byte
	ValueBuf []byte
}

type KVCursor added in v0.3.1 

type KVCursor struct {
	// contains filtered or unexported fields
}

func (*KVCursor) IndexKeys added in v0.3.2 

func (c *KVCursor) IndexKeys() func(yield func(k []byte) bool)

func (*KVCursor) Keys added in v0.3.1 

func (c *KVCursor) Keys() func(yield func(k []byte) bool)

func (*KVCursor) Next added in v0.3.1 

func (c *KVCursor) Next() bool

func (*KVCursor) Object added in v0.3.1 

func (c *KVCursor) Object() kvo.ImmutableMap

func (*KVCursor) Objects added in v0.3.1 

func (c *KVCursor) Objects() func(yield func(k []byte, o kvo.ImmutableMap) bool)

func (*KVCursor) RawIndexKey added in v0.3.2 

func (c *KVCursor) RawIndexKey() []byte

func (*KVCursor) RawKey added in v0.3.1 

func (c *KVCursor) RawKey() []byte

func (*KVCursor) RawValue added in v0.3.1 

func (c *KVCursor) RawValue() []byte

func (*KVCursor) RawValues added in v0.3.1 

func (c *KVCursor) RawValues() func(yield func(k, v []byte) bool)

type KVIndex added in v0.3.2 

type KVIndex struct {
	// contains filtered or unexported fields
}

func (*KVIndex) FullName added in v0.3.2 

func (idx *KVIndex) FullName() string

func (*KVIndex) ShortName added in v0.3.2 

func (idx *KVIndex) ShortName() string

func (*KVIndex) Table added in v0.3.2 

func (idx *KVIndex) Table() *KVTable

type KVIndexContentBuilder added in v0.3.2 

type KVIndexContentBuilder struct {
	// contains filtered or unexported fields
}

func (*KVIndexContentBuilder) Add added in v0.3.2 

func (b *KVIndexContentBuilder) Add(ik []byte)

type KVIndexKey added in v0.3.2 

type KVIndexKey interface {
	Clone() KVIndexKey
	PutBytes(raw []byte) bool
	String() string
	FillPrimaryKey(pk KVKey)
}

type KVIndexKeyToPrimaryKey added in v0.3.2 

type KVIndexKeyToPrimaryKey = func(ik []byte) []byte

type KVIndexer added in v0.3.2 

type KVIndexer = func(b *KVIndexContentBuilder, pk []byte, v kvo.ImmutableRecord)

type KVKey added in v0.3.2 

type KVKey interface {
	Clone() KVKey
	PutBytes(raw []byte) bool
	String() string
}

type KVMap 

type KVMap struct {
	// contains filtered or unexported fields
}

func AddKVMap 

func AddKVMap(scm *Schema, name string) *KVMap

type KVTable added in v0.3.0 

type KVTable struct {
	TaggableImpl
	// contains filtered or unexported fields
}

func DefineKVTable added in v0.3.0 

func DefineKVTable(scm *Schema, name string, rootModel *kvo.Model, keySample KVKey, build func(b *KVTableBuilder)) *KVTable

func DefineRawTable added in v0.3.3 

func DefineRawTable(scm *Schema, name string, build func(b *KVTableBuilder)) *KVTable

func (*KVTable) KeyBytesToString added in v0.3.2 

func (tbl *KVTable) KeyBytesToString(k []byte) string

func (*KVTable) Name added in v0.3.0 

func (tbl *KVTable) Name() string

func (*KVTable) NewKey added in v0.3.2 

func (tbl *KVTable) NewKey() KVKey

func (*KVTable) RootModel added in v0.3.2 

func (tbl *KVTable) RootModel() *kvo.Model

func (*KVTable) RootType added in v0.3.4 

func (tbl *KVTable) RootType() kvo.AnyType

type KVTableBuilder added in v0.3.0 

type KVTableBuilder struct {
	// contains filtered or unexported fields
}

func (*KVTableBuilder) DefineIndex added in v0.3.2 

func (b *KVTableBuilder) DefineIndex(name string, keySample KVIndexKey, resolver KVIndexKeyToPrimaryKey, indexer KVIndexer) *KVIndex

func (*KVTableBuilder) Tag added in v0.3.8 

func (b *KVTableBuilder) Tag(tag *Tag)

type Op added in v0.2.0 

type Op int
const (
	OpNone   Op = 0
	OpPut    Op = 1
	OpDelete Op = 2
)

func (Op) String added in v0.2.0 

func (v Op) String() string

type Options 

type Options struct {
	Logf      func(format string, args ...any)
	Verbose   bool
	IsTesting bool
	MmapSize  int

	NoPersistentFreeList bool
}

type RawCursor 

type RawCursor interface {
	Table() *Table
	Tx() *Tx
	Next() bool
	Key() any
	RawKey() []byte
	RowVal() (reflect.Value, ValueMeta)
	TryRowVal() (reflect.Value, ValueMeta, error)
	Meta() ValueMeta
	Row() (any, ValueMeta)
	TryRow() (any, ValueMeta, error)
	RawRow() []byte
}

type RawIndexCursor 

type RawIndexCursor struct {
	// contains filtered or unexported fields
}

func (*RawIndexCursor) IndexKey added in v0.1.6 

func (c *RawIndexCursor) IndexKey() any

func (*RawIndexCursor) Key 

func (c *RawIndexCursor) Key() any

func (*RawIndexCursor) Meta 

func (c *RawIndexCursor) Meta() ValueMeta

func (*RawIndexCursor) Next 

func (c *RawIndexCursor) Next() bool

func (*RawIndexCursor) RawKey 

func (c *RawIndexCursor) RawKey() []byte

func (*RawIndexCursor) RawRow added in v0.2.0 

func (c *RawIndexCursor) RawRow() []byte

func (*RawIndexCursor) Row 

func (c *RawIndexCursor) Row() (any, ValueMeta)

func (*RawIndexCursor) RowVal 

func (c *RawIndexCursor) RowVal() (reflect.Value, ValueMeta)

func (*RawIndexCursor) Table 

func (c *RawIndexCursor) Table() *Table

func (*RawIndexCursor) TryRow added in v0.2.1 

func (c *RawIndexCursor) TryRow() (any, ValueMeta, error)

func (*RawIndexCursor) TryRowVal added in v0.2.1 

func (c *RawIndexCursor) TryRowVal() (reflect.Value, ValueMeta, error)

func (*RawIndexCursor) Tx 

func (c *RawIndexCursor) Tx() *Tx

type RawRange added in v0.3.1 

type RawRange struct {
	Prefix   []byte
	Lower    []byte
	Upper    []byte
	LowerInc bool
	UpperInc bool
	Reverse  bool
}

RawRange defines a range of byte strings. The constructors use mnemonics: O means open, I means inclusive, E means exclusive; the first letter is for the lower bound, the second for the upper bound.

func RawEE added in v0.3.1 

func RawEE(l, u []byte) RawRange

func RawEI added in v0.3.1 

func RawEI(l, u []byte) RawRange

func RawEO added in v0.3.1 

func RawEO(l []byte) RawRange

func RawIE added in v0.3.1 

func RawIE(l, u []byte) RawRange

func RawII added in v0.3.1 

func RawII(l, u []byte) RawRange

func RawIO added in v0.3.1 

func RawIO(l []byte) RawRange

func RawOE added in v0.3.1 

func RawOE(u []byte) RawRange

func RawOI added in v0.3.1 

func RawOI(u []byte) RawRange

func RawOO added in v0.3.1 

func RawOO() RawRange

func RawPrefix added in v0.3.1 

func RawPrefix(p []byte) RawRange

func (RawRange) Prefixed added in v0.3.1 

func (rang RawRange) Prefixed(p []byte) RawRange

func (RawRange) Reversed added in v0.3.1 

func (rang RawRange) Reversed() RawRange

type RawRangeCursor added in v0.3.1 

type RawRangeCursor struct {
	// contains filtered or unexported fields
}

func (*RawRangeCursor) Key added in v0.3.1 

func (c *RawRangeCursor) Key() []byte

func (*RawRangeCursor) Next added in v0.3.1 

func (c *RawRangeCursor) Next() bool

func (*RawRangeCursor) Value added in v0.3.1 

func (c *RawRangeCursor) Value() []byte

type RawTableCursor 

type RawTableCursor struct {
	// contains filtered or unexported fields
}

func (*RawTableCursor) Key 

func (c *RawTableCursor) Key() any

func (*RawTableCursor) Meta 

func (c *RawTableCursor) Meta() ValueMeta

func (*RawTableCursor) Next 

func (c *RawTableCursor) Next() bool

func (*RawTableCursor) RawKey 

func (c *RawTableCursor) RawKey() []byte

func (*RawTableCursor) RawRow added in v0.2.0 

func (c *RawTableCursor) RawRow() []byte

func (*RawTableCursor) Row 

func (c *RawTableCursor) Row() (any, ValueMeta)

func (*RawTableCursor) RowVal 

func (c *RawTableCursor) RowVal() (reflect.Value, ValueMeta)

func (*RawTableCursor) Table 

func (c *RawTableCursor) Table() *Table

func (*RawTableCursor) TryRow added in v0.2.1 

func (c *RawTableCursor) TryRow() (any, ValueMeta, error)

func (*RawTableCursor) TryRowVal added in v0.2.1 

func (c *RawTableCursor) TryRowVal() (reflect.Value, ValueMeta, error)

func (*RawTableCursor) Tx 

func (c *RawTableCursor) Tx() *Tx

type SKey 

type SKey struct {
	// contains filtered or unexported fields
}

func AddSingletonKey 

func AddSingletonKey[T any](mp *KVMap, key string) *SKey

func (*SKey) Raw 

func (sk *SKey) Raw() []byte

func (*SKey) String 

func (sk *SKey) String() string

type ScanMethod 

type ScanMethod int

type ScanOptions 

type ScanOptions struct {
	Reverse  bool
	Method   ScanMethod
	Lower    reflect.Value
	Upper    reflect.Value
	LowerInc bool
	UpperInc bool
	Els      int
	Extra    reflect.Value
}

func ExactIDRangeScan added in v0.3.1 

func ExactIDRangeScan(exact, lower, upper any, lowerInc, upperInc bool) ScanOptions

func ExactIDRangeScanVal added in v0.3.1 

func ExactIDRangeScanVal(exact, lower, upper reflect.Value, lowerInc, upperInc bool) ScanOptions

func ExactScan 

func ExactScan(v any) ScanOptions

func ExactScanVal 

func ExactScanVal(val reflect.Value) ScanOptions

func FullScan 

func FullScan() ScanOptions

func LowerBoundScan added in v0.2.0 

func LowerBoundScan(lower any, includeEqual bool) ScanOptions

func RangeScan added in v0.1.6 

func RangeScan(lower, upper any, lowerInc, upperInc bool) ScanOptions

func RangeScanVal added in v0.1.6 

func RangeScanVal(lower, upper reflect.Value, lowerInc, upperInc bool) ScanOptions

func UpperBoundScan added in v0.2.0 

func UpperBoundScan(upper any, includeEqual bool) ScanOptions

func (ScanOptions) LogString added in v0.1.2 

func (so ScanOptions) LogString() string

func (ScanOptions) Prefix 

func (so ScanOptions) Prefix(els int) ScanOptions

func (ScanOptions) Reversed 

func (so ScanOptions) Reversed() ScanOptions

type Schema 

type Schema struct {
	Name string
	// contains filtered or unexported fields
}

func (*Schema) Include added in v0.1.3 

func (scm *Schema) Include(peer *Schema)

func (*Schema) KVTableNamed added in v0.3.2 

func (scm *Schema) KVTableNamed(name string) *KVTable

func (*Schema) KVTables added in v0.3.7 

func (scm *Schema) KVTables() []*KVTable

func (*Schema) TableByRow added in v0.1.2 

func (scm *Schema) TableByRow(row any) *Table

func (*Schema) TableByRowType added in v0.1.2 

func (scm *Schema) TableByRowType(rt reflect.Type) *Table

func (*Schema) TableNamed 

func (scm *Schema) TableNamed(name string) *Table

func (*Schema) Tables 

func (scm *Schema) Tables() []*Table

type Table 

type Table struct {
	TaggableImpl
	// contains filtered or unexported fields
}

func AddTable 

func AddTable[Row any](scm *Schema, name string, latestSchemaVer uint64, indexer func(row *Row, ib *IndexBuilder), migrator func(tx *Tx, row *Row, oldVer uint64), indices []*Index, opts ...any) *Table

func DefineTable added in v0.2.0 

func DefineTable[Row, Key any](scm *Schema, name string, f func(b *TableBuilder[Row, Key])) *Table

func (*Table) AddIndex added in v0.1.6 

func (tbl *Table) AddIndex(idx *Index) *Table

func (*Table) DecodeKeyVal 

func (tbl *Table) DecodeKeyVal(rawKey []byte) reflect.Value

func (*Table) DecodeKeyValInto added in v0.2.0 

func (tbl *Table) DecodeKeyValInto(keyVal reflect.Value, rawKey []byte)

func (*Table) EncodeKey 

func (tbl *Table) EncodeKey(key any) []byte

func (*Table) EncodeKeyVal added in v0.1.6 

func (tbl *Table) EncodeKeyVal(keyVal reflect.Value) []byte

func (*Table) IndexNamed added in v0.1.6 

func (tbl *Table) IndexNamed(name string) *Index

func (*Table) KeyString 

func (tbl *Table) KeyString(key any) string

func (*Table) KeyType 

func (tbl *Table) KeyType() reflect.Type

func (*Table) Name 

func (tbl *Table) Name() string

func (*Table) NewRow 

func (tbl *Table) NewRow() any

func (*Table) NewRowVal 

func (tbl *Table) NewRowVal() reflect.Value

func (*Table) ParseKey 

func (tbl *Table) ParseKey(s string) (any, error)

func (*Table) ParseKeyVal 

func (tbl *Table) ParseKeyVal(s string) (reflect.Value, error)

func (*Table) RawKeyString 

func (tbl *Table) RawKeyString(keyRaw []byte) string

func (*Table) RowHasZeroKey added in v0.1.2 

func (tbl *Table) RowHasZeroKey(row any) bool

func (*Table) RowKey 

func (tbl *Table) RowKey(row any) any

func (*Table) RowKeyVal 

func (tbl *Table) RowKeyVal(rowVal reflect.Value) reflect.Value

func (*Table) RowType added in v0.3.8 

func (tbl *Table) RowType() reflect.Type

func (*Table) RowValHasZeroKey added in v0.1.2 

func (tbl *Table) RowValHasZeroKey(rowVal reflect.Value) bool

func (*Table) SetRowKey 

func (tbl *Table) SetRowKey(row, key any)

func (*Table) SetRowKeyVal 

func (tbl *Table) SetRowKeyVal(rowVal, keyVal reflect.Value)

type TableBuilder added in v0.2.0 

type TableBuilder[Row, Key any] struct {
	// contains filtered or unexported fields
}

func (*TableBuilder[Row, Key]) AddIndex added in v0.2.0 

func (b *TableBuilder[Row, Key]) AddIndex(idx *Index)

func (*TableBuilder[Row, Key]) Indexer added in v0.2.0 

func (b *TableBuilder[Row, Key]) Indexer(f func(row *Row, ib *IndexBuilder))

func (*TableBuilder[Row, Key]) Migrate added in v0.2.0 

func (b *TableBuilder[Row, Key]) Migrate(f func(tx *Tx, row *Row, oldVer uint64))

func (*TableBuilder[Row, Key]) SetSchemaVersion added in v0.2.0 

func (b *TableBuilder[Row, Key]) SetSchemaVersion(ver uint64)

func (*TableBuilder[Row, Key]) SuppressContentWhenLogging added in v0.2.0 

func (b *TableBuilder[Row, Key]) SuppressContentWhenLogging()

func (*TableBuilder[Row, Key]) Tag added in v0.3.8 

func (b *TableBuilder[Row, Key]) Tag(tag *Tag)

type TableError 

type TableError struct {
	Table string
	Index string
	Key   []byte
	Msg   string
	Err   error
}

func (*TableError) Error 

func (e *TableError) Error() string

func (*TableError) Unwrap 

func (e *TableError) Unwrap() error

type TableStats 

type TableStats struct {
	Rows      int64
	IndexRows int64

	DataSize   int64
	DataAlloc  int64
	IndexSize  int64
	IndexAlloc int64
}

func (*TableStats) TotalAlloc 

func (ts *TableStats) TotalAlloc() int64

func (*TableStats) TotalSize 

func (ts *TableStats) TotalSize() int64

type Tag added in v0.3.8 

type Tag struct {
	// contains filtered or unexported fields
}

func NewTag added in v0.3.8 

func NewTag(name string) *Tag

func (*Tag) Name added in v0.3.8 

func (tag *Tag) Name() string

func (*Tag) String added in v0.3.8 

func (tag *Tag) String() string

type TagFamily added in v0.3.8 

type TagFamily struct {
	// contains filtered or unexported fields
}

func NewTagFamily added in v0.3.8 

func NewTagFamily(name string, tags []*Tag, defaultTag *Tag, card TagFamilyCardinality) *TagFamily

func (*TagFamily) Name added in v0.3.8 

func (fam *TagFamily) Name() string

func (*TagFamily) String added in v0.3.8 

func (fam *TagFamily) String() string

type TagFamilyCardinality added in v0.3.8 

type TagFamilyCardinality int
const (
	TagFamilyCardinalityOptionalOne TagFamilyCardinality = iota
	TagFamilyCardinalityRequiredOne
	TagFamilyCardinalityOptionalMany
	TagFamilyCardinalityRequiredMany
)

type TaggableImpl added in v0.3.8 

type TaggableImpl struct {
	// contains filtered or unexported fields
}

func (*TaggableImpl) HasTag added in v0.3.8 

func (i *TaggableImpl) HasTag(tag *Tag) bool

func (*TaggableImpl) TagInFamily added in v0.3.8 

func (i *TaggableImpl) TagInFamily(fam *TagFamily) *Tag

type Tx 

type Tx struct {
	// contains filtered or unexported fields
}

func (*Tx) BeginVerbose added in v0.2.0 

func (tx *Tx) BeginVerbose()

func (*Tx) Close 

func (tx *Tx) Close()

func (*Tx) Commit 

func (tx *Tx) Commit() error

func (*Tx) CommitDespiteError 

func (tx *Tx) CommitDespiteError()

func (*Tx) DB added in v0.1.2 

func (tx *Tx) DB() *DB

func (*Tx) DBTx 

func (tx *Tx) DBTx() *Tx

DBTx implements Txish

func (*Tx) DeleteByKey 

func (tx *Tx) DeleteByKey(tbl *Table, key any) bool

func (*Tx) DeleteByKeyRaw 

func (tx *Tx) DeleteByKeyRaw(tbl *Table, keyRaw []byte) bool

func (*Tx) DeleteByKeyVal 

func (tx *Tx) DeleteByKeyVal(tbl *Table, keyVal reflect.Value) bool

func (*Tx) Dump 

func (tx *Tx) Dump(f DumpFlags) string

func (*Tx) EndVerbose added in v0.2.0 

func (tx *Tx) EndVerbose()

func (*Tx) Exists 

func (tx *Tx) Exists(tbl *Table, key any) bool

func (*Tx) ExistsByKeyRaw added in v0.1.6 

func (tx *Tx) ExistsByKeyRaw(tbl *Table, keyRaw []byte) bool

func (*Tx) Get 

func (tx *Tx) Get(tbl *Table, key any) (any, ValueMeta)

func (*Tx) GetByKeyRaw added in v0.1.6 

func (tx *Tx) GetByKeyRaw(tbl *Table, keyRaw []byte) (any, ValueMeta)

func (*Tx) GetByKeyVal 

func (tx *Tx) GetByKeyVal(tbl *Table, keyVal reflect.Value) (any, ValueMeta)

func (*Tx) GetMemo 

func (tx *Tx) GetMemo(key string) (any, bool)

func (*Tx) GetMeta 

func (tx *Tx) GetMeta(tbl *Table, key any) ValueMeta

func (*Tx) GetMetaByKeyVal 

func (tx *Tx) GetMetaByKeyVal(tbl *Table, keyVal reflect.Value) ValueMeta

func (*Tx) IndexScan 

func (tx *Tx) IndexScan(idx *Index, opt ScanOptions) *RawIndexCursor

func (*Tx) IsWritable 

func (tx *Tx) IsWritable() bool

func (*Tx) KVGet added in v0.3.0 

func (tx *Tx) KVGet(tbl *KVTable, key []byte) kvo.ImmutableMap

func (*Tx) KVGetRaw added in v0.3.0 

func (tx *Tx) KVGetRaw(tbl *KVTable, key []byte) []byte

func (*Tx) KVIndexScan added in v0.3.2 

func (tx *Tx) KVIndexScan(idx *KVIndex, rang RawRange) *KVCursor

func (*Tx) KVPut added in v0.3.0 

func (tx *Tx) KVPut(tbl *KVTable, key []byte, value kvo.Packable)

func (*Tx) KVPutRaw added in v0.3.0 

func (tx *Tx) KVPutRaw(tbl *KVTable, key, value []byte)

func (*Tx) KVTableScan added in v0.3.1 

func (tx *Tx) KVTableScan(tbl *KVTable, rang RawRange) *KVCursor

func (*Tx) KVTableStats added in v0.3.7 

func (tx *Tx) KVTableStats(tbl *KVTable) TableStats

func (*Tx) Lookup 

func (tx *Tx) Lookup(idx *Index, indexKey any) any

func (*Tx) LookupExists added in v0.2.0 

func (tx *Tx) LookupExists(idx *Index, indexKeyVal reflect.Value) bool

func (*Tx) LookupKey 

func (tx *Tx) LookupKey(idx *Index, indexKey any) any

func (*Tx) LookupKeyVal 

func (tx *Tx) LookupKeyVal(idx *Index, indexKeyVal reflect.Value) reflect.Value

func (*Tx) LookupVal 

func (tx *Tx) LookupVal(idx *Index, indexKeyVal reflect.Value) (reflect.Value, ValueMeta)

func (*Tx) Memo 

func (tx *Tx) Memo(key string, f func() (any, error)) (any, error)

func (*Tx) OnChange added in v0.1.4 

func (tx *Tx) OnChange(opts map[*Table]ChangeFlags, f func(tx *Tx, chg *Change))

func (*Tx) Put 

func (tx *Tx) Put(tbl *Table, row any) (oldMeta, newMeta ValueMeta)

func (*Tx) PutVal 

func (tx *Tx) PutVal(tbl *Table, rowVal reflect.Value) (oldMeta, newMeta ValueMeta)

func (*Tx) Reindex 

func (tx *Tx) Reindex(tbl *Table, idx *Index)

func (*Tx) Schema added in v0.1.2 

func (tx *Tx) Schema() *Schema

func (*Tx) SetLogger added in v0.3.1 

func (tx *Tx) SetLogger(logger *slog.Logger)

func (*Tx) TableScan 

func (tx *Tx) TableScan(tbl *Table, opt ScanOptions) *RawTableCursor

func (*Tx) TableStats 

func (tx *Tx) TableStats(tbl *Table) TableStats

func (*Tx) TryGet added in v0.2.1 

func (tx *Tx) TryGet(tbl *Table, key any) (any, ValueMeta, error)

func (*Tx) TryGetByKeyVal added in v0.2.1 

func (tx *Tx) TryGetByKeyVal(tbl *Table, keyVal reflect.Value) (any, ValueMeta, error)

func (*Tx) UnsafeDeleteByKeyRawSkippingIndex added in v0.1.4 

func (tx *Tx) UnsafeDeleteByKeyRawSkippingIndex(tbl *Table, keyRaw []byte) bool

type Txish 

type Txish interface {
	DBTx() *Tx
}

type ValueMeta 

type ValueMeta struct {
	SchemaVer uint64
	ModCount  uint64
}

func (ValueMeta) Exists 

func (vm ValueMeta) Exists() bool

func (ValueMeta) IsMissing 

func (vm ValueMeta) IsMissing() bool

func (ValueMeta) IsModified added in v0.2.3 

func (vm ValueMeta) IsModified(oldMeta ValueMeta) bool

Source Files

View all Source files

Directories

Path Synopsis
Package journal implements WAL-like append-only journals.
 Package journal implements WAL-like append-only journals.
journaltest
Package kvo (stands for Key Value Objects) manages trees of objects mainly represented as key-value pairs.
 Package kvo (stands for Key Value Objects) manages trees of objects mainly represented as key-value pairs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.