txkey

Documentation

Overview

Copyright 2022 Molecula Corp. (DBA FeatureBase). SPDX-License-Identifier: Apache-2.0 Package txkey consolidates in one place the use of keys to index into our various storage/txn back-ends. Databases LMDB and rbfDB both use it, so that debug Dumps are comparable.

Index

• func AllShardPrefix(index, field, view string) (r []byte)
• func FieldPrefix(index, field string) (r []byte)
• func IndexOnlyPrefix(indexName string) (r []byte)
• func Key(index, field, view string, shard uint64, roaringContainerKey uint64) (r []byte)
• func KeyAndPrefix(index, field, view string, shard uint64, roaringContainerKey uint64) (key, prefix []byte)
• func KeyExtractContainerKey(bkey []byte) (containerKey uint64)
• func MustValidateKey(bkey []byte)
• func Prefix(index, field, view string, shard uint64) (r []byte)
• func PrefixFromKey(bkey []byte) (prefix []byte)
• func PrefixToString(pre []byte) (r string)
• func ShardFromKey(bkey []byte) (shard uint64)
• func ShardFromPrefix(prefix []byte) (shard uint64)
• func Split(bkey []byte) (index, field, view string, shard, ckey uint64)
• func SplitPrefix(pre []byte) (index, field, view string, shard uint64)
• func ToString(bkey []byte) (r string)
• type FieldView
  • func FieldViewFromFullKey(fullKey []byte) FieldView
  • func FieldViewFromPrefix(prefix []byte) FieldView

Functions

func AllShardPrefix

func AllShardPrefix(index, field, view string) (r []byte)

func FieldPrefix

func FieldPrefix(index, field string) (r []byte)

same for deleting a whole field.

func IndexOnlyPrefix

func IndexOnlyPrefix(indexName string) (r []byte)

IndexOnlyPrefix returns a prefix suitable for DeleteIndex and a key-scan to remove all storage associated with one index.

The full name of the index must be provided, no partial index names will work.

The returned prefix is terminated by '%' and so DeleteIndex("i") will not delete the index "i2".

func Key

func Key(index, field, view string, shard uint64, roaringContainerKey uint64) (r []byte)

Key produces the bytes that we use as a key to query the storage/tx engine. The roaringContainerKey argument to Key() is a container key into a roaring Container. The return value from Key() is constructed as follows:

~index%field;view:shard<ckey#

where shard and ckey are always exactly 8 bytes, uint64 big-endian encoded.

Keys always start with either '~' or '>'. Keys always end with '#'. Keys always contain exactly one each of '%', ';', ':' and '<', in that order. The index is between the first byte and the '%'. It must be at least 1 byte long. The field is between the '%' and the ';'. It must be at least 1 byte long. The view is between the ';' and the ':'. It must be at least 1 byte long. The shard is the 8 bytes between the ':' and the '<'. The ckey is the 8 bytes between the '<' and the '#'. The Prefix of a key ends at, and includes, the '<'. It is at least 16 bytes long. The index, field, and view are not allowed to contain these reserved bytes:

{'~', '>', ';', ':', '<', '#', '$', '%', '^', '(', ')', '*', '!'}

The bytes {'+', '/', '-', '_', '.', and '=' can be used in index, field, and view; to enable base-64 encoding.

The shortest possible key is 25 bytes. It would be laid out like this:

~i%f;v:12345678<12345678#
1234567890123456789012345

keys starting with '~' are regular value keys. keys starting with '>' are symlink keys.

NB must be kept in sync with Prefix() and KeyExtractContainerKey().

func KeyAndPrefix

func KeyAndPrefix(index, field, view string, shard uint64, roaringContainerKey uint64) (key, prefix []byte)

KeyAndPrefix returns the equivalent of Key() and Prefix() calls.

func KeyExtractContainerKey

func KeyExtractContainerKey(bkey []byte) (containerKey uint64)

KeyExtractContainerKey extracts the containerKey from bkey. key example: index/field;view:shard<ckey shortest: =i%f;v:12345678<12345678#

         1234567890123456789012345
numbering len(bkey) - i:
         5432109876543210987654321

func MustValidateKey

func MustValidateKey(bkey []byte)

func Prefix

func Prefix(index, field, view string, shard uint64) (r []byte)

Prefix returns everything from Key up to and including the '<' byte in a Key. The prefix excludes the roaring container key itself. NB must be kept in sync with Key() and KeyExtractContainerKey().

func PrefixFromKey

func PrefixFromKey(bkey []byte) (prefix []byte)

PrefixFromKey key example: index/field;view:shard<ckey

n-9    n-1

... : 01234567 < 01234567 #

shard       ckey

func PrefixToString

func PrefixToString(pre []byte) (r string)

func ShardFromKey

func ShardFromKey(bkey []byte) (shard uint64)

ShardFromKey key example: index/field;view:shard<ckey

n-9    n-1

... : 01234567 < 01234567 #

shard       ckey

func ShardFromPrefix

func ShardFromPrefix(prefix []byte) (shard uint64)

func Split

func Split(bkey []byte) (index, field, view string, shard, ckey uint64)

func SplitPrefix

func SplitPrefix(pre []byte) (index, field, view string, shard uint64)

full key: ~index%field;view:shard<ckey# prefix : ~index%field;view:shard<

func ToString

func ToString(bkey []byte) (r string)

Types

type FieldView

type FieldView struct {
	Field string
	View  string
}

FieldView is here to avoid circular import.

func FieldViewFromFullKey

func FieldViewFromFullKey(fullKey []byte) FieldView

func FieldViewFromPrefix

func FieldViewFromPrefix(prefix []byte) FieldView