rbf

package
v2.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 4, 2021 License: Apache-2.0 Imports: 28 Imported by: 0

README

Roaring B-tree Format

The RBF format represents a Roaring bitmap whose containers are stored in the leafs of a b-tree. This allows the bitmap to be efficiently queried & updated.

File Format

The RBF file is divided into equal 8KB pages. Each page after the meta page is numbered incrementally from 1 to 1^31.

Pages can be one of the following types:

  • Meta page: contains header information.
  • Branch page: contains pointers to lower branch & leaf pages.
  • Leaf page: contains array and RLE container data.
  • Bitmap page: contains bitmap container data.

All integer values are little endian encoded.

Page header

Every page type except the bitmap page contains the following header:

Meta page

The meta page contains the following header:

[4]  magic (\xFFRBF)
[4]  flags
[4]  page count
[8]  wal ID
[4]  root records pgno
[4]  freelist pgno
Root Records page

A list of all b-tree names & their respective root page numbers are stored in root record pages. Once a bitmap root is created, it is never moved so the root record pages only need to be rewritten when creating, renaming, or deleting a b-tree. If records exceed the size of a page then they are overflowed to additional pages.

[4] page number
[4] flags
[4] overflow pgno
[*] bitmap records

Each bitmap record is represented as:

[4] pgno
[2] name size
[*] name

All bitmap records are loaded into memory when the file is opened.

Branch page

The branch page contains the following header:

[4] page number
[4] flags
[2] cell count
[*] cell index (2 * cell count)
[*] padding for 4-byte alignment

Each cell is formatted as:

[8] highbits
[4] flags
[4] page number
Leaf page

The leaf page contains the following header:

[4] page number
[4] flags
[2] cell count
[*] cell index (2 * cell count)

The leaf page contains a series of cells with the header of:

[8] highbits
[4] flag
[4] child count
[*] array or RLE data
Bitmap page

The data for the bitmap page takes up the entire 8KB.

Proof of Concept Notes

The following are notes made that are temporary for the RBF format. This will change as development progresses:

  • Transaction support is deferred
  • WAL support is deferred

Pattern of branch splits when adding data in ascending sorted order.

In this example, fan-out is restricted to 2 to make the drawings easy and the splits obvious. The data leaves are only allowed one roaring.Container key (ckey) in this example.

Each frame adds the next datum: A,B,C,D,E,... in order.

The letter represent data leaves, while numbers represent branch pages. The one exception is the first frame where the root is a leaf with data. Every frame after has normal branch at the root.

NB: there are only three places pages get written on addition: a) writeRoot b) putLeafCell c) putBranchCells

add A: root 3 A

add B: root 3 4 5 A B

add C: this sequence of updates occurs

  1. putLeafCell writes leaf B to page 5

  2. putLeafCell writes leaf C to page 6

  3. putBranchCells writes branch page 7 with children 4,5

  4. putBranchCells writes branch page 8 with child 6

  5. writeRoot writes branch cells to pgno 3, children: 7,8

    root 3 7 8 4 5 6 A B C

add D: root
3 7 8 4 5 6 9 A B C D

add E: root 3 12 13 7 8 11 4 5 6 9 10 A B C D E

add F: root 3 12 13 7 8 11 4 5 6 9 10 14 A B C D E F

add G: root 3 12 13 7 8 11 16 4 5 6 9 10 14 15 A B C D E F G

add H: root 3 12 13 7 8 11 16 4 5 6 9 10 14 15 17 A B C D E F G H

add I: root 3 21 22 12 13 20 7 8 11 16 19 4 5 6 9 10 14 15 17 18 A B C D E F G H I

Documentation

Overview

Copyright 2017 Pilosa Corp.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright 2017 Pilosa Corp.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright 2017 Pilosa Corp.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

start with https://github.com/benbjohnson/immutable and specialize Map<uint32,int64>

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Package rbf implements the roaring b-tree file format.

Copyright 2017 Pilosa Corp.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Copyright 2017 Pilosa Corp.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at 

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Index

Constants

View Source 
const (
	// Magic is the first 4 bytes of the RBF file.
	Magic = "\xFFRBF"

	// PageSize is the fixed size for every database page.
	PageSize = 8192

	// ShardWidth represents the number of bits per shard.
	ShardWidth = 1 << shardwidth.Exponent

	// RowValueMask masks the low bits for a row.
	RowValueMask = ShardWidth - 1

	// ArrayMaxSize represents the maximum size of array containers.
	// This is sligtly less than roaring to accommodate the page header.
	ArrayMaxSize = 4079

	// RLEMaxSize represents the maximum size of run length encoded containers.
	RLEMaxSize = 2039
)
View Source 
const (
	PageTypeRootRecord   = 1
	PageTypeLeaf         = 2
	PageTypeBranch       = 4
	PageTypeBitmapHeader = 8  // Only used by the WAL for marking next page
	PageTypeBitmap       = 16 // Only used internally when walking the b-tree
)

Page types.

View Source 
const (
	MetaPageFlagCommit   = 1
	MetaPageFlagRollback = 2
)

Meta commit/rollback flags.

View Source 
const (
	BitmapN = (1 << 16) / 64
)
View Source 
const RFC3339MsecTz0 = "2006-01-02T15:04:05.000Z07:00"
View Source 
const RFC3339UsecTz0 = "2006-01-02T15:04:05.000000Z07:00"

Variables

View Source 
var (
	ErrTxClosed           = errors.New("transaction closed")
	ErrTxNotWritable      = errors.New("transaction not writable")
	ErrBitmapNameRequired = errors.New("bitmap name required")
	ErrBitmapNotFound     = errors.New("bitmap not found")
	ErrBitmapExists       = errors.New("bitmap already exists")
	ErrTxTooLarge         = errors.New("rbf tx too large")
)
View Source 
var Debug bool

Debug is just a temporary flag used for debugging.

View Source 
var (
	ErrClosed = errors.New("rbf: database closed")
)
View Source 
var OurStdout io.Writer = os.Stdout

so we can multi write easily, use our own printf

View Source 
var VerboseVerbose bool = false

for tons of debug output

Functions

func AlwaysPrintf 

func AlwaysPrintf(format string, a ...interface{})

func Caller 

func Caller(upStack int) string

Caller returns the name of the calling function.

func ConvertToLeafArgs 

func ConvertToLeafArgs(key uint64, c *roaring.Container) (result leafCell)

func CreateDirIfNotExist 

func CreateDirIfNotExist(path string)

func DirExists 

func DirExists(name string) bool

func EnableRowCache 

func EnableRowCache() bool

func FileExists 

func FileExists(name string) bool

func FileLine 

func FileLine(depth int) string

func FileSize 

func FileSize(name string) (int64, error)

func IsBitmapHeader 

func IsBitmapHeader(page []byte) bool

func IsMetaPage 

func IsMetaPage(page []byte) bool

IsMetaPage returns true if page is a meta page.

func Magic32 

func Magic32() uint32

Magic32 returns the magic bytes as a big endian encoded uint32.

func PP 

func PP(format string, a ...interface{})

func Pagedump 

func Pagedump(b []byte, indent string, writer io.Writer)

func PanicOn 

func PanicOn(err error)

func Printf 

func Printf(format string, a ...interface{}) (n int, err error)

Printf formats according to a format specifier and writes to standard output. It returns the number of bytes written and any write error encountered.

func RowValues 

func RowValues(b []uint64) []uint64

RowValues returns a list of integer values from a row bitmap.

func SetRowcacheOn added in v2.3.0 

func SetRowcacheOn(on bool)

SetEnableRowCache should only be called in NewHolder before all other reads.

func TSPrintf 

func TSPrintf(format string, a ...interface{})

time-stamped printf

func VV 

func VV(format string, a ...interface{})

func Walk 

func Walk(tx *Tx, pgno uint32, v func(uint32, []*RootRecord))

func WalkPage added in v2.3.0 

func WalkPage(tx *Tx, pgno uint32, walker Walker)

func WalkRootRecordPages 

func WalkRootRecordPages(page []byte) uint32

func WriteRootRecord 

func WriteRootRecord(data []byte, rec *RootRecord) (remaining []byte, err error)

WriteRootRecord writes a root record with the pgno & name. Returns io.ErrShortBuffer if there is not enough space.

Types

type BitmapPage added in v2.3.0 

type BitmapPage struct {
	*BitmapPageInfo
	Values []uint16
}

type BitmapPageInfo added in v2.3.0 

type BitmapPageInfo struct {
	Pgno   uint32
	Parent uint32
	Tree   string
}

type BranchCell added in v2.3.0 

type BranchCell struct {
	Key   uint64
	Flags uint32
	Pgno  uint32
}

BranchCell represents a branch cell in the public API.

type BranchPage added in v2.3.0 

type BranchPage struct {
	*BranchPageInfo
	Cells []*BranchCell
}

type BranchPageInfo added in v2.3.0 

type BranchPageInfo struct {
	Pgno   uint32
	Parent uint32
	Tree   string
	Flags  uint32
	CellN  int
}

type ContainerType added in v2.4.0 

type ContainerType int
const (
	ContainerTypeNone ContainerType = iota
	ContainerTypeArray
	ContainerTypeRLE
	ContainerTypeBitmap
	ContainerTypeBitmapPtr
)

Container types.

func (ContainerType) String added in v2.4.0 

func (typ ContainerType) String() string

ContainerTypeString returns a string representation of the container type.

type Cursor 

type Cursor struct {
	// contains filtered or unexported fields
}

func (*Cursor) Add 

func (c *Cursor) Add(v uint64) (changed bool, err error)

Add sets a bit on the underlying bitmap.

func (*Cursor) AddRoaring 

func (c *Cursor) AddRoaring(bm *roaring.Bitmap) (changed bool, err error)

func (*Cursor) Close added in v2.4.0 

func (c *Cursor) Close()

func (*Cursor) Contains 

func (c *Cursor) Contains(v uint64) (exists bool, err error)

Contains returns true if a bit is set on the underlying bitmap.

func (*Cursor) CurrentPageType 

func (c *Cursor) CurrentPageType() ContainerType

CurrentPageType returns the type of the container currently pointed to by cursor used in testing sometimes the cursor needs to be positions prior to this call with First/Last etc.

func (*Cursor) Dump 

func (c *Cursor) Dump(name string)

func (*Cursor) DumpKeys 

func (c *Cursor) DumpKeys()

func (*Cursor) DumpStack 

func (c *Cursor) DumpStack()

func (*Cursor) First 

func (c *Cursor) First() error

First moves to the first element of the btree.

func (*Cursor) Key 

func (c *Cursor) Key() uint64

Key returns the key for the container the cursor is currently pointing to.

func (*Cursor) Last 

func (c *Cursor) Last() error

Last moves to the last element of the btree.

func (*Cursor) Next 

func (c *Cursor) Next() error

Next moves to the next element of the btree. Returns EOF if no more elements exist.

func (*Cursor) Prev 

func (c *Cursor) Prev() error

Prev moves to the previous element of the btree.

func (*Cursor) Remove 

func (c *Cursor) Remove(v uint64) (changed bool, err error)

Remove unsets a bit on the underlying bitmap.

func (*Cursor) RemoveRoaring 

func (c *Cursor) RemoveRoaring(bm *roaring.Bitmap) (changed bool, err error)

func (*Cursor) Row 

func (c *Cursor) Row(shard, rowID uint64) (*roaring.Bitmap, error)

func (*Cursor) Rows 

func (c *Cursor) Rows() ([]uint64, error)

probably should just implement the container interface but for now i'll do it

func (*Cursor) Seek 

func (c *Cursor) Seek(key uint64) (exact bool, err error)

Seek moves to the specified container of the btree. If the container does not exist then it moves to the next container after the key.

func (*Cursor) Values 

func (c *Cursor) Values() []uint16

Values returns the values for the container the cursor is currently pointing to.

type DB 

type DB struct {

	// Path represents the path to the database file.
	Path string
	// contains filtered or unexported fields
}

DB options like MaxSize, FsyncEnabled, DoAllocZero can be set before calling DB.Open().

func NewDB 

func NewDB(path string, cfg *rbfcfg.Config) *DB

NewDB returns a new instance of DB. If cfg is nil we will use the rbfcfg.DefaultConfig().

func (*DB) Begin 

func (db *DB) Begin(writable bool) (_ *Tx, err error)

Begin starts a new transaction.

func (*DB) Check 

func (db *DB) Check() error

Check performs an integrity check.

func (*DB) Close 

func (db *DB) Close() (err error)

Close closes the database.

func (*DB) DataPath 

func (db *DB) DataPath() string

DataPath returns the path to the data file for the DB.

func (*DB) HasData 

func (db *DB) HasData(requireOneHotBit bool) (hasAnyRecords bool, err error)

HasData with requireOneHotBit=false returns hasAnyRecords true if any record has been stored, even if the value for that bitmap record turned out to have no bits hot (be all zeroes).

In this case, we are taking the attempted storage of any named bitmap into the database as evidence that the db is in use, and we return hasAnyRecords true.

Conversely, if requireOneHotBit is true, then a database consisting of only a named bitmap with an all zeroes (no bits hot) will return hasAnyRecords false. We must find at least a single hot bit inside the db in order to return hasAnyRecords true.

HasData is used by backend migration and blue/green checks.

If there is a disk error we return (false, error), so always check the error before deciding if hasAnyRecords is valid.

We will internally create and rollback a read-only transaction to answer this query.

func (*DB) Open 

func (db *DB) Open() (err error)

Open opens a database with the file specified in Path. Creates a new file if one does not already exist.

func (*DB) Size 

func (db *DB) Size() (int64, error)

Size returns the size of the database & WAL, in bytes.

func (*DB) TxN added in v2.3.0 

func (db *DB) TxN() int

TxN returns the number of active transactions.

func (*DB) WALPath 

func (db *DB) WALPath() string

WALPath returns the path to the WAL file.

func (*DB) WALSize 

func (db *DB) WALSize() int64

WALSize returns the size of the WAL, in bytes.

type FreePage added in v2.3.0 

type FreePage struct {
	*FreePageInfo
}

type FreePageInfo added in v2.3.0 

type FreePageInfo struct {
	Pgno uint32
}

type LeafCell added in v2.3.0 

type LeafCell struct {
	Key    uint64
	Type   ContainerType
	Pgno   uint32   // bitmap pointer only
	Values []uint16 // array & rle containers only
}

LeafCell represents a leaf cell in the public API.

type LeafPage added in v2.3.0 

type LeafPage struct {
	*LeafPageInfo
	Cells []*LeafCell
}

type LeafPageInfo added in v2.3.0 

type LeafPageInfo struct {
	Pgno   uint32
	Parent uint32
	Tree   string
	Flags  uint32
	CellN  int
}

type MetaPage added in v2.3.0 

type MetaPage struct {
	*MetaPageInfo
}

type MetaPageInfo added in v2.3.0 

type MetaPageInfo struct {
	Pgno             uint32
	Magic            []byte
	PageN            uint32
	WALID            int64
	RootRecordPageNo uint32
	FreelistPageNo   uint32
}

type Metric added in v2.4.0 

type Metric struct {
	// contains filtered or unexported fields
}

Metric is a simple, internal metric for check duration of operations.

func NewMetric added in v2.4.0 

func NewMetric(name string, interval int) Metric

func (*Metric) Inc added in v2.4.0 

func (m *Metric) Inc(d time.Duration)

type Nodetype 

type Nodetype int
const (
	Branch Nodetype = iota
	Leaf
	Bitmap
)

type Page 

type Page interface {
	// contains filtered or unexported methods
}

type PageInfo added in v2.3.0 

type PageInfo interface {
	// contains filtered or unexported methods
}

type PageMap added in v2.4.0 

type PageMap struct {
	// contains filtered or unexported fields
}

Map represents an immutable hash map implementation. The map uses a Hasher to generate hashes and check for equality of key values.

It is implemented as an Hash Array Mapped Trie.

func NewPageMap added in v2.4.0 

func NewPageMap() *PageMap

NewMap returns a new instance of Map. If hasher is nil, a default hasher implementation will automatically be chosen based on the first key added. Default hasher implementations only exist for int, string, and byte slice types.

func (*PageMap) Delete added in v2.4.0 

func (m *PageMap) Delete(key uint32) *PageMap

Delete returns a map with the given key removed. Removing a non-existent key will cause this method to return the same map.

func (*PageMap) Get added in v2.4.0 

func (m *PageMap) Get(key uint32) (value int64, ok bool)

Get returns the value for a given key and a flag indicating whether the key exists. This flag distinguishes a zero value set on a key versus a non-existent key in the map.

func (*PageMap) Iterator added in v2.4.0 

func (m *PageMap) Iterator() *PageMapIterator

Iterator returns a new iterator for the map.

func (*PageMap) Len added in v2.4.0 

func (m *PageMap) Len() int

Len returns the number of elements in the map.

func (*PageMap) Set added in v2.4.0 

func (m *PageMap) Set(key uint32, value int64) *PageMap

Set returns a map with the key set to the new value. A nil value is allowed.

This function will return a new map even if the updated value is the same as the existing value because Map does not track value equality.

type PageMapBuilder added in v2.4.0 

type PageMapBuilder struct {
	// contains filtered or unexported fields
}

PageMapBuilder represents an efficient builder for creating Maps.

func NewPageMapBuilder added in v2.4.0 

func NewPageMapBuilder() *PageMapBuilder

NewPageMapBuilder returns a new instance of PageMapBuilder.

func (*PageMapBuilder) Delete added in v2.4.0 

func (b *PageMapBuilder) Delete(key uint32)

Delete removes the given key. See Map.Delete() for additional details.

func (*PageMapBuilder) Get added in v2.4.0 

func (b *PageMapBuilder) Get(key uint32) (value int64, ok bool)

Get returns the value for the given key.

func (*PageMapBuilder) Iterator added in v2.4.0 

func (b *PageMapBuilder) Iterator() *PageMapIterator

Iterator returns a new iterator for the underlying map.

func (*PageMapBuilder) Len added in v2.4.0 

func (b *PageMapBuilder) Len() int

Len returns the number of elements in the underlying map.

func (*PageMapBuilder) Map added in v2.4.0 

func (b *PageMapBuilder) Map() *PageMap

Map returns the underlying map. Only call once. Builder is invalid after call. Will panic on second invocation.

func (*PageMapBuilder) Set added in v2.4.0 

func (b *PageMapBuilder) Set(key uint32, value int64)

Set sets the value of the given key. See Map.Set() for additional details.

type PageMapIterator added in v2.4.0 

type PageMapIterator struct {
	// contains filtered or unexported fields
}

MapIterator represents an iterator over a map's key/value pairs. Although map keys are not sorted, the iterator's order is deterministic.

func (*PageMapIterator) Done added in v2.4.0 

func (itr *PageMapIterator) Done() bool

Done returns true if no more elements remain in the iterator.

func (*PageMapIterator) First added in v2.4.0 

func (itr *PageMapIterator) First()

First resets the iterator to the first key/value pair.

func (*PageMapIterator) Next added in v2.4.0 

func (itr *PageMapIterator) Next() (key uint32, value int64, ok bool)

Next returns the next key/value pair. Returns a nil key when no elements remain.

type RootRecord 

type RootRecord struct {
	Name string
	Pgno uint32
}

func ReadRootRecord 

func ReadRootRecord(data []byte) (rec *RootRecord, remaining []byte, err error)

ReadRootRecord reads the page number & name for a root record. If there is not enough space or the pgno is zero then a nil record is returned. Returns the remaining buffer.

type RootRecordPage added in v2.3.0 

type RootRecordPage struct {
	*RootRecordPageInfo
	Records []*RootRecord
}

type RootRecordPageInfo added in v2.3.0 

type RootRecordPageInfo struct {
	Pgno uint32
	Next uint32
}

type Tx 

type Tx struct {

	// DeleteEmptyContainer lets us by default match the roaring
	// behavior where an existing container has all its bits cleared
	// but still sticks around in the database.
	DeleteEmptyContainer bool
	// contains filtered or unexported fields
}

Tx represents a transaction.

func (*Tx) Add 

func (tx *Tx) Add(name string, a ...uint64) (changeCount int, err error)

Add sets a given bit on the bitmap.

func (*Tx) AddRoaring 

func (tx *Tx) AddRoaring(name string, bm *roaring.Bitmap) (changed bool, err error)

func (*Tx) ApplyFilter added in v2.7.0 

func (tx *Tx) ApplyFilter(name string, key uint64, filter roaring.BitmapFilter) (err error)

func (*Tx) BitmapNames 

func (tx *Tx) BitmapNames() ([]string, error)

BitmapNames returns a list of all bitmap names.

func (*Tx) Check 

func (tx *Tx) Check() error

Check verifies the integrity of the database.

func (*Tx) Commit 

func (tx *Tx) Commit() error

Commit completes the transaction and persists data changes.

func (*Tx) Container 

func (tx *Tx) Container(name string, key uint64) (*roaring.Container, error)

Container returns a Roaring container by key.

func (*Tx) ContainerIterator 

func (tx *Tx) ContainerIterator(name string, key uint64) (citer roaring.ContainerIterator, found bool, err error)

func (*Tx) Contains 

func (tx *Tx) Contains(name string, v uint64) (bool, error)

Contains returns true if the given bit is set on the bitmap.

func (*Tx) Count 

func (tx *Tx) Count(name string) (uint64, error)

func (*Tx) CountRange 

func (tx *Tx) CountRange(name string, start, end uint64) (uint64, error)

roaring.countRange counts the number of bits set between [start, end).

func (*Tx) CreateBitmap 

func (tx *Tx) CreateBitmap(name string) error

CreateBitmap creates a new empty bitmap with the given name. Returns an error if the bitmap already exists.

func (*Tx) CreateBitmapIfNotExists 

func (tx *Tx) CreateBitmapIfNotExists(name string) error

CreateBitmapIfNotExists creates a new empty bitmap with the given name. This is a no-op if the bitmap already exists.

func (*Tx) Cursor 

func (tx *Tx) Cursor(name string) (*Cursor, error)

Cursor returns an instance of a cursor this bitmap.

func (*Tx) DBPath 

func (tx *Tx) DBPath() string

func (*Tx) DeleteBitmap 

func (tx *Tx) DeleteBitmap(name string) error

DeleteBitmap removes a bitmap with the given name. Returns an error if the bitmap does not exist.

func (*Tx) DeleteBitmapsWithPrefix 

func (tx *Tx) DeleteBitmapsWithPrefix(prefix string) error

DeleteBitmapsWithPrefix removes all bitmaps with a given prefix.

func (*Tx) Dump 

func (tx *Tx) Dump(short bool, shard uint64)

func (*Tx) DumpString 

func (tx *Tx) DumpString(short bool, shard uint64) (r string)

func (*Tx) FieldViews 

func (tx *Tx) FieldViews() []string

func (*Tx) ForEach 

func (tx *Tx) ForEach(name string, fn func(i uint64) error) error

func (*Tx) ForEachRange 

func (tx *Tx) ForEachRange(name string, start, end uint64, fn func(uint64) error) error

func (*Tx) GetSizeBytesWithPrefix added in v2.8.0 

func (tx *Tx) GetSizeBytesWithPrefix(prefix string) (n uint64, err error)

GetSizeBytesWithPrefix returns the size of bitmaps with a given key prefix.

func (*Tx) GetSortedFieldViewList added in v2.7.0 

func (tx *Tx) GetSortedFieldViewList() (fvs []txkey.FieldView, _ error)

func (*Tx) ImportRoaringBits 

func (tx *Tx) ImportRoaringBits(name string, itr roaring.RoaringIterator, clear bool, log bool, rowSize uint64, data []byte) (changed int, rowSet map[uint64]int, err error)

func (*Tx) Max 

func (tx *Tx) Max(name string) (uint64, error)

func (*Tx) Min 

func (tx *Tx) Min(name string) (uint64, bool, error)

func (*Tx) OffsetRange 

func (tx *Tx) OffsetRange(name string, offset, start, endx uint64) (*roaring.Bitmap, error)

func (*Tx) PageData added in v2.3.0 

func (tx *Tx) PageData(pgno uint32) ([]byte, error)

PageData returns the raw page data for a single page.

func (*Tx) PageInfos added in v2.3.0 

func (tx *Tx) PageInfos() ([]PageInfo, error)

PageInfos returns meta data about all pages in the database.

func (*Tx) PageN added in v2.3.0 

func (tx *Tx) PageN() int

PageN returns the number of pages in the database as seen by this transaction.

func (*Tx) Pages added in v2.3.0 

func (tx *Tx) Pages(pgnos []uint32) ([]Page, error)

Pages returns meta & record data for a list of pages.

func (*Tx) PutContainer 

func (tx *Tx) PutContainer(name string, key uint64, ct *roaring.Container) error

PutContainer inserts a container into a bitmap. Overwrites if key already exists.

func (*Tx) Remove 

func (tx *Tx) Remove(name string, a ...uint64) (changeCount int, err error)

Remove unsets a given bit on the bitmap.

func (*Tx) RemoveContainer 

func (tx *Tx) RemoveContainer(name string, key uint64) error

RemoveContainer removes a container from the bitmap by key.

func (*Tx) RenameBitmap 

func (tx *Tx) RenameBitmap(oldname, newname string) error

RenameBitmap updates the name of an existing bitmap. Returns an error if the bitmap does not exist.

func (*Tx) RoaringBitmap 

func (tx *Tx) RoaringBitmap(name string) (*roaring.Bitmap, error)

RoaringBitmap returns a bitmap as a Roaring bitmap.

func (*Tx) Rollback 

func (tx *Tx) Rollback()

func (*Tx) Root 

func (tx *Tx) Root(name string) (uint32, error)

Root returns the root page number for a bitmap. Returns 0 if the bitmap does not exist.

func (*Tx) RootRecords 

func (tx *Tx) RootRecords() (records *immutable.SortedMap, err error)

RootRecords returns a list of root records.

func (*Tx) UnionInPlace 

func (tx *Tx) UnionInPlace(name string, others ...*roaring.Bitmap) error

func (*Tx) Writable 

func (tx *Tx) Writable() bool

Writable returns true if the transaction can mutate data.

type Walker 

type Walker interface {
	Visitor(pgno uint32, records []*RootRecord)
	VisitRoot(pgno uint32, name string)
	Visit(pgno uint32, n Nodetype)
}

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.