mergeset

Documentation

Index

• func SetDataBlocksCacheSize(size int)
• func SetIndexBlocksCacheSize(size int)
• type Item
  • func (it Item) Bytes(data []byte) []byte
  • func (it Item) String(data []byte) string
• type PrepareBlockCallback
• type Table
  • func OpenTable(path string, flushCallback func(), prepareBlock PrepareBlockCallback, ...) (*Table, error)
  • func (tb *Table) AddItems(items [][]byte) error
  • func (tb *Table) CreateSnapshotAt(dstDir string) error
  • func (tb *Table) DebugFlush()
  • func (tb *Table) MustClose()
  • func (tb *Table) Path() string
  • func (tb *Table) UpdateMetrics(m *TableMetrics)
• type TableMetrics
• type TableSearch
  • func (ts *TableSearch) Error() error
  • func (ts *TableSearch) FirstItemWithPrefix(prefix []byte) error
  • func (ts *TableSearch) Init(tb *Table)
  • func (ts *TableSearch) MustClose()
  • func (ts *TableSearch) NextItem() bool
  • func (ts *TableSearch) Seek(k []byte)

Functions

func SetDataBlocksCacheSize

func SetDataBlocksCacheSize(size int)

SetDataBlocksCacheSize overrides the default size of indexdb/dataBlocks cache

func SetIndexBlocksCacheSize

func SetIndexBlocksCacheSize(size int)

SetIndexBlocksCacheSize overrides the default size of indexdb/indexBlock cache

Types

type Item

type Item struct {
	// Start is start offset for the item in data.
	Start uint32

	// End is end offset for the item in data.
	End uint32
}

Item represents a single item for storing in a mergeset.

func (Item) Bytes

func (it Item) Bytes(data []byte) []byte

Bytes returns bytes representation of it obtained from data.

The returned bytes representation belongs to data.

func (Item) String

func (it Item) String(data []byte) string

String returns string representation of it obtained from data.

The returned string representation belongs to data.

type PrepareBlockCallback

type PrepareBlockCallback func(data []byte, items []Item) ([]byte, []Item)

PrepareBlockCallback can transform the passed items allocated at the given data.

The callback is called during merge before flushing full block of the given items to persistent storage.

The callback must return sorted items. The first and the last item must be unchanged. The callback can re-use data and items for storing the result.

type Table

type Table struct {
	// contains filtered or unexported fields
}

Table represents mergeset table.

func OpenTable

func OpenTable(path string, flushCallback func(), prepareBlock PrepareBlockCallback, isReadOnly *uint32) (*Table, error)

OpenTable opens a table on the given path.

Optional flushCallback is called every time new data batch is flushed to the underlying storage and becomes visible to search.

Optional prepareBlock is called during merge before flushing the prepared block to persistent storage.

The table is created if it doesn't exist yet.

func (*Table) AddItems

func (tb *Table) AddItems(items [][]byte) error

AddItems adds the given items to the tb.

func (*Table) CreateSnapshotAt

func (tb *Table) CreateSnapshotAt(dstDir string) error

CreateSnapshotAt creates tb snapshot in the given dstDir.

Snapshot is created using linux hard links, so it is usually created very quickly.

func (*Table) DebugFlush

func (tb *Table) DebugFlush()

DebugFlush flushes all the added items to the storage, so they become visible to search.

This function is only for debugging and testing.

func (*Table) MustClose

func (tb *Table) MustClose()

MustClose closes the table.

func (*Table) Path

func (tb *Table) Path() string

Path returns the path to tb on the filesystem.

func (*Table) UpdateMetrics

func (tb *Table) UpdateMetrics(m *TableMetrics)

UpdateMetrics updates m with metrics from tb.

type TableMetrics

type TableMetrics struct {
	ActiveMerges        uint64
	MergesCount         uint64
	ItemsMerged         uint64
	AssistedMerges      uint64
	ItemsAdded          uint64
	ItemsAddedSizeBytes uint64

	PendingItems uint64

	PartsCount uint64

	BlocksCount uint64
	ItemsCount  uint64
	SizeBytes   uint64

	DataBlocksCacheSize         uint64
	DataBlocksCacheSizeBytes    uint64
	DataBlocksCacheSizeMaxBytes uint64
	DataBlocksCacheRequests     uint64
	DataBlocksCacheMisses       uint64

	IndexBlocksCacheSize         uint64
	IndexBlocksCacheSizeBytes    uint64
	IndexBlocksCacheSizeMaxBytes uint64
	IndexBlocksCacheRequests     uint64
	IndexBlocksCacheMisses       uint64

	PartsRefCount uint64
}

TableMetrics contains essential metrics for the Table.

type TableSearch

type TableSearch struct {
	// Item contains the next item after successful NextItem
	// or FirstItemWithPrefix call.
	//
	// Item contents breaks after the next call to NextItem.
	Item []byte
	// contains filtered or unexported fields
}

TableSearch is a reusable cursor used for searching in the Table.

func (*TableSearch) Error

func (ts *TableSearch) Error() error

Error returns the last error in ts.

func (*TableSearch) FirstItemWithPrefix

func (ts *TableSearch) FirstItemWithPrefix(prefix []byte) error

FirstItemWithPrefix seeks for the first item with the given prefix in the ts.

It returns io.EOF if such an item doesn't exist.

func (*TableSearch) Init

func (ts *TableSearch) Init(tb *Table)

Init initializes ts for searching in the tb.

MustClose must be called when the ts is no longer needed.

func (*TableSearch) MustClose

func (ts *TableSearch) MustClose()

MustClose closes the ts.

func (*TableSearch) NextItem

func (ts *TableSearch) NextItem() bool

NextItem advances to the next item.

func (*TableSearch) Seek

func (ts *TableSearch) Seek(k []byte)

Seek seeks for the first item greater or equal to k in the ts.