Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SetDataBlocksCacheSize ¶ added in v1.73.1
func SetDataBlocksCacheSize(size int)
SetDataBlocksCacheSize overrides the default size of indexdb/dataBlocks cache
func SetDataFlushInterval ¶ added in v1.85.0
SetDataFlushInterval sets the interval for guaranteed flush of recently ingested data from memory to disk.
The data can be flushed from memory to disk more frequently if it doesn't fit the memory limit.
This function must be called before initializing the indexdb.
func SetIndexBlocksCacheSize ¶ added in v1.73.1
func SetIndexBlocksCacheSize(size int)
SetIndexBlocksCacheSize overrides the default size of indexdb/indexBlocks cache
Types ¶
type Item ¶ added in v1.55.0
type Item struct {
// Start is start offset for the item in data.
Start uint32
// End is end offset for the item in data.
End uint32
}
Item represents a single item for storing in a mergeset.
type PrepareBlockCallback ¶ added in v1.28.0
PrepareBlockCallback can transform the passed items allocated at the given data.
The callback is called during merge before flushing full block of the given items to persistent storage.
The callback must return sorted items. The first and the last item must be unchanged. The callback can re-use data and items for storing the result.
type Table ¶
type Table struct {
// contains filtered or unexported fields
}
Table represents mergeset table.
func MustOpenTable ¶ added in v1.91.0
func MustOpenTable(path string, flushCallback func(), prepareBlock PrepareBlockCallback, isReadOnly *atomic.Bool) *Table
MustOpenTable opens a table on the given path.
Optional flushCallback is called every time new data batch is flushed to the underlying storage and becomes visible to search.
Optional prepareBlock is called during merge before flushing the prepared block to persistent storage.
The table is created if it doesn't exist yet.
func (*Table) AddItems ¶
AddItems adds the given items to the tb.
The function ignores items with length exceeding maxInmemoryBlockSize. It logs the ignored items, so users could notice and fix the issue.
func (*Table) CreateSnapshotAt ¶
CreateSnapshotAt creates tb snapshot in the given dstDir.
Snapshot is created using linux hard links, so it is usually created very quickly.
The caller is responsible for data removal at dstDir on unsuccessful snapshot creation.
func (*Table) DebugFlush ¶
func (tb *Table) DebugFlush()
DebugFlush makes sure all the recently added data is visible to search.
Note: this function doesn't store all the in-memory data to disk - it just converts recently added items to searchable parts, which can be stored either in memory (if they are quite small) or to persistent disk.
This function is for debugging and testing purposes only, since it may slow down data ingestion when used frequently.
func (*Table) NotifyReadWriteMode ¶ added in v1.97.0
func (tb *Table) NotifyReadWriteMode()
NotifyReadWriteMode notifies tb that it may be switched from read-only mode to read-write mode.
func (*Table) UpdateMetrics ¶
func (tb *Table) UpdateMetrics(m *TableMetrics)
UpdateMetrics updates m with metrics from tb.
type TableMetrics ¶
type TableMetrics struct {
ActiveInmemoryMerges uint64
ActiveFileMerges uint64
InmemoryMergesCount uint64
FileMergesCount uint64
InmemoryItemsMerged uint64
FileItemsMerged uint64
ItemsAdded uint64
ItemsAddedSizeBytes uint64
InmemoryPartsLimitReachedCount uint64
PendingItems uint64
InmemoryPartsCount uint64
FilePartsCount uint64
InmemoryBlocksCount uint64
FileBlocksCount uint64
InmemoryItemsCount uint64
FileItemsCount uint64
InmemorySizeBytes uint64
FileSizeBytes uint64
DataBlocksCacheSize uint64
DataBlocksCacheSizeBytes uint64
DataBlocksCacheSizeMaxBytes uint64
DataBlocksCacheRequests uint64
DataBlocksCacheMisses uint64
IndexBlocksCacheSize uint64
IndexBlocksCacheSizeBytes uint64
IndexBlocksCacheSizeMaxBytes uint64
IndexBlocksCacheRequests uint64
IndexBlocksCacheMisses uint64
PartsRefCount uint64
TooLongItemsDroppedTotal uint64
}
TableMetrics contains essential metrics for the Table.
func (*TableMetrics) TotalItemsCount ¶ added in v1.85.0
func (tm *TableMetrics) TotalItemsCount() uint64
TotalItemsCount returns the total number of items in the table.
type TableSearch ¶
type TableSearch struct {
// Item contains the next item after successful NextItem
// or FirstItemWithPrefix call.
//
// Item contents breaks after the next call to NextItem.
Item []byte
// contains filtered or unexported fields
}
TableSearch is a reusable cursor used for searching in the Table.
func (*TableSearch) Error ¶
func (ts *TableSearch) Error() error
Error returns the last error in ts.
func (*TableSearch) FirstItemWithPrefix ¶
func (ts *TableSearch) FirstItemWithPrefix(prefix []byte) error
FirstItemWithPrefix seeks for the first item with the given prefix in the ts.
It returns io.EOF if such an item doesn't exist.
func (*TableSearch) Init ¶
func (ts *TableSearch) Init(tb *Table)
Init initializes ts for searching in the tb.
MustClose must be called when the ts is no longer needed.
func (*TableSearch) NextItem ¶
func (ts *TableSearch) NextItem() bool
NextItem advances to the next item.
func (*TableSearch) Seek ¶
func (ts *TableSearch) Seek(k []byte)
Seek seeks for the first item greater or equal to k in the ts.
Source Files