index

package
v0.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 15, 2019 License: Apache-2.0 Imports: 33 Imported by: 22

Documentation

Overview

Package index is a generated GoMock package.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrUnableToQueryBlockClosed is returned when querying closed block.
	ErrUnableToQueryBlockClosed = errors.New("unable to query, index block is closed")
	// ErrUnableReportStatsBlockClosed is returned from Stats when the block is closed.
	ErrUnableReportStatsBlockClosed = errors.New("unable to report stats, block is closed")
)
View Source 
var (
	// ReservedFieldNameID is the field name used to index the ID in the
	// m3ninx subsytem.
	ReservedFieldNameID = doc.IDReservedFieldName
)

Functions

func NewBootstrapResultMutableSegmentAllocator 

func NewBootstrapResultMutableSegmentAllocator(
	opts Options,
) result.MutableSegmentAllocator

NewBootstrapResultMutableSegmentAllocator returns a default mutable segment allocator for a bootstrap result index block given index options.

func NewPostingsListCache added in v0.6.0 

func NewPostingsListCache(size int, opts PostingsListCacheOptions) (*PostingsListCache, Closer, error)

NewPostingsListCache creates a new query cache.

func NewReadThroughSegment added in v0.6.0 

func NewReadThroughSegment(
	seg segment.Segment,
	cache *PostingsListCache,
	opts ReadThroughSegmentOptions,
) segment.Segment

NewReadThroughSegment creates a new read through segment.

Types

type Block 

type Block interface {
	// StartTime returns the start time of the period this Block indexes.
	StartTime() time.Time

	// EndTime returns the end time of the period this Block indexes.
	EndTime() time.Time

	// WriteBatch writes a batch of provided entries.
	WriteBatch(inserts *WriteBatch) (WriteBatchResult, error)

	// Query resolves the given query into known IDs.
	Query(
		query Query,
		opts QueryOptions,
		results Results,
	) (exhaustive bool, err error)

	// AddResults adds bootstrap results to the block, if c.
	AddResults(results result.IndexBlock) error

	// Tick does internal house keeping operations.
	Tick(c context.Cancellable, tickStart time.Time) (BlockTickResult, error)

	// Stats returns block stats.
	Stats(reporter BlockStatsReporter) error

	// Seal prevents the block from taking any more writes, but, it still permits
	// addition of segments via Bootstrap().
	Seal() error

	// IsSealed returns whether this block was sealed.
	IsSealed() bool

	// NeedsMutableSegmentsEvicted returns whether this block has any mutable segments
	// that are not-empty and sealed.
	// A sealed non-empty mutable segment needs to get evicted from memory as
	// soon as it can be to reduce memory footprint.
	NeedsMutableSegmentsEvicted() bool

	// EvictMutableSegments closes any mutable segments, this is only applicable
	// valid to be called once the block and hence mutable segments are sealed.
	// It is expected that results have been added to the block that covers any
	// data the mutable segments should have held at this time.
	EvictMutableSegments() error

	// Close will release any held resources and close the Block.
	Close() error
}

Block represents a collection of segments. Each `Block` is a complete reverse index for a period of time defined by [StartTime, EndTime).

func NewBlock 

func NewBlock(
	blockStart time.Time,
	md namespace.Metadata,
	opts Options,
) (Block, error)

NewBlock returns a new Block, representing a complete reverse index for the duration of time specified. It is backed by one or more segments.

type BlockSegmentStats added in v0.5.0 

type BlockSegmentStats struct {
	Type    BlockSegmentType
	Mutable bool
	Age     time.Duration
	Size    int64
}

BlockSegmentStats has segment stats.

type BlockSegmentType added in v0.5.0 

type BlockSegmentType uint

BlockSegmentType is a block segment type 

const (
	// ActiveForegroundSegment is an active foreground compacted segment.
	ActiveForegroundSegment BlockSegmentType = iota
	// ActiveBackgroundSegment is an active background compacted segment.
	ActiveBackgroundSegment
	// FlushedSegment is an immutable segment that can't change any longer.
	FlushedSegment
)

type BlockStatsReporter added in v0.5.0 

type BlockStatsReporter interface {
	ReportSegmentStats(stats BlockSegmentStats)
}

BlockStatsReporter is a block stats reporter that collects block stats on a per block basis (without needing to query each block and get an immutable list of segments back).

type BlockStatsReporterFn added in v0.5.0 

type BlockStatsReporterFn func(stats BlockSegmentStats)

BlockStatsReporterFn implements the block stats reporter using a callback function.

func (BlockStatsReporterFn) ReportSegmentStats added in v0.5.0 

func (f BlockStatsReporterFn) ReportSegmentStats(stats BlockSegmentStats)

ReportSegmentStats implements the BlockStatsReporter interface.

type BlockTickResult 

type BlockTickResult struct {
	NumSegments int64
	NumDocs     int64
}

BlockTickResult returns statistics about tick.

type Closer added in v0.6.0 

type Closer func()

Closer represents a function that will close managed resources.

type EvictMutableSegmentResults 

type EvictMutableSegmentResults struct {
	NumMutableSegments int64
	NumDocs            int64
}

EvictMutableSegmentResults returns statistics about the EvictMutableSegments execution.

func (*EvictMutableSegmentResults) Add 

func (e *EvictMutableSegmentResults) Add(o EvictMutableSegmentResults)

Add adds the provided results to the receiver.

type ForEachWriteBatchByBlockStartFn 

type ForEachWriteBatchByBlockStartFn func(
	blockStart time.Time,
	batch *WriteBatch,
)

ForEachWriteBatchByBlockStartFn allows a caller to perform an operation with reference to a restricted set of the write batch for each unique block start.

type ForEachWriteBatchEntryFn 

type ForEachWriteBatchEntryFn func(
	idx int,
	entry WriteBatchEntry,
	doc doc.Document,
	result WriteBatchEntryResult,
)

ForEachWriteBatchEntryFn allows a caller to perform an operation for each batch entry.

type InsertMode 

type InsertMode byte

InsertMode specifies whether inserts are synchronous or asynchronous. 

const (
	InsertSync InsertMode = iota
	InsertAsync
)

nolint

type MockBlock 

type MockBlock struct {
	// contains filtered or unexported fields
}

MockBlock is a mock of Block interface

func NewMockBlock 

func NewMockBlock(ctrl *gomock.Controller) *MockBlock

NewMockBlock creates a new mock instance

func (*MockBlock) AddResults 

func (m *MockBlock) AddResults(arg0 result.IndexBlock) error

AddResults mocks base method

func (*MockBlock) Close 

func (m *MockBlock) Close() error

Close mocks base method

func (*MockBlock) EXPECT 

func (m *MockBlock) EXPECT() *MockBlockMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockBlock) EndTime 

func (m *MockBlock) EndTime() time.Time

EndTime mocks base method

func (*MockBlock) EvictMutableSegments 

func (m *MockBlock) EvictMutableSegments() error

EvictMutableSegments mocks base method

func (*MockBlock) IsSealed 

func (m *MockBlock) IsSealed() bool

IsSealed mocks base method

func (*MockBlock) NeedsMutableSegmentsEvicted 

func (m *MockBlock) NeedsMutableSegmentsEvicted() bool

NeedsMutableSegmentsEvicted mocks base method

func (*MockBlock) Query 

func (m *MockBlock) Query(arg0 Query, arg1 QueryOptions, arg2 Results) (bool, error)

Query mocks base method

func (*MockBlock) Seal 

func (m *MockBlock) Seal() error

Seal mocks base method

func (*MockBlock) StartTime 

func (m *MockBlock) StartTime() time.Time

StartTime mocks base method

func (*MockBlock) Stats added in v0.5.0 

func (m *MockBlock) Stats(arg0 BlockStatsReporter) error

Stats mocks base method

func (*MockBlock) Tick 

func (m *MockBlock) Tick(arg0 context.Cancellable, arg1 time.Time) (BlockTickResult, error)

Tick mocks base method

func (*MockBlock) WriteBatch 

func (m *MockBlock) WriteBatch(arg0 *WriteBatch) (WriteBatchResult, error)

WriteBatch mocks base method

type MockBlockMockRecorder 

type MockBlockMockRecorder struct {
	// contains filtered or unexported fields
}

MockBlockMockRecorder is the mock recorder for MockBlock

func (*MockBlockMockRecorder) AddResults 

func (mr *MockBlockMockRecorder) AddResults(arg0 interface{}) *gomock.Call

AddResults indicates an expected call of AddResults

func (*MockBlockMockRecorder) Close 

func (mr *MockBlockMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

func (*MockBlockMockRecorder) EndTime 

func (mr *MockBlockMockRecorder) EndTime() *gomock.Call

EndTime indicates an expected call of EndTime

func (*MockBlockMockRecorder) EvictMutableSegments 

func (mr *MockBlockMockRecorder) EvictMutableSegments() *gomock.Call

EvictMutableSegments indicates an expected call of EvictMutableSegments

func (*MockBlockMockRecorder) IsSealed 

func (mr *MockBlockMockRecorder) IsSealed() *gomock.Call

IsSealed indicates an expected call of IsSealed

func (*MockBlockMockRecorder) NeedsMutableSegmentsEvicted 

func (mr *MockBlockMockRecorder) NeedsMutableSegmentsEvicted() *gomock.Call

NeedsMutableSegmentsEvicted indicates an expected call of NeedsMutableSegmentsEvicted

func (*MockBlockMockRecorder) Query 

func (mr *MockBlockMockRecorder) Query(arg0, arg1, arg2 interface{}) *gomock.Call

Query indicates an expected call of Query

func (*MockBlockMockRecorder) Seal 

func (mr *MockBlockMockRecorder) Seal() *gomock.Call

Seal indicates an expected call of Seal

func (*MockBlockMockRecorder) StartTime 

func (mr *MockBlockMockRecorder) StartTime() *gomock.Call

StartTime indicates an expected call of StartTime

func (*MockBlockMockRecorder) Stats added in v0.5.0 

func (mr *MockBlockMockRecorder) Stats(arg0 interface{}) *gomock.Call

Stats indicates an expected call of Stats

func (*MockBlockMockRecorder) Tick 

func (mr *MockBlockMockRecorder) Tick(arg0, arg1 interface{}) *gomock.Call

Tick indicates an expected call of Tick

func (*MockBlockMockRecorder) WriteBatch 

func (mr *MockBlockMockRecorder) WriteBatch(arg0 interface{}) *gomock.Call

WriteBatch indicates an expected call of WriteBatch

type MockOnIndexSeries 

type MockOnIndexSeries struct {
	// contains filtered or unexported fields
}

MockOnIndexSeries is a mock of OnIndexSeries interface

func NewMockOnIndexSeries 

func NewMockOnIndexSeries(ctrl *gomock.Controller) *MockOnIndexSeries

NewMockOnIndexSeries creates a new mock instance

func (*MockOnIndexSeries) EXPECT 

func (m *MockOnIndexSeries) EXPECT() *MockOnIndexSeriesMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockOnIndexSeries) OnIndexFinalize 

func (m *MockOnIndexSeries) OnIndexFinalize(arg0 time0.UnixNano)

OnIndexFinalize mocks base method

func (*MockOnIndexSeries) OnIndexSuccess 

func (m *MockOnIndexSeries) OnIndexSuccess(arg0 time0.UnixNano)

OnIndexSuccess mocks base method

type MockOnIndexSeriesMockRecorder 

type MockOnIndexSeriesMockRecorder struct {
	// contains filtered or unexported fields
}

MockOnIndexSeriesMockRecorder is the mock recorder for MockOnIndexSeries

func (*MockOnIndexSeriesMockRecorder) OnIndexFinalize 

func (mr *MockOnIndexSeriesMockRecorder) OnIndexFinalize(arg0 interface{}) *gomock.Call

OnIndexFinalize indicates an expected call of OnIndexFinalize

func (*MockOnIndexSeriesMockRecorder) OnIndexSuccess 

func (mr *MockOnIndexSeriesMockRecorder) OnIndexSuccess(arg0 interface{}) *gomock.Call

OnIndexSuccess indicates an expected call of OnIndexSuccess

type MockResults 

type MockResults struct {
	// contains filtered or unexported fields
}

MockResults is a mock of Results interface

func NewMockResults 

func NewMockResults(ctrl *gomock.Controller) *MockResults

NewMockResults creates a new mock instance

func (*MockResults) AddDocument added in v0.5.0 

func (m *MockResults) AddDocument(arg0 doc.Document) (bool, int, error)

AddDocument mocks base method

func (*MockResults) AddIDAndTags added in v0.5.0 

func (m *MockResults) AddIDAndTags(arg0 ident.ID, arg1 ident.Tags) (bool, int, error)

AddIDAndTags mocks base method

func (*MockResults) EXPECT 

func (m *MockResults) EXPECT() *MockResultsMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockResults) Finalize 

func (m *MockResults) Finalize()

Finalize mocks base method

func (*MockResults) Map 

func (m *MockResults) Map() *ResultsMap

Map mocks base method

func (*MockResults) Namespace 

func (m *MockResults) Namespace() ident.ID

Namespace mocks base method

func (*MockResults) NoFinalize added in v0.6.0 

func (m *MockResults) NoFinalize()

NoFinalize mocks base method

func (*MockResults) Reset 

func (m *MockResults) Reset(arg0 ident.ID)

Reset mocks base method

func (*MockResults) Size 

func (m *MockResults) Size() int

Size mocks base method

type MockResultsMockRecorder 

type MockResultsMockRecorder struct {
	// contains filtered or unexported fields
}

MockResultsMockRecorder is the mock recorder for MockResults

func (*MockResultsMockRecorder) AddDocument added in v0.5.0 

func (mr *MockResultsMockRecorder) AddDocument(arg0 interface{}) *gomock.Call

AddDocument indicates an expected call of AddDocument

func (*MockResultsMockRecorder) AddIDAndTags added in v0.5.0 

func (mr *MockResultsMockRecorder) AddIDAndTags(arg0, arg1 interface{}) *gomock.Call

AddIDAndTags indicates an expected call of AddIDAndTags

func (*MockResultsMockRecorder) Finalize 

func (mr *MockResultsMockRecorder) Finalize() *gomock.Call

Finalize indicates an expected call of Finalize

func (*MockResultsMockRecorder) Map 

func (mr *MockResultsMockRecorder) Map() *gomock.Call

Map indicates an expected call of Map

func (*MockResultsMockRecorder) Namespace 

func (mr *MockResultsMockRecorder) Namespace() *gomock.Call

Namespace indicates an expected call of Namespace

func (*MockResultsMockRecorder) NoFinalize added in v0.6.0 

func (mr *MockResultsMockRecorder) NoFinalize() *gomock.Call

NoFinalize indicates an expected call of NoFinalize

func (*MockResultsMockRecorder) Reset 

func (mr *MockResultsMockRecorder) Reset(arg0 interface{}) *gomock.Call

Reset indicates an expected call of Reset

func (*MockResultsMockRecorder) Size 

func (mr *MockResultsMockRecorder) Size() *gomock.Call

Size indicates an expected call of Size

type OnIndexSeries 

type OnIndexSeries interface {
	// OnIndexSuccess is executed when an entry is successfully indexed. The
	// provided value for `blockStart` is the blockStart for which the write
	// was indexed.
	OnIndexSuccess(blockStart xtime.UnixNano)

	// OnIndexFinalize is executed when the index no longer holds any references
	// to the provided resources. It can be used to cleanup any resources held
	// during the course of indexing. `blockStart` is the startTime of the index
	// block for which the write was attempted.
	OnIndexFinalize(blockStart xtime.UnixNano)
}

OnIndexSeries provides a set of callback hooks to allow the reverse index to do lifecycle management of any resources retained during indexing.

type Options 

type Options interface {
	// Validate validates assumptions baked into the code.
	Validate() error

	// SetIndexInsertMode sets the index insert mode (sync/async).
	SetInsertMode(value InsertMode) Options

	// IndexInsertMode returns the index's insert mode (sync/async).
	InsertMode() InsertMode

	// SetClockOptions sets the clock options.
	SetClockOptions(value clock.Options) Options

	// ClockOptions returns the clock options.
	ClockOptions() clock.Options

	// SetInstrumentOptions sets the instrument options.
	SetInstrumentOptions(value instrument.Options) Options

	// InstrumentOptions returns the instrument options.
	InstrumentOptions() instrument.Options

	// SetSegmentBuilderOptions sets the mem segment options.
	SetSegmentBuilderOptions(value builder.Options) Options

	// SegmentBuilderOptions returns the mem segment options.
	SegmentBuilderOptions() builder.Options

	// SetMemSegmentOptions sets the mem segment options.
	SetMemSegmentOptions(value mem.Options) Options

	// MemSegmentOptions returns the mem segment options.
	MemSegmentOptions() mem.Options

	// SetFSTSegmentOptions sets the fst segment options.
	SetFSTSegmentOptions(value fst.Options) Options

	// FSTSegmentOptions returns the fst segment options.
	FSTSegmentOptions() fst.Options

	// SetIdentifierPool sets the identifier pool.
	SetIdentifierPool(value ident.Pool) Options

	// IdentifierPool returns the identifier pool.
	IdentifierPool() ident.Pool

	// SetCheckedBytesPool sets the checked bytes pool.
	SetCheckedBytesPool(value pool.CheckedBytesPool) Options

	// CheckedBytesPool returns the checked bytes pool.
	CheckedBytesPool() pool.CheckedBytesPool

	// SetResultsPool updates the results pool.
	SetResultsPool(values ResultsPool) Options

	// ResultsPool returns the results pool.
	ResultsPool() ResultsPool

	// SetDocumentArrayPool sets the document array pool.
	SetDocumentArrayPool(value doc.DocumentArrayPool) Options

	// DocumentArrayPool returns the document array pool.
	DocumentArrayPool() doc.DocumentArrayPool

	// SetForegroundCompactionPlannerOptions sets the compaction planner options.
	SetForegroundCompactionPlannerOptions(v compaction.PlannerOptions) Options

	// ForegroundCompactionPlannerOptions returns the compaction planner options.
	ForegroundCompactionPlannerOptions() compaction.PlannerOptions

	// SetBackgroundCompactionPlannerOptions sets the compaction planner options.
	SetBackgroundCompactionPlannerOptions(v compaction.PlannerOptions) Options

	// BackgroundCompactionPlannerOptions returns the compaction planner options.
	BackgroundCompactionPlannerOptions() compaction.PlannerOptions

	// SetPostingsListCache sets the postings list cache.
	SetPostingsListCache(value *PostingsListCache) Options

	// PostingsListCache returns the postings list cache.
	PostingsListCache() *PostingsListCache

	// SetReadThroughSegmentOptions sets the read through segment cache options.
	SetReadThroughSegmentOptions(value ReadThroughSegmentOptions) Options

	// ReadThroughSegmentOptions returns the read through segment cache options.
	ReadThroughSegmentOptions() ReadThroughSegmentOptions
}

Options control the Indexing knobs.

func NewOptions 

func NewOptions() Options

NewOptions returns a new Options object with default properties.

type PatternType added in v0.6.0 

type PatternType int

PatternType is an enum for the various pattern types. It allows us separate them logically within the cache. 

const (
	// PatternTypeRegexp indicates that the pattern is of type regexp.
	PatternTypeRegexp PatternType = iota
	// PatternTypeTerm indicates that the pattern is of type term.
	PatternTypeTerm
)

type PostingsListCache added in v0.6.0 

type PostingsListCache struct {
	sync.Mutex
	// contains filtered or unexported fields
}

PostingsListCache implements an LRU for caching queries and their results.

func (*PostingsListCache) GetRegexp added in v0.6.0 

func (q *PostingsListCache) GetRegexp(
	segmentUUID uuid.UUID,
	field string,
	pattern string,
) (postings.List, bool)

GetRegexp returns the cached results for the provided regexp query, if any.

func (*PostingsListCache) GetTerm added in v0.6.0 

func (q *PostingsListCache) GetTerm(
	segmentUUID uuid.UUID,
	field string,
	pattern string,
) (postings.List, bool)

GetTerm returns the cached results for the provided term query, if any.

func (*PostingsListCache) PurgeSegment added in v0.6.0 

func (q *PostingsListCache) PurgeSegment(segmentUUID uuid.UUID)

PurgeSegment removes all postings lists associated with the specified segment from the cache.

func (*PostingsListCache) PutRegexp added in v0.6.0 

func (q *PostingsListCache) PutRegexp(
	segmentUUID uuid.UUID,
	field string,
	pattern string,
	pl postings.List,
)

PutRegexp updates the LRU with the result of the regexp query.

func (*PostingsListCache) PutTerm added in v0.6.0 

func (q *PostingsListCache) PutTerm(
	segmentUUID uuid.UUID,
	field string,
	pattern string,
	pl postings.List,
)

PutTerm updates the LRU with the result of the term query.

func (*PostingsListCache) Report added in v0.6.0 

func (q *PostingsListCache) Report()

Report will emit metrics about the status of the cache.

type PostingsListCacheOptions added in v0.6.0 

type PostingsListCacheOptions struct {
	InstrumentOptions instrument.Options
}

PostingsListCacheOptions is the options struct for the query cache.

type Query 

type Query struct {
	idx.Query
}

Query is a rich end user query to describe a set of constraints on required IDs.

type QueryMatcher 

type QueryMatcher interface {
	gomock.Matcher
}

QueryMatcher is a gomock.Matcher that matches index.Query

func NewQueryMatcher 

func NewQueryMatcher(q Query) QueryMatcher

NewQueryMatcher returns a new QueryMatcher

type QueryOptions 

type QueryOptions struct {
	StartInclusive time.Time
	EndExclusive   time.Time
	Limit          int
}

QueryOptions enables users to specify constraints on query execution.

func (QueryOptions) LimitExceeded added in v0.5.0 

func (o QueryOptions) LimitExceeded(size int) bool

LimitExceeded returns whether a given size exceeds the limit the query options imposes, if it is enabled.

type QueryResults 

type QueryResults struct {
	Results    Results
	Exhaustive bool
}

QueryResults is the collection of results for a query.

type ReadThroughSegment added in v0.6.0 

type ReadThroughSegment struct {
	segment.Segment
	sync.RWMutex
	// contains filtered or unexported fields
}

ReadThroughSegment wraps a segment with a postings list cache so that queries can be transparently cached in a read through manner. In addition, the postings lists returned by the segments may not be safe to use once the underlying segments are closed due to the postings lists pointing into the segments mmap'd region. As a result, the close method of the ReadThroughSegment will make sure that the cache is purged of all the segments postings lists before the segment itself is closed.

func (*ReadThroughSegment) Close added in v0.6.0 

func (r *ReadThroughSegment) Close() error

Close purges all entries in the cache associated with this segment, and then closes the underlying segment.

func (*ReadThroughSegment) Reader added in v0.6.0 

func (r *ReadThroughSegment) Reader() (index.Reader, error)

Reader returns a read through reader for the read through segment.

type ReadThroughSegmentOptions added in v0.6.0 

type ReadThroughSegmentOptions struct {
	// Whether the postings list for regexp queries should be cached.
	CacheRegexp bool
	// Whether the postings list for term queries should be cached.
	CacheTerms bool
}

ReadThroughSegmentOptions is the options struct for the ReadThroughSegment.

type Results 

type Results interface {
	// Namespace returns the namespace associated with the result.
	Namespace() ident.ID

	// Map returns a map from seriesID -> seriesTags, comprising index results.
	Map() *ResultsMap

	// Reset resets the Results object to initial state.
	Reset(nsID ident.ID)

	// Finalize releases any resources held by the Results object,
	// including returning it to a backing pool.
	Finalize()

	// NoFinalize marks the Results such that a subsequent call to Finalize() will
	// be a no-op and will not return the object to the pool or release any of its
	// resources.
	NoFinalize()

	// Size returns the number of IDs tracked.
	Size() int

	// Add converts the provided document to a metric and adds it to the results.
	// This method makes a copy of the bytes backing the document, so the original
	// may be modified after this function returns without affecting the results map.
	// NB: it returns a bool to indicate if the doc was added (it won't be added
	// if it already existed in the ResultsMap).
	AddDocument(
		document doc.Document,
	) (added bool, size int, err error)

	// AddIDAndTagsOrFinalize adds ID and tags to the results and takes ownership
	// if it does not exist, otherwise if it's already contained it returns added
	// false.
	AddIDAndTags(
		id ident.ID,
		tags ident.Tags,
	) (added bool, size int, err error)
}

Results is a collection of results for a query.

func NewResults 

func NewResults(opts Options) Results

NewResults returns a new results object.

type ResultsAllocator 

type ResultsAllocator func() Results

ResultsAllocator allocates Results types.

type ResultsMap 

type ResultsMap struct {
	// contains filtered or unexported fields
}

ResultsMap uses the genny package to provide a generic hash map that can be specialized by running the following command from this root of the repository: ``` make hashmap-gen pkg=outpkg key_type=Type value_type=Type out_dir=/tmp ``` Or if you would like to use bytes or ident.ID as keys you can use the partially specialized maps to generate your own maps as well: ``` make byteshashmap-gen pkg=outpkg value_type=Type out_dir=/tmp make idhashmap-gen pkg=outpkg value_type=Type out_dir=/tmp ``` This will output to stdout the generated source file to use for your map. It uses linear probing by incrementing the number of the hash created when hashing the identifier if there is a collision. ResultsMap is a value type and not an interface to allow for less painful upgrades when adding/removing methods, it is not likely to need mocking so an interface would not be super useful either.

func (*ResultsMap) Contains 

func (m *ResultsMap) Contains(k ident.ID) bool

Contains returns true if value exists for key, false otherwise, it is shorthand for a call to Get that doesn't return the value.

func (*ResultsMap) Delete 

func (m *ResultsMap) Delete(k ident.ID)

Delete will remove a value set in the map for the specified key.

func (*ResultsMap) Get 

func (m *ResultsMap) Get(k ident.ID) (ident.Tags, bool)

Get returns a value in the map for an identifier if found.

func (*ResultsMap) Iter 

func (m *ResultsMap) Iter() map[ResultsMapHash]ResultsMapEntry

Iter provides the underlying map to allow for using a native Go for loop to iterate the map, however callers should only ever read and not write the map.

func (*ResultsMap) Len 

func (m *ResultsMap) Len() int

Len returns the number of map entries in the map.

func (*ResultsMap) Reallocate 

func (m *ResultsMap) Reallocate()

Reallocate will avoid deleting all keys and reallocate a new map, this is useful if you believe you have a large map and will not need to grow back to a similar size.

func (*ResultsMap) Reset 

func (m *ResultsMap) Reset()

Reset will reset the map by simply deleting all keys to avoid allocating a new map.

func (*ResultsMap) Set 

func (m *ResultsMap) Set(k ident.ID, v ident.Tags)

Set will set the value for an identifier.

func (*ResultsMap) SetUnsafe 

func (m *ResultsMap) SetUnsafe(k ident.ID, v ident.Tags, opts ResultsMapSetUnsafeOptions)

SetUnsafe will set the value for an identifier with unsafe options for how the map treats the key.

type ResultsMapCopyFn 

type ResultsMapCopyFn func(ident.ID) ident.ID

ResultsMapCopyFn is the copy key function to execute when copying the key.

type ResultsMapEntry 

type ResultsMapEntry struct {
	// contains filtered or unexported fields
}

ResultsMapEntry is an entry in the map, this is public to support iterating over the map using a native Go for loop.

func (ResultsMapEntry) Key 

func (e ResultsMapEntry) Key() ident.ID

Key returns the map entry key.

func (ResultsMapEntry) Value 

func (e ResultsMapEntry) Value() ident.Tags

Value returns the map entry value.

type ResultsMapEqualsFn 

type ResultsMapEqualsFn func(ident.ID, ident.ID) bool

ResultsMapEqualsFn is the equals key function to execute when detecting equality of a key.

type ResultsMapFinalizeFn 

type ResultsMapFinalizeFn func(ident.ID)

ResultsMapFinalizeFn is the finalize key function to execute when finished with a key.

type ResultsMapHash 

type ResultsMapHash uint64

ResultsMapHash is the hash for a given map entry, this is public to support iterating over the map using a native Go for loop.

type ResultsMapHashFn 

type ResultsMapHashFn func(ident.ID) ResultsMapHash

ResultsMapHashFn is the hash function to execute when hashing a key.

type ResultsMapSetUnsafeOptions 

type ResultsMapSetUnsafeOptions struct {
	NoCopyKey     bool
	NoFinalizeKey bool
}

ResultsMapSetUnsafeOptions is a set of options to use when setting a value with the SetUnsafe method.

type ResultsPool 

type ResultsPool interface {
	// Init initialized the results pool.
	Init(alloc ResultsAllocator)

	// Get retrieves a Results object for use.
	Get() Results

	// Put returns the provide value to the pool.
	Put(value Results)
}

ResultsPool allows users to pool `Results` types.

func NewResultsPool 

func NewResultsPool(opts pool.ObjectPoolOptions) ResultsPool

NewResultsPool creates a new ResultsPool.

type WriteBatch 

type WriteBatch struct {
	// contains filtered or unexported fields
}

WriteBatch is a batch type that allows for building of a slice of documents with metadata in a separate slice, this allows the documents slice to be passed to the segment to batch insert without having to copy into a buffer again.

func NewWriteBatch 

func NewWriteBatch(opts WriteBatchOptions) *WriteBatch

NewWriteBatch creates a new write batch.

func (*WriteBatch) Append 

func (b *WriteBatch) Append(
	entry WriteBatchEntry,
	doc doc.Document,
)

Append appends an entry with accompanying document.

func (*WriteBatch) AppendAll added in v0.5.0 

func (b *WriteBatch) AppendAll(from *WriteBatch)

AppendAll appends all entries from another batch to this batch and ensures they share the same result struct.

func (*WriteBatch) ForEach 

func (b *WriteBatch) ForEach(fn ForEachWriteBatchEntryFn)

ForEach allows a caller to perform an operation for each batch entry.

func (*WriteBatch) ForEachUnmarkedBatchByBlockStart 

func (b *WriteBatch) ForEachUnmarkedBatchByBlockStart(
	fn ForEachWriteBatchByBlockStartFn,
)

ForEachUnmarkedBatchByBlockStart allows a caller to perform an operation with reference to a restricted set of the write batch for each unique block start for entries that have not been marked completed yet. The underlying batch returned is simply the current batch but with updated subslices to the relevant entries and documents that are restored at the end of `fn` being applied. NOTE: This means `fn` cannot perform any asynchronous work that uses the arguments passed to it as the args will be invalid at the synchronous execution of `fn`.

func (*WriteBatch) Len 

func (b *WriteBatch) Len() int

Len returns the length of the batch.

func (*WriteBatch) Less 

func (b *WriteBatch) Less(i, j int) bool

Less returns whether an entry appears before another depending on the type of sort.

func (*WriteBatch) MarkUnmarkedEntriesError 

func (b *WriteBatch) MarkUnmarkedEntriesError(err error)

MarkUnmarkedEntriesError marks all unmarked entries as error.

func (*WriteBatch) MarkUnmarkedEntriesSuccess 

func (b *WriteBatch) MarkUnmarkedEntriesSuccess()

MarkUnmarkedEntriesSuccess marks all unmarked entries as success.

func (*WriteBatch) MarkUnmarkedEntryError 

func (b *WriteBatch) MarkUnmarkedEntryError(
	err error,
	idx int,
)

MarkUnmarkedEntryError marks an unmarked entry at index as error.

func (*WriteBatch) NumErrs 

func (b *WriteBatch) NumErrs() int

NumErrs returns the number of errors encountered by the batch.

func (*WriteBatch) PendingDocs 

func (b *WriteBatch) PendingDocs() []doc.Document

PendingDocs returns all the docs in this batch that are unmarked.

func (*WriteBatch) PendingEntries 

func (b *WriteBatch) PendingEntries() []WriteBatchEntry

PendingEntries returns all the entries in this batch that are unmarked.

func (*WriteBatch) Reset 

func (b *WriteBatch) Reset()

Reset resets the batch for use.

func (*WriteBatch) SortByEnqueued 

func (b *WriteBatch) SortByEnqueued()

SortByEnqueued sorts the entries and documents back to the sort order they were enqueued as.

func (*WriteBatch) SortByUnmarkedAndIndexBlockStart 

func (b *WriteBatch) SortByUnmarkedAndIndexBlockStart()

SortByUnmarkedAndIndexBlockStart sorts the batch by unmarked first and then by index block start time.

func (*WriteBatch) Swap 

func (b *WriteBatch) Swap(i, j int)

Swap will swap two entries and the corresponding docs.

type WriteBatchEntry 

type WriteBatchEntry struct {
	// Timestamp is the timestamp that this entry should be indexed for
	Timestamp time.Time
	// OnIndexSeries is a listener/callback for when this entry is marked done
	// it is set to nil when the entry is marked done
	OnIndexSeries OnIndexSeries
	// EnqueuedAt is the timestamp that this entry was enqueued for indexing
	// so that we can calculate the latency it takes to index the entry
	EnqueuedAt time.Time
	// contains filtered or unexported fields
}

WriteBatchEntry represents the metadata accompanying the document that is being inserted.

func (WriteBatchEntry) Result 

func (e WriteBatchEntry) Result() WriteBatchEntryResult

Result returns the result for this entry.

type WriteBatchEntryResult 

type WriteBatchEntryResult struct {
	Done bool
	Err  error
}

WriteBatchEntryResult represents a result.

type WriteBatchOptions 

type WriteBatchOptions struct {
	InitialCapacity int
	IndexBlockSize  time.Duration
}

WriteBatchOptions is a set of options required for a write batch.

type WriteBatchResult 

type WriteBatchResult struct {
	NumSuccess int64
	NumError   int64
}

WriteBatchResult returns statistics about the WriteBatch execution.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.