index

package
v0.4.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 10, 2018 License: Apache-2.0 Imports: 25 Imported by: 22

Documentation

Overview

Package index is a generated GoMock package.

Index

Constants

This section is empty.

Variables

View Source
var (
	// ReservedFieldNameID is the field name used to index the ID in the
	// m3ninx subsytem.
	ReservedFieldNameID = doc.IDReservedFieldName
)

Functions

func NewBootstrapResultMutableSegmentAllocator

func NewBootstrapResultMutableSegmentAllocator(
	opts Options,
) result.MutableSegmentAllocator

NewBootstrapResultMutableSegmentAllocator returns a default mutable segment allocator for a bootstrap result index block given index options.

Types

type Block

type Block interface {
	// StartTime returns the start time of the period this Block indexes.
	StartTime() time.Time

	// EndTime returns the end time of the period this Block indexes.
	EndTime() time.Time

	// WriteBatch writes a batch of provided entries.
	WriteBatch(inserts *WriteBatch) (WriteBatchResult, error)

	// Query resolves the given query into known IDs.
	Query(
		query Query,
		opts QueryOptions,
		results Results,
	) (exhaustive bool, err error)

	// AddResults adds bootstrap results to the block, if c.
	AddResults(results result.IndexBlock) error

	// Tick does internal house keeping operations.
	Tick(c context.Cancellable, tickStart time.Time) (BlockTickResult, error)

	// Seal prevents the block from taking any more writes, but, it still permits
	// addition of segments via Bootstrap().
	Seal() error

	// IsSealed returns whether this block was sealed.
	IsSealed() bool

	// NeedsMutableSegmentsEvicted returns whether this block has any mutable segments
	// that are not-empty and sealed.
	// A sealed non-empty mutable segment needs to get evicted from memory as
	// soon as it can be to reduce memory footprint.
	NeedsMutableSegmentsEvicted() bool

	// EvictMutableSegments closes any mutable segments, this is only applicable
	// valid to be called once the block and hence mutable segments are sealed.
	// It is expected that results have been added to the block that covers any
	// data the mutable segments should have held at this time.
	EvictMutableSegments() (EvictMutableSegmentResults, error)

	// Close will release any held resources and close the Block.
	Close() error
}

Block represents a collection of segments. Each `Block` is a complete reverse index for a period of time defined by [StartTime, EndTime).

func NewBlock

func NewBlock(
	startTime time.Time,
	md namespace.Metadata,
	opts Options,
) (Block, error)

NewBlock returns a new Block, representing a complete reverse index for the duration of time specified. It is backed by one or more segments.

type BlockTickResult

type BlockTickResult struct {
	NumSegments int64
	NumDocs     int64
}

BlockTickResult returns statistics about tick.

type EvictMutableSegmentResults

type EvictMutableSegmentResults struct {
	NumMutableSegments int64
	NumDocs            int64
}

EvictMutableSegmentResults returns statistics about the EvictMutableSegments execution.

func (*EvictMutableSegmentResults) Add

Add adds the provided results to the receiver.

type ForEachWriteBatchByBlockStartFn

type ForEachWriteBatchByBlockStartFn func(
	blockStart time.Time,
	batch *WriteBatch,
)

ForEachWriteBatchByBlockStartFn allows a caller to perform an operation with reference to a restricted set of the write batch for each unique block start.

type ForEachWriteBatchEntryFn

type ForEachWriteBatchEntryFn func(
	idx int,
	entry WriteBatchEntry,
	doc doc.Document,
	result WriteBatchEntryResult,
)

ForEachWriteBatchEntryFn allows a caller to perform an operation for each batch entry.

type InsertMode

type InsertMode byte

InsertMode specifies whether inserts are synchronous or asynchronous.

const (
	InsertSync InsertMode = iota
	InsertAsync
)

nolint

type MockBlock

type MockBlock struct {
	// contains filtered or unexported fields
}

MockBlock is a mock of Block interface

func NewMockBlock

func NewMockBlock(ctrl *gomock.Controller) *MockBlock

NewMockBlock creates a new mock instance

func (*MockBlock) AddResults

func (m *MockBlock) AddResults(arg0 result.IndexBlock) error

AddResults mocks base method

func (*MockBlock) Close

func (m *MockBlock) Close() error

Close mocks base method

func (*MockBlock) EXPECT

func (m *MockBlock) EXPECT() *MockBlockMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockBlock) EndTime

func (m *MockBlock) EndTime() time.Time

EndTime mocks base method

func (*MockBlock) EvictMutableSegments

func (m *MockBlock) EvictMutableSegments() (EvictMutableSegmentResults, error)

EvictMutableSegments mocks base method

func (*MockBlock) IsSealed

func (m *MockBlock) IsSealed() bool

IsSealed mocks base method

func (*MockBlock) NeedsMutableSegmentsEvicted

func (m *MockBlock) NeedsMutableSegmentsEvicted() bool

NeedsMutableSegmentsEvicted mocks base method

func (*MockBlock) Query

func (m *MockBlock) Query(arg0 Query, arg1 QueryOptions, arg2 Results) (bool, error)

Query mocks base method

func (*MockBlock) Seal

func (m *MockBlock) Seal() error

Seal mocks base method

func (*MockBlock) StartTime

func (m *MockBlock) StartTime() time.Time

StartTime mocks base method

func (*MockBlock) Tick

func (m *MockBlock) Tick(arg0 context.Cancellable, arg1 time.Time) (BlockTickResult, error)

Tick mocks base method

func (*MockBlock) WriteBatch

func (m *MockBlock) WriteBatch(arg0 *WriteBatch) (WriteBatchResult, error)

WriteBatch mocks base method

type MockBlockMockRecorder

type MockBlockMockRecorder struct {
	// contains filtered or unexported fields
}

MockBlockMockRecorder is the mock recorder for MockBlock

func (*MockBlockMockRecorder) AddResults

func (mr *MockBlockMockRecorder) AddResults(arg0 interface{}) *gomock.Call

AddResults indicates an expected call of AddResults

func (*MockBlockMockRecorder) Close

func (mr *MockBlockMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

func (*MockBlockMockRecorder) EndTime

func (mr *MockBlockMockRecorder) EndTime() *gomock.Call

EndTime indicates an expected call of EndTime

func (*MockBlockMockRecorder) EvictMutableSegments

func (mr *MockBlockMockRecorder) EvictMutableSegments() *gomock.Call

EvictMutableSegments indicates an expected call of EvictMutableSegments

func (*MockBlockMockRecorder) IsSealed

func (mr *MockBlockMockRecorder) IsSealed() *gomock.Call

IsSealed indicates an expected call of IsSealed

func (*MockBlockMockRecorder) NeedsMutableSegmentsEvicted

func (mr *MockBlockMockRecorder) NeedsMutableSegmentsEvicted() *gomock.Call

NeedsMutableSegmentsEvicted indicates an expected call of NeedsMutableSegmentsEvicted

func (*MockBlockMockRecorder) Query

func (mr *MockBlockMockRecorder) Query(arg0, arg1, arg2 interface{}) *gomock.Call

Query indicates an expected call of Query

func (*MockBlockMockRecorder) Seal

func (mr *MockBlockMockRecorder) Seal() *gomock.Call

Seal indicates an expected call of Seal

func (*MockBlockMockRecorder) StartTime

func (mr *MockBlockMockRecorder) StartTime() *gomock.Call

StartTime indicates an expected call of StartTime

func (*MockBlockMockRecorder) Tick

func (mr *MockBlockMockRecorder) Tick(arg0, arg1 interface{}) *gomock.Call

Tick indicates an expected call of Tick

func (*MockBlockMockRecorder) WriteBatch

func (mr *MockBlockMockRecorder) WriteBatch(arg0 interface{}) *gomock.Call

WriteBatch indicates an expected call of WriteBatch

type MockOnIndexSeries

type MockOnIndexSeries struct {
	// contains filtered or unexported fields
}

MockOnIndexSeries is a mock of OnIndexSeries interface

func NewMockOnIndexSeries

func NewMockOnIndexSeries(ctrl *gomock.Controller) *MockOnIndexSeries

NewMockOnIndexSeries creates a new mock instance

func (*MockOnIndexSeries) EXPECT

EXPECT returns an object that allows the caller to indicate expected use

func (*MockOnIndexSeries) OnIndexFinalize

func (m *MockOnIndexSeries) OnIndexFinalize(arg0 time0.UnixNano)

OnIndexFinalize mocks base method

func (*MockOnIndexSeries) OnIndexSuccess

func (m *MockOnIndexSeries) OnIndexSuccess(arg0 time0.UnixNano)

OnIndexSuccess mocks base method

type MockOnIndexSeriesMockRecorder

type MockOnIndexSeriesMockRecorder struct {
	// contains filtered or unexported fields
}

MockOnIndexSeriesMockRecorder is the mock recorder for MockOnIndexSeries

func (*MockOnIndexSeriesMockRecorder) OnIndexFinalize

func (mr *MockOnIndexSeriesMockRecorder) OnIndexFinalize(arg0 interface{}) *gomock.Call

OnIndexFinalize indicates an expected call of OnIndexFinalize

func (*MockOnIndexSeriesMockRecorder) OnIndexSuccess

func (mr *MockOnIndexSeriesMockRecorder) OnIndexSuccess(arg0 interface{}) *gomock.Call

OnIndexSuccess indicates an expected call of OnIndexSuccess

type MockResults

type MockResults struct {
	// contains filtered or unexported fields
}

MockResults is a mock of Results interface

func NewMockResults

func NewMockResults(ctrl *gomock.Controller) *MockResults

NewMockResults creates a new mock instance

func (*MockResults) Add

func (m *MockResults) Add(arg0 doc.Document) (bool, int, error)

Add mocks base method

func (*MockResults) EXPECT

func (m *MockResults) EXPECT() *MockResultsMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockResults) Finalize

func (m *MockResults) Finalize()

Finalize mocks base method

func (*MockResults) Map

func (m *MockResults) Map() *ResultsMap

Map mocks base method

func (*MockResults) Namespace

func (m *MockResults) Namespace() ident.ID

Namespace mocks base method

func (*MockResults) Reset

func (m *MockResults) Reset(arg0 ident.ID)

Reset mocks base method

func (*MockResults) Size

func (m *MockResults) Size() int

Size mocks base method

type MockResultsMockRecorder

type MockResultsMockRecorder struct {
	// contains filtered or unexported fields
}

MockResultsMockRecorder is the mock recorder for MockResults

func (*MockResultsMockRecorder) Add

func (mr *MockResultsMockRecorder) Add(arg0 interface{}) *gomock.Call

Add indicates an expected call of Add

func (*MockResultsMockRecorder) Finalize

func (mr *MockResultsMockRecorder) Finalize() *gomock.Call

Finalize indicates an expected call of Finalize

func (*MockResultsMockRecorder) Map

func (mr *MockResultsMockRecorder) Map() *gomock.Call

Map indicates an expected call of Map

func (*MockResultsMockRecorder) Namespace

func (mr *MockResultsMockRecorder) Namespace() *gomock.Call

Namespace indicates an expected call of Namespace

func (*MockResultsMockRecorder) Reset

func (mr *MockResultsMockRecorder) Reset(arg0 interface{}) *gomock.Call

Reset indicates an expected call of Reset

func (*MockResultsMockRecorder) Size

func (mr *MockResultsMockRecorder) Size() *gomock.Call

Size indicates an expected call of Size

type OnIndexSeries

type OnIndexSeries interface {
	// OnIndexSuccess is executed when an entry is successfully indexed. The
	// provided value for `blockStart` is the blockStart for which the write
	// was indexed.
	OnIndexSuccess(blockStart xtime.UnixNano)

	// OnIndexFinalize is executed when the index no longer holds any references
	// to the provided resources. It can be used to cleanup any resources held
	// during the course of indexing. `blockStart` is the startTime of the index
	// block for which the write was attempted.
	OnIndexFinalize(blockStart xtime.UnixNano)
}

OnIndexSeries provides a set of callback hooks to allow the reverse index to do lifecycle management of any resources retained during indexing.

type Options

type Options interface {
	// Validate validates assumptions baked into the code.
	Validate() error

	// SetIndexInsertMode sets the index insert mode (sync/async).
	SetInsertMode(value InsertMode) Options

	// IndexInsertMode returns the index's insert mode (sync/async).
	InsertMode() InsertMode

	// SetClockOptions sets the clock options.
	SetClockOptions(value clock.Options) Options

	// ClockOptions returns the clock options.
	ClockOptions() clock.Options

	// SetInstrumentOptions sets the instrument options.
	SetInstrumentOptions(value instrument.Options) Options

	// InstrumentOptions returns the instrument options.
	InstrumentOptions() instrument.Options

	// SetMemSegmentOptions sets the mem segment options.
	SetMemSegmentOptions(value mem.Options) Options

	// MemSegmentOptions returns the mem segment options.
	MemSegmentOptions() mem.Options

	// SetIdentifierPool sets the identifier pool.
	SetIdentifierPool(value ident.Pool) Options

	// IdentifierPool returns the identifier pool.
	IdentifierPool() ident.Pool

	// SetCheckedBytesPool sets the checked bytes pool.
	SetCheckedBytesPool(value pool.CheckedBytesPool) Options

	// CheckedBytesPool returns the checked bytes pool.
	CheckedBytesPool() pool.CheckedBytesPool

	// SetResultsPool updates the results pool.
	SetResultsPool(values ResultsPool) Options

	// ResultsPool returns the results pool.
	ResultsPool() ResultsPool
}

Options control the Indexing knobs.

func NewOptions

func NewOptions() Options

NewOptions returns a new index.Options object with default properties.

type Query

type Query struct {
	idx.Query
}

Query is a rich end user query to describe a set of constraints on required IDs.

type QueryMatcher

type QueryMatcher interface {
	gomock.Matcher
}

QueryMatcher is a gomock.Matcher that matches index.Query

func NewQueryMatcher

func NewQueryMatcher(q Query) QueryMatcher

NewQueryMatcher returns a new QueryMatcher

type QueryOptions

type QueryOptions struct {
	StartInclusive time.Time
	EndExclusive   time.Time
	Limit          int
}

QueryOptions enables users to specify constraints on query execution.

type QueryResults

type QueryResults struct {
	Results    Results
	Exhaustive bool
}

QueryResults is the collection of results for a query.

type Results

type Results interface {
	// Namespace returns the namespace associated with the result.
	Namespace() ident.ID

	// Map returns a map from seriesID -> seriesTags, comprising index results.
	Map() *ResultsMap

	// Reset resets the Results object to initial state.
	Reset(nsID ident.ID)

	// Finalize releases any resources held by the Results object,
	// including returning it to a backing pool.
	Finalize()

	// Size returns the number of IDs tracked.
	Size() int

	// Add converts the provided document to a metric and adds it to the results.
	// This method makes a copy of the bytes backing the document, so the original
	// may be modified after this function returns without affecting the results map.
	// NB: it returns a bool to indicate if the doc was added (it won't be added
	// if it already existed in the ResultsMap).
	Add(d doc.Document) (added bool, size int, err error)
}

Results is a collection of results for a query.

func NewResults

func NewResults(opts Options) Results

NewResults returns a new results object.

type ResultsAllocator

type ResultsAllocator func() Results

ResultsAllocator allocates Results types.

type ResultsMap

type ResultsMap struct {
	// contains filtered or unexported fields
}

ResultsMap uses the genny package to provide a generic hash map that can be specialized by running the following command from this root of the repository: ``` make hashmap-gen pkg=outpkg key_type=Type value_type=Type out_dir=/tmp ``` Or if you would like to use bytes or ident.ID as keys you can use the partially specialized maps to generate your own maps as well: ``` make byteshashmap-gen pkg=outpkg value_type=Type out_dir=/tmp make idhashmap-gen pkg=outpkg value_type=Type out_dir=/tmp ``` This will output to stdout the generated source file to use for your map. It uses linear probing by incrementing the number of the hash created when hashing the identifier if there is a collision. ResultsMap is a value type and not an interface to allow for less painful upgrades when adding/removing methods, it is not likely to need mocking so an interface would not be super useful either.

func (*ResultsMap) Contains

func (m *ResultsMap) Contains(k ident.ID) bool

Contains returns true if value exists for key, false otherwise, it is shorthand for a call to Get that doesn't return the value.

func (*ResultsMap) Delete

func (m *ResultsMap) Delete(k ident.ID)

Delete will remove a value set in the map for the specified key.

func (*ResultsMap) Get

func (m *ResultsMap) Get(k ident.ID) (ident.Tags, bool)

Get returns a value in the map for an identifier if found.

func (*ResultsMap) Iter

Iter provides the underlying map to allow for using a native Go for loop to iterate the map, however callers should only ever read and not write the map.

func (*ResultsMap) Len

func (m *ResultsMap) Len() int

Len returns the number of map entries in the map.

func (*ResultsMap) Reallocate

func (m *ResultsMap) Reallocate()

Reallocate will avoid deleting all keys and reallocate a new map, this is useful if you believe you have a large map and will not need to grow back to a similar size.

func (*ResultsMap) Reset

func (m *ResultsMap) Reset()

Reset will reset the map by simply deleting all keys to avoid allocating a new map.

func (*ResultsMap) Set

func (m *ResultsMap) Set(k ident.ID, v ident.Tags)

Set will set the value for an identifier.

func (*ResultsMap) SetUnsafe

func (m *ResultsMap) SetUnsafe(k ident.ID, v ident.Tags, opts ResultsMapSetUnsafeOptions)

SetUnsafe will set the value for an identifier with unsafe options for how the map treats the key.

type ResultsMapCopyFn

type ResultsMapCopyFn func(ident.ID) ident.ID

ResultsMapCopyFn is the copy key function to execute when copying the key.

type ResultsMapEntry

type ResultsMapEntry struct {
	// contains filtered or unexported fields
}

ResultsMapEntry is an entry in the map, this is public to support iterating over the map using a native Go for loop.

func (ResultsMapEntry) Key

func (e ResultsMapEntry) Key() ident.ID

Key returns the map entry key.

func (ResultsMapEntry) Value

func (e ResultsMapEntry) Value() ident.Tags

Value returns the map entry value.

type ResultsMapEqualsFn

type ResultsMapEqualsFn func(ident.ID, ident.ID) bool

ResultsMapEqualsFn is the equals key function to execute when detecting equality of a key.

type ResultsMapFinalizeFn

type ResultsMapFinalizeFn func(ident.ID)

ResultsMapFinalizeFn is the finalize key function to execute when finished with a key.

type ResultsMapHash

type ResultsMapHash uint64

ResultsMapHash is the hash for a given map entry, this is public to support iterating over the map using a native Go for loop.

type ResultsMapHashFn

type ResultsMapHashFn func(ident.ID) ResultsMapHash

ResultsMapHashFn is the hash function to execute when hashing a key.

type ResultsMapSetUnsafeOptions

type ResultsMapSetUnsafeOptions struct {
	NoCopyKey     bool
	NoFinalizeKey bool
}

ResultsMapSetUnsafeOptions is a set of options to use when setting a value with the SetUnsafe method.

type ResultsPool

type ResultsPool interface {
	// Init initialized the results pool.
	Init(alloc ResultsAllocator)

	// Get retrieves a Results object for use.
	Get() Results

	// Put returns the provide value to the pool.
	Put(value Results)
}

ResultsPool allows users to pool `Results` types.

func NewResultsPool

func NewResultsPool(opts pool.ObjectPoolOptions) ResultsPool

NewResultsPool creates a new ResultsPool.

type WriteBatch

type WriteBatch struct {
	// contains filtered or unexported fields
}

WriteBatch is a batch type that allows for building of a slice of documents with metadata in a separate slice, this allows the documents slice to be passed to the segment to batch insert without having to copy into a buffer again.

func NewWriteBatch

func NewWriteBatch(opts WriteBatchOptions) *WriteBatch

NewWriteBatch creates a new write batch.

func (*WriteBatch) Append

func (b *WriteBatch) Append(
	entry WriteBatchEntry,
	doc doc.Document,
)

Append appends an entry with accompanying document.

func (*WriteBatch) ForEach

func (b *WriteBatch) ForEach(fn ForEachWriteBatchEntryFn)

ForEach allows a caller to perform an operation for each batch entry.

func (*WriteBatch) ForEachUnmarkedBatchByBlockStart

func (b *WriteBatch) ForEachUnmarkedBatchByBlockStart(
	fn ForEachWriteBatchByBlockStartFn,
)

ForEachUnmarkedBatchByBlockStart allows a caller to perform an operation with reference to a restricted set of the write batch for each unique block start for entries that have not been marked completed yet. The underlying batch returned is simply the current batch but with updated subslices to the relevant entries and documents that are restored at the end of `fn` being applied. NOTE: This means `fn` cannot perform any asynchronous work that uses the arguments passed to it as the args will be invalid at the synchronous execution of `fn`.

func (*WriteBatch) Len

func (b *WriteBatch) Len() int

Len returns the length of the batch.

func (*WriteBatch) Less

func (b *WriteBatch) Less(i, j int) bool

Less returns whether an entry appears before another depending on the type of sort.

func (*WriteBatch) MarkUnmarkedEntriesError

func (b *WriteBatch) MarkUnmarkedEntriesError(err error)

MarkUnmarkedEntriesError marks all unmarked entries as error.

func (*WriteBatch) MarkUnmarkedEntriesSuccess

func (b *WriteBatch) MarkUnmarkedEntriesSuccess()

MarkUnmarkedEntriesSuccess marks all unmarked entries as success.

func (*WriteBatch) MarkUnmarkedEntryError

func (b *WriteBatch) MarkUnmarkedEntryError(
	err error,
	idx int,
)

MarkUnmarkedEntryError marks an unmarked entry at index as error.

func (*WriteBatch) NumErrs

func (b *WriteBatch) NumErrs() int

NumErrs returns the number of errors encountered by the batch.

func (*WriteBatch) PendingDocs

func (b *WriteBatch) PendingDocs() []doc.Document

PendingDocs returns all the docs in this batch that are unmarked.

func (*WriteBatch) PendingEntries

func (b *WriteBatch) PendingEntries() []WriteBatchEntry

PendingEntries returns all the entries in this batch that are unmarked.

func (*WriteBatch) Reset

func (b *WriteBatch) Reset()

Reset resets the batch for use.

func (*WriteBatch) SortByEnqueued

func (b *WriteBatch) SortByEnqueued()

SortByEnqueued sorts the entries and documents back to the sort order they were enqueued as.

func (*WriteBatch) SortByUnmarkedAndIndexBlockStart

func (b *WriteBatch) SortByUnmarkedAndIndexBlockStart()

SortByUnmarkedAndIndexBlockStart sorts the batch by unmarked first and then by index block start time.

func (*WriteBatch) Swap

func (b *WriteBatch) Swap(i, j int)

Swap will swap two entries and the corresponding docs.

type WriteBatchEntry

type WriteBatchEntry struct {
	// Timestamp is the timestamp that this entry should be indexed for
	Timestamp time.Time
	// OnIndexSeries is a listener/callback for when this entry is marked done
	// it is set to nil when the entry is marked done
	OnIndexSeries OnIndexSeries
	// EnqueuedAt is the timestamp that this entry was enqueued for indexing
	// so that we can calculate the latency it takes to index the entry
	EnqueuedAt time.Time
	// contains filtered or unexported fields
}

WriteBatchEntry represents the metadata accompanying the document that is being inserted.

func (WriteBatchEntry) Result

Result returns the result for this entry.

type WriteBatchEntryResult

type WriteBatchEntryResult struct {
	Err error
}

WriteBatchEntryResult represents a result.

type WriteBatchOptions

type WriteBatchOptions struct {
	InitialCapacity int
	IndexBlockSize  time.Duration
}

WriteBatchOptions is a set of options required for a write batch.

type WriteBatchResult

type WriteBatchResult struct {
	NumSuccess int64
	NumError   int64
}

WriteBatchResult returns statistics about the WriteBatch execution.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL