result

package
v0.15.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 21, 2020 License: Apache-2.0 Imports: 18 Imported by: 28

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EstimateMapBytesSize added in v0.12.0 

func EstimateMapBytesSize(m *Map) int64

EstimateMapBytesSize estimates the size (in bytes) of the results map. It's only an estimate because its impossible to know if some of the references like the series name as well as tags are exclusive to this object or shared with other structures in memory.

Types

type CopyFn 

type CopyFn func(ident.ID) ident.ID

CopyFn is the copy key function to execute when copying the key.

type DataBootstrapResult 

type DataBootstrapResult interface {
	// Unfulfilled is the unfulfilled time ranges for the bootstrap.
	Unfulfilled() ShardTimeRanges

	// SetUnfulfilled sets the current unfulfilled shard time ranges.
	SetUnfulfilled(unfulfilled ShardTimeRanges)
}

DataBootstrapResult is the result of a bootstrap of series data.

func MergedDataBootstrapResult 

func MergedDataBootstrapResult(i, j DataBootstrapResult) DataBootstrapResult

MergedDataBootstrapResult returns a merged result of two bootstrap results. It is a mutating function that mutates the larger result by adding the smaller result to it and then finally returns the mutated result.

func NewDataBootstrapResult 

func NewDataBootstrapResult() DataBootstrapResult

NewDataBootstrapResult creates a new result.

type DatabaseSeriesBlocks 

type DatabaseSeriesBlocks struct {
	ID     ident.ID
	Tags   ident.Tags
	Blocks block.DatabaseSeriesBlocks
}

DatabaseSeriesBlocks represents a series of blocks and a associated series ID.

type DocumentsBuilderAllocator added in v0.15.0 

type DocumentsBuilderAllocator func() (segment.DocumentsBuilder, error)

DocumentsBuilderAllocator allocates a new DocumentsBuilder type when creating a bootstrap result to return to the index.

func NewDefaultDocumentsBuilderAllocator added in v0.15.0 

func NewDefaultDocumentsBuilderAllocator() DocumentsBuilderAllocator

NewDefaultDocumentsBuilderAllocator returns a default mutable segment allocator.

type EqualsFn 

type EqualsFn func(ident.ID, ident.ID) bool

EqualsFn is the equals key function to execute when detecting equality of a key.

type FinalizeFn 

type FinalizeFn func(ident.ID)

FinalizeFn is the finalize key function to execute when finished with a key.

type HashFn 

type HashFn func(ident.ID) MapHash

HashFn is the hash function to execute when hashing a key.

type IndexBlock 

type IndexBlock struct {
	// contains filtered or unexported fields
}

IndexBlock is an index block for a index volume type.

func NewIndexBlock 

func NewIndexBlock(
	segments []Segment,
	fulfilled ShardTimeRanges,
) IndexBlock

NewIndexBlock returns a new bootstrap index block result.

func (IndexBlock) Fulfilled 

func (b IndexBlock) Fulfilled() ShardTimeRanges

Fulfilled returns the fulfilled time ranges by this index block.

func (IndexBlock) Merged 

func (b IndexBlock) Merged(other IndexBlock) IndexBlock

Merged returns a new merged index block, currently it just appends the list of segments from the other index block and the caller merges as they see necessary.

func (IndexBlock) Segments 

func (b IndexBlock) Segments() []Segment

Segments returns the segments.

type IndexBlockByVolumeType added in v0.15.0 

type IndexBlockByVolumeType struct {
	// contains filtered or unexported fields
}

IndexBlockByVolumeType contains the bootstrap data structures for an index block by volume type.

func NewIndexBlockByVolumeType added in v0.15.0 

func NewIndexBlockByVolumeType(blockStart time.Time) IndexBlockByVolumeType

NewIndexBlockByVolumeType returns a new bootstrap index blocks by volume type result.

func (IndexBlockByVolumeType) BlockStart added in v0.15.0 

func (b IndexBlockByVolumeType) BlockStart() time.Time

BlockStart returns the block start.

func (IndexBlockByVolumeType) GetBlock added in v0.15.0 

func (b IndexBlockByVolumeType) GetBlock(volumeType persist.IndexVolumeType) (IndexBlock, bool)

GetBlock returns an IndexBlock for volumeType.

func (IndexBlockByVolumeType) Iter added in v0.15.0 

func (b IndexBlockByVolumeType) Iter() map[persist.IndexVolumeType]IndexBlock

Iter returns the underlying iterable map data.

func (IndexBlockByVolumeType) Merged added in v0.15.0 

func (b IndexBlockByVolumeType) Merged(other IndexBlockByVolumeType) IndexBlockByVolumeType

Merged returns a new merged index block by volume type. It merges the underlying index blocks together by index volume type.

func (IndexBlockByVolumeType) SetBlock added in v0.15.0 

func (b IndexBlockByVolumeType) SetBlock(volumeType persist.IndexVolumeType, block IndexBlock)

SetBlock sets an IndexBlock for volumeType.

type IndexBootstrapResult 

type IndexBootstrapResult interface {
	// IndexResults returns a map of all index block results.
	IndexResults() IndexResults

	// Unfulfilled is the unfulfilled time ranges for the bootstrap.
	Unfulfilled() ShardTimeRanges

	// SetUnfulfilled sets the current unfulfilled shard time ranges.
	SetUnfulfilled(unfulfilled ShardTimeRanges)

	// Add adds an index block result.
	Add(blocks IndexBlockByVolumeType, unfulfilled ShardTimeRanges)

	// NumSeries returns the total number of series across all segments.
	NumSeries() int
}

IndexBootstrapResult is the result of a bootstrap of series index metadata.

func MergedIndexBootstrapResult 

func MergedIndexBootstrapResult(i, j IndexBootstrapResult) IndexBootstrapResult

MergedIndexBootstrapResult returns a merged result of two bootstrap results. It is a mutating function that mutates the larger result by adding the smaller result to it and then finally returns the mutated result.

func NewIndexBootstrapResult 

func NewIndexBootstrapResult() IndexBootstrapResult

NewIndexBootstrapResult returns a new index bootstrap result.

type IndexBuilder added in v0.15.0 

type IndexBuilder struct {
	// contains filtered or unexported fields
}

IndexBuilder wraps a index segment builder w/ batching.

func NewIndexBuilder added in v0.15.0 

func NewIndexBuilder(builder segment.DocumentsBuilder) *IndexBuilder

NewIndexBuilder creates a wrapped locakble index seg builder.

func (*IndexBuilder) Builder added in v0.15.0 

func (b *IndexBuilder) Builder() segment.DocumentsBuilder

Builder returns the underlying index segment docs builder.

func (*IndexBuilder) FlushBatch added in v0.15.0 

func (b *IndexBuilder) FlushBatch(batch []doc.Document) ([]doc.Document, error)

FlushBatch flushes a batch of documents to the underlying segment builder.

type IndexResults 

type IndexResults map[xtime.UnixNano]IndexBlockByVolumeType

IndexResults is a set of index blocks indexed by block start.

func (IndexResults) Add 

func (r IndexResults) Add(blocks IndexBlockByVolumeType)

Add will add an index block to the collection, merging if one already exists.

func (IndexResults) AddBlockIfNotExists added in v0.15.0 

func (r IndexResults) AddBlockIfNotExists(
	t time.Time,
	idxopts namespace.IndexOptions,
)

AddBlockIfNotExists adds an index block if it does not already exist to the index results.

func (IndexResults) AddResults 

func (r IndexResults) AddResults(other IndexResults)

AddResults will add another set of index results to the collection, merging if index blocks already exists.

func (IndexResults) MarkFulfilled 

func (r IndexResults) MarkFulfilled(
	t time.Time,
	fulfilled ShardTimeRanges,
	indexVolumeType persist.IndexVolumeType,
	idxopts namespace.IndexOptions,
) error

MarkFulfilled will mark an index block as fulfilled, either partially or wholly as specified by the shard time ranges passed.

type Map 

type Map struct {
	// contains filtered or unexported fields
}

Map uses the genny package to provide a generic hash map that can be specialized by running the following command from this root of the repository: ``` make hashmap-gen pkg=outpkg key_type=Type value_type=Type out_dir=/tmp ``` Or if you would like to use bytes or ident.ID as keys you can use the partially specialized maps to generate your own maps as well: ``` make byteshashmap-gen pkg=outpkg value_type=Type out_dir=/tmp make idhashmap-gen pkg=outpkg value_type=Type out_dir=/tmp ``` This will output to stdout the generated source file to use for your map. It uses linear probing by incrementing the number of the hash created when hashing the identifier if there is a collision. Map is a value type and not an interface to allow for less painful upgrades when adding/removing methods, it is not likely to need mocking so an interface would not be super useful either.

func NewMap 

func NewMap(opts MapOptions) *Map

NewMap returns a new byte keyed map.

func (*Map) Contains 

func (m *Map) Contains(k ident.ID) bool

Contains returns true if value exists for key, false otherwise, it is shorthand for a call to Get that doesn't return the value.

func (*Map) Delete 

func (m *Map) Delete(k ident.ID)

Delete will remove a value set in the map for the specified key.

func (*Map) Get 

func (m *Map) Get(k ident.ID) (DatabaseSeriesBlocks, bool)

Get returns a value in the map for an identifier if found.

func (*Map) Iter 

func (m *Map) Iter() map[MapHash]MapEntry

Iter provides the underlying map to allow for using a native Go for loop to iterate the map, however callers should only ever read and not write the map.

func (*Map) Len 

func (m *Map) Len() int

Len returns the number of map entries in the map.

func (*Map) Reallocate 

func (m *Map) Reallocate()

Reallocate will avoid deleting all keys and reallocate a new map, this is useful if you believe you have a large map and will not need to grow back to a similar size.

func (*Map) Reset 

func (m *Map) Reset()

Reset will reset the map by simply deleting all keys to avoid allocating a new map.

func (*Map) Set 

func (m *Map) Set(k ident.ID, v DatabaseSeriesBlocks)

Set will set the value for an identifier.

func (*Map) SetUnsafe 

func (m *Map) SetUnsafe(k ident.ID, v DatabaseSeriesBlocks, opts SetUnsafeOptions)

SetUnsafe will set the value for an identifier with unsafe options for how the map treats the key.

type MapEntry 

type MapEntry struct {
	// contains filtered or unexported fields
}

MapEntry is an entry in the map, this is public to support iterating over the map using a native Go for loop.

func (MapEntry) Key 

func (e MapEntry) Key() ident.ID

Key returns the map entry key.

func (MapEntry) Value 

func (e MapEntry) Value() DatabaseSeriesBlocks

Value returns the map entry value.

type MapHash 

type MapHash uint64

MapHash is the hash for a given map entry, this is public to support iterating over the map using a native Go for loop.

type MapOptions 

type MapOptions struct {
	InitialSize int
	KeyCopyPool pool.BytesPool
}

MapOptions provides options used when created the map.

type Options 

type Options interface {
	// SetClockOptions sets the clock options.
	SetClockOptions(value clock.Options) Options

	// ClockOptions returns the clock options.
	ClockOptions() clock.Options

	// SetInstrumentOptions sets the instrumentation options.
	SetInstrumentOptions(value instrument.Options) Options

	// InstrumentOptions returns the instrumentation options.
	InstrumentOptions() instrument.Options

	// SetDatabaseBlockOptions sets the database block options.
	SetDatabaseBlockOptions(value block.Options) Options

	// DatabaseBlockOptions returns the database block options.
	DatabaseBlockOptions() block.Options

	// SetNewBlocksLen sets the size of a new blocks map size.
	SetNewBlocksLen(value int) Options

	// NewBlocksLen returns the size of a new blocks map size.
	NewBlocksLen() int

	// SetSeriesCachePolicy sets the series cache policy.
	SetSeriesCachePolicy(value series.CachePolicy) Options

	// SeriesCachePolicy returns the series cache policy.
	SeriesCachePolicy() series.CachePolicy

	// SetIndexDocumentsBuilderAllocator sets the index mutable segment allocator.
	SetIndexDocumentsBuilderAllocator(value DocumentsBuilderAllocator) Options

	// IndexDocumentsBuilderAllocator returns the index documents builder allocator.
	IndexDocumentsBuilderAllocator() DocumentsBuilderAllocator
}

Options represents the options for bootstrap results.

func NewOptions 

func NewOptions() Options

NewOptions creates new bootstrap options

type Segment added in v0.15.11 

type Segment struct {
	// contains filtered or unexported fields
}

Segment wraps an index segment so we can easily determine whether or not the segment is persisted to disk.

func NewSegment added in v0.15.11 

func NewSegment(segment segment.Segment, persisted bool) Segment

NewSegment returns an index segment w/ persistence metadata.

func (Segment) IsPersisted added in v0.15.11 

func (s Segment) IsPersisted() bool

IsPersisted returns whether or not the underlying segment was persisted to disk.

func (Segment) Segment added in v0.15.11 

func (s Segment) Segment() segment.Segment

Segment returns a segment.

type SetUnsafeOptions 

type SetUnsafeOptions struct {
	NoCopyKey     bool
	NoFinalizeKey bool
}

SetUnsafeOptions is a set of options to use when setting a value with the SetUnsafe method.

type ShardResult 

type ShardResult interface {
	// IsEmpty returns whether the result is empty.
	IsEmpty() bool

	// BlockAt returns the block at a given time for a given id,
	// or nil if there is no such block.
	BlockAt(id ident.ID, t time.Time) (block.DatabaseBlock, bool)

	// AllSeries returns a map of all series with their associated blocks.
	AllSeries() *Map

	// NumSeries returns the number of distinct series'.
	NumSeries() int64

	// AddBlock adds a data block.
	AddBlock(id ident.ID, tags ident.Tags, block block.DatabaseBlock)

	// AddSeries adds a single series of blocks.
	AddSeries(id ident.ID, tags ident.Tags, rawSeries block.DatabaseSeriesBlocks)

	// AddResult adds a shard result.
	AddResult(other ShardResult)

	// RemoveBlockAt removes a data block at a given timestamp
	RemoveBlockAt(id ident.ID, t time.Time)

	// RemoveSeries removes a single series of blocks.
	RemoveSeries(id ident.ID)

	// Close closes a shard result.
	Close()
}

ShardResult returns the bootstrap result for a shard.

func NewShardResult 

func NewShardResult(capacity int, opts Options) ShardResult

NewShardResult creates a new shard result.

type ShardResults 

type ShardResults map[uint32]ShardResult

ShardResults is a map of shards to shard results.

func (ShardResults) AddResults 

func (r ShardResults) AddResults(other ShardResults)

AddResults adds other shard results to the current shard results.

func (ShardResults) Equal 

func (r ShardResults) Equal(other ShardResults) bool

Equal returns whether another shard results is equal to the current shard results, will not perform a deep equal only a shallow equal of series and their block addresses.

func (ShardResults) NumSeries 

func (r ShardResults) NumSeries() int64

NumSeries returns the number of series' across all shards.

type ShardTimeRanges 

type ShardTimeRanges interface {
	// Get time ranges for a shard.
	Get(shard uint32) (xtime.Ranges, bool)

	// Set time ranges for a shard.
	Set(shard uint32, ranges xtime.Ranges) ShardTimeRanges

	// GetOrAdd gets or adds time ranges for a shard.
	GetOrAdd(shard uint32) xtime.Ranges

	// AddRanges adds other shard time ranges to the current shard time ranges.
	AddRanges(ranges ShardTimeRanges)

	// Iter returns the underlying map.
	Iter() map[uint32]xtime.Ranges

	Copy() ShardTimeRanges

	// IsSuperset returns whether the current shard time ranges are a
	// superset of the other shard time ranges.
	IsSuperset(other ShardTimeRanges) bool

	// Equal returns whether two shard time ranges are equal.
	Equal(other ShardTimeRanges) bool

	// ToUnfulfilledDataResult will return a result that is comprised of wholly
	// unfufilled time ranges from the set of shard time ranges.
	ToUnfulfilledDataResult() DataBootstrapResult

	// ToUnfulfilledIndexResult will return a result that is comprised of wholly
	// unfufilled time ranges from the set of shard time ranges.
	ToUnfulfilledIndexResult() IndexBootstrapResult

	// Subtract will subtract another range from the current range.
	Subtract(other ShardTimeRanges)

	// MinMax will return the very minimum time as a start and the
	// maximum time as an end in the ranges.
	MinMax() (time.Time, time.Time)

	// MinMaxRange returns the min and max times, and the duration for this range.
	MinMaxRange() (time.Time, time.Time, time.Duration)

	// String returns a description of the time ranges
	String() string

	// SummaryString returns a summary description of the time ranges
	SummaryString() string

	// IsEmpty returns whether the shard time ranges is empty or not.
	IsEmpty() bool

	// Len returns the number of shards
	Len() int
}

ShardTimeRanges is a map of shards to time ranges.

func NewShardTimeRanges 

func NewShardTimeRanges() ShardTimeRanges

NewShardTimeRanges returns an empty ShardTimeRanges.

func NewShardTimeRangesFromRange added in v0.15.0 

func NewShardTimeRangesFromRange(start, end time.Time, shards ...uint32) ShardTimeRanges

NewShardTimeRangesFromRange returns a new ShardTimeRanges with provided shards and time range.

func NewShardTimeRangesFromSize added in v0.15.0 

func NewShardTimeRangesFromSize(size int) ShardTimeRanges

NewShardTimeRangesFromSize returns a new ShardTimeRanges with provided shards and time range.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.