bootstrap

package
v0.15.0-rc.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 5, 2020 License: Apache-2.0 Imports: 32 Imported by: 0

Documentation

Overview

Package bootstrap is a generated GoMock package.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Bootstrapper 

type Bootstrapper interface {
	// String returns the name of the bootstrapper
	String() string

	// Bootstrap performs bootstrapping of series data and index metadata
	// for the  given time ranges, returning the bootstrapped series data
	// and index blocks for the time ranges it's unable to fulfill in parallel.
	// A bootstrapper should only return an error should it want to entirely
	// cancel the bootstrapping of the node, i.e. non-recoverable situation
	// like not being able to read from the filesystem.
	Bootstrap(namespaces Namespaces) (NamespaceResults, error)
}

Bootstrapper is the interface for different bootstrapping mechanisms. Note that a bootstrapper can and will be reused so it is important to not rely on state stored in the bootstrapper itself with the mindset that it will always be set to default values from the constructor.

type BootstrapperProvider 

type BootstrapperProvider interface {
	// String returns the name of the bootstrapper.
	String() string

	// Provide constructs a bootstrapper.
	Provide() (Bootstrapper, error)
}

BootstrapperProvider constructs a bootstrapper.

type CheckoutSeriesResult 

type CheckoutSeriesResult struct {
	// Series is the series for the checkout operation.
	Series series.DatabaseSeries
	// Shard is the shard for the series.
	Shard uint32
	// UniqueIndex is the unique index for the series.
	UniqueIndex uint64
}

CheckoutSeriesResult is the result of a checkout series operation.

type DecodedBlockMap 

type DecodedBlockMap map[string]DecodedValues

DecodedBlockMap is a map of decoded datapoints per series ID.

func (DecodedBlockMap) VerifyEquals 

func (m DecodedBlockMap) VerifyEquals(other DecodedBlockMap) error

VerifyEquals verifies that two DecodedBlockMap are equal; errors otherwise.

type DecodedNamespaceMap 

type DecodedNamespaceMap map[string]DecodedBlockMap

DecodedNamespaceMap is a map of decoded blocks per namespace ID.

type DecodedValues 

type DecodedValues []series.DecodedTestValue

DecodedValues is a slice of series datapoints.

type MockBootstrapper 

type MockBootstrapper struct {
	// contains filtered or unexported fields
}

MockBootstrapper is a mock of Bootstrapper interface

func NewMockBootstrapper 

func NewMockBootstrapper(ctrl *gomock.Controller) *MockBootstrapper

NewMockBootstrapper creates a new mock instance

func (*MockBootstrapper) Bootstrap 

func (m *MockBootstrapper) Bootstrap(namespaces Namespaces) (NamespaceResults, error)

Bootstrap mocks base method

func (*MockBootstrapper) EXPECT 

func (m *MockBootstrapper) EXPECT() *MockBootstrapperMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockBootstrapper) String 

func (m *MockBootstrapper) String() string

String mocks base method

type MockBootstrapperMockRecorder 

type MockBootstrapperMockRecorder struct {
	// contains filtered or unexported fields
}

MockBootstrapperMockRecorder is the mock recorder for MockBootstrapper

func (*MockBootstrapperMockRecorder) Bootstrap 

func (mr *MockBootstrapperMockRecorder) Bootstrap(namespaces interface{}) *gomock.Call

Bootstrap indicates an expected call of Bootstrap

func (*MockBootstrapperMockRecorder) String 

func (mr *MockBootstrapperMockRecorder) String() *gomock.Call

String indicates an expected call of String

type MockBootstrapperProvider 

type MockBootstrapperProvider struct {
	// contains filtered or unexported fields
}

MockBootstrapperProvider is a mock of BootstrapperProvider interface

func NewMockBootstrapperProvider 

func NewMockBootstrapperProvider(ctrl *gomock.Controller) *MockBootstrapperProvider

NewMockBootstrapperProvider creates a new mock instance

func (*MockBootstrapperProvider) EXPECT 

func (m *MockBootstrapperProvider) EXPECT() *MockBootstrapperProviderMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockBootstrapperProvider) Provide 

func (m *MockBootstrapperProvider) Provide() (Bootstrapper, error)

Provide mocks base method

func (*MockBootstrapperProvider) String 

func (m *MockBootstrapperProvider) String() string

String mocks base method

type MockBootstrapperProviderMockRecorder 

type MockBootstrapperProviderMockRecorder struct {
	// contains filtered or unexported fields
}

MockBootstrapperProviderMockRecorder is the mock recorder for MockBootstrapperProvider

func (*MockBootstrapperProviderMockRecorder) Provide 

func (mr *MockBootstrapperProviderMockRecorder) Provide() *gomock.Call

Provide indicates an expected call of Provide

func (*MockBootstrapperProviderMockRecorder) String 

func (mr *MockBootstrapperProviderMockRecorder) String() *gomock.Call

String indicates an expected call of String

type MockNamespaceDataAccumulator 

type MockNamespaceDataAccumulator struct {
	// contains filtered or unexported fields
}

MockNamespaceDataAccumulator is a mock of NamespaceDataAccumulator interface

func NewMockNamespaceDataAccumulator 

func NewMockNamespaceDataAccumulator(ctrl *gomock.Controller) *MockNamespaceDataAccumulator

NewMockNamespaceDataAccumulator creates a new mock instance

func (*MockNamespaceDataAccumulator) CheckoutSeriesWithLock 

func (m *MockNamespaceDataAccumulator) CheckoutSeriesWithLock(shardID uint32, id ident.ID, tags ident.TagIterator) (CheckoutSeriesResult, bool, error)

CheckoutSeriesWithLock mocks base method

func (*MockNamespaceDataAccumulator) CheckoutSeriesWithoutLock 

func (m *MockNamespaceDataAccumulator) CheckoutSeriesWithoutLock(shardID uint32, id ident.ID, tags ident.TagIterator) (CheckoutSeriesResult, bool, error)

CheckoutSeriesWithoutLock mocks base method

func (*MockNamespaceDataAccumulator) Close 

func (m *MockNamespaceDataAccumulator) Close() error

Close mocks base method

func (*MockNamespaceDataAccumulator) EXPECT 

func (m *MockNamespaceDataAccumulator) EXPECT() *MockNamespaceDataAccumulatorMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

type MockNamespaceDataAccumulatorMockRecorder 

type MockNamespaceDataAccumulatorMockRecorder struct {
	// contains filtered or unexported fields
}

MockNamespaceDataAccumulatorMockRecorder is the mock recorder for MockNamespaceDataAccumulator

func (*MockNamespaceDataAccumulatorMockRecorder) CheckoutSeriesWithLock 

func (mr *MockNamespaceDataAccumulatorMockRecorder) CheckoutSeriesWithLock(shardID, id, tags interface{}) *gomock.Call

CheckoutSeriesWithLock indicates an expected call of CheckoutSeriesWithLock

func (*MockNamespaceDataAccumulatorMockRecorder) CheckoutSeriesWithoutLock 

func (mr *MockNamespaceDataAccumulatorMockRecorder) CheckoutSeriesWithoutLock(shardID, id, tags interface{}) *gomock.Call

CheckoutSeriesWithoutLock indicates an expected call of CheckoutSeriesWithoutLock

func (*MockNamespaceDataAccumulatorMockRecorder) Close 

func (mr *MockNamespaceDataAccumulatorMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close

type MockProcess 

type MockProcess struct {
	// contains filtered or unexported fields
}

MockProcess is a mock of Process interface

func NewMockProcess 

func NewMockProcess(ctrl *gomock.Controller) *MockProcess

NewMockProcess creates a new mock instance

func (*MockProcess) EXPECT 

func (m *MockProcess) EXPECT() *MockProcessMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockProcess) Run 

func (m *MockProcess) Run(start time.Time, namespaces []ProcessNamespace) (NamespaceResults, error)

Run mocks base method

type MockProcessMockRecorder 

type MockProcessMockRecorder struct {
	// contains filtered or unexported fields
}

MockProcessMockRecorder is the mock recorder for MockProcess

func (*MockProcessMockRecorder) Run 

func (mr *MockProcessMockRecorder) Run(start, namespaces interface{}) *gomock.Call

Run indicates an expected call of Run

type MockProcessOptions 

type MockProcessOptions struct {
	// contains filtered or unexported fields
}

MockProcessOptions is a mock of ProcessOptions interface

func NewMockProcessOptions 

func NewMockProcessOptions(ctrl *gomock.Controller) *MockProcessOptions

NewMockProcessOptions creates a new mock instance

func (*MockProcessOptions) CacheSeriesMetadata 

func (m *MockProcessOptions) CacheSeriesMetadata() bool

CacheSeriesMetadata mocks base method

func (*MockProcessOptions) EXPECT 

func (m *MockProcessOptions) EXPECT() *MockProcessOptionsMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockProcessOptions) Origin added in v0.4.6 

func (m *MockProcessOptions) Origin() topology.Host

Origin mocks base method

func (*MockProcessOptions) SetCacheSeriesMetadata 

func (m *MockProcessOptions) SetCacheSeriesMetadata(value bool) ProcessOptions

SetCacheSeriesMetadata mocks base method

func (*MockProcessOptions) SetOrigin added in v0.4.6 

func (m *MockProcessOptions) SetOrigin(value topology.Host) ProcessOptions

SetOrigin mocks base method

func (*MockProcessOptions) SetTopologyMapProvider added in v0.4.6 

func (m *MockProcessOptions) SetTopologyMapProvider(value topology.MapProvider) ProcessOptions

SetTopologyMapProvider mocks base method

func (*MockProcessOptions) TopologyMapProvider added in v0.4.6 

func (m *MockProcessOptions) TopologyMapProvider() topology.MapProvider

TopologyMapProvider mocks base method

func (*MockProcessOptions) Validate added in v0.4.4 

func (m *MockProcessOptions) Validate() error

Validate mocks base method

type MockProcessOptionsMockRecorder 

type MockProcessOptionsMockRecorder struct {
	// contains filtered or unexported fields
}

MockProcessOptionsMockRecorder is the mock recorder for MockProcessOptions

func (*MockProcessOptionsMockRecorder) CacheSeriesMetadata 

func (mr *MockProcessOptionsMockRecorder) CacheSeriesMetadata() *gomock.Call

CacheSeriesMetadata indicates an expected call of CacheSeriesMetadata

func (*MockProcessOptionsMockRecorder) Origin added in v0.4.6 

func (mr *MockProcessOptionsMockRecorder) Origin() *gomock.Call

Origin indicates an expected call of Origin

func (*MockProcessOptionsMockRecorder) SetCacheSeriesMetadata 

func (mr *MockProcessOptionsMockRecorder) SetCacheSeriesMetadata(value interface{}) *gomock.Call

SetCacheSeriesMetadata indicates an expected call of SetCacheSeriesMetadata

func (*MockProcessOptionsMockRecorder) SetOrigin added in v0.4.6 

func (mr *MockProcessOptionsMockRecorder) SetOrigin(value interface{}) *gomock.Call

SetOrigin indicates an expected call of SetOrigin

func (*MockProcessOptionsMockRecorder) SetTopologyMapProvider added in v0.4.6 

func (mr *MockProcessOptionsMockRecorder) SetTopologyMapProvider(value interface{}) *gomock.Call

SetTopologyMapProvider indicates an expected call of SetTopologyMapProvider

func (*MockProcessOptionsMockRecorder) TopologyMapProvider added in v0.4.6 

func (mr *MockProcessOptionsMockRecorder) TopologyMapProvider() *gomock.Call

TopologyMapProvider indicates an expected call of TopologyMapProvider

func (*MockProcessOptionsMockRecorder) Validate added in v0.4.4 

func (mr *MockProcessOptionsMockRecorder) Validate() *gomock.Call

Validate indicates an expected call of Validate

type MockProcessProvider 

type MockProcessProvider struct {
	// contains filtered or unexported fields
}

MockProcessProvider is a mock of ProcessProvider interface

func NewMockProcessProvider 

func NewMockProcessProvider(ctrl *gomock.Controller) *MockProcessProvider

NewMockProcessProvider creates a new mock instance

func (*MockProcessProvider) BootstrapperProvider 

func (m *MockProcessProvider) BootstrapperProvider() BootstrapperProvider

BootstrapperProvider mocks base method

func (*MockProcessProvider) EXPECT 

func (m *MockProcessProvider) EXPECT() *MockProcessProviderMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockProcessProvider) Provide 

func (m *MockProcessProvider) Provide() (Process, error)

Provide mocks base method

func (*MockProcessProvider) SetBootstrapperProvider 

func (m *MockProcessProvider) SetBootstrapperProvider(bootstrapper BootstrapperProvider)

SetBootstrapperProvider mocks base method

type MockProcessProviderMockRecorder 

type MockProcessProviderMockRecorder struct {
	// contains filtered or unexported fields
}

MockProcessProviderMockRecorder is the mock recorder for MockProcessProvider

func (*MockProcessProviderMockRecorder) BootstrapperProvider 

func (mr *MockProcessProviderMockRecorder) BootstrapperProvider() *gomock.Call

BootstrapperProvider indicates an expected call of BootstrapperProvider

func (*MockProcessProviderMockRecorder) Provide 

func (mr *MockProcessProviderMockRecorder) Provide() *gomock.Call

Provide indicates an expected call of Provide

func (*MockProcessProviderMockRecorder) SetBootstrapperProvider 

func (mr *MockProcessProviderMockRecorder) SetBootstrapperProvider(bootstrapper interface{}) *gomock.Call

SetBootstrapperProvider indicates an expected call of SetBootstrapperProvider

type MockRunOptions 

type MockRunOptions struct {
	// contains filtered or unexported fields
}

MockRunOptions is a mock of RunOptions interface

func NewMockRunOptions 

func NewMockRunOptions(ctrl *gomock.Controller) *MockRunOptions

NewMockRunOptions creates a new mock instance

func (*MockRunOptions) CacheSeriesMetadata 

func (m *MockRunOptions) CacheSeriesMetadata() bool

CacheSeriesMetadata mocks base method

func (*MockRunOptions) EXPECT 

func (m *MockRunOptions) EXPECT() *MockRunOptionsMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockRunOptions) InitialTopologyState added in v0.4.4 

func (m *MockRunOptions) InitialTopologyState() *topology.StateSnapshot

InitialTopologyState mocks base method

func (*MockRunOptions) PersistConfig added in v0.4.4 

func (m *MockRunOptions) PersistConfig() PersistConfig

PersistConfig mocks base method

func (*MockRunOptions) SetCacheSeriesMetadata 

func (m *MockRunOptions) SetCacheSeriesMetadata(value bool) RunOptions

SetCacheSeriesMetadata mocks base method

func (*MockRunOptions) SetInitialTopologyState added in v0.4.4 

func (m *MockRunOptions) SetInitialTopologyState(value *topology.StateSnapshot) RunOptions

SetInitialTopologyState mocks base method

func (*MockRunOptions) SetPersistConfig added in v0.4.4 

func (m *MockRunOptions) SetPersistConfig(value PersistConfig) RunOptions

SetPersistConfig mocks base method

type MockRunOptionsMockRecorder 

type MockRunOptionsMockRecorder struct {
	// contains filtered or unexported fields
}

MockRunOptionsMockRecorder is the mock recorder for MockRunOptions

func (*MockRunOptionsMockRecorder) CacheSeriesMetadata 

func (mr *MockRunOptionsMockRecorder) CacheSeriesMetadata() *gomock.Call

CacheSeriesMetadata indicates an expected call of CacheSeriesMetadata

func (*MockRunOptionsMockRecorder) InitialTopologyState added in v0.4.4 

func (mr *MockRunOptionsMockRecorder) InitialTopologyState() *gomock.Call

InitialTopologyState indicates an expected call of InitialTopologyState

func (*MockRunOptionsMockRecorder) PersistConfig added in v0.4.4 

func (mr *MockRunOptionsMockRecorder) PersistConfig() *gomock.Call

PersistConfig indicates an expected call of PersistConfig

func (*MockRunOptionsMockRecorder) SetCacheSeriesMetadata 

func (mr *MockRunOptionsMockRecorder) SetCacheSeriesMetadata(value interface{}) *gomock.Call

SetCacheSeriesMetadata indicates an expected call of SetCacheSeriesMetadata

func (*MockRunOptionsMockRecorder) SetInitialTopologyState added in v0.4.4 

func (mr *MockRunOptionsMockRecorder) SetInitialTopologyState(value interface{}) *gomock.Call

SetInitialTopologyState indicates an expected call of SetInitialTopologyState

func (*MockRunOptionsMockRecorder) SetPersistConfig added in v0.4.4 

func (mr *MockRunOptionsMockRecorder) SetPersistConfig(value interface{}) *gomock.Call

SetPersistConfig indicates an expected call of SetPersistConfig

type MockSource 

type MockSource struct {
	// contains filtered or unexported fields
}

MockSource is a mock of Source interface

func NewMockSource 

func NewMockSource(ctrl *gomock.Controller) *MockSource

NewMockSource creates a new mock instance

func (*MockSource) AvailableData 

func (m *MockSource) AvailableData(ns namespace.Metadata, shardsTimeRanges result.ShardTimeRanges, runOpts RunOptions) (result.ShardTimeRanges, error)

AvailableData mocks base method

func (*MockSource) AvailableIndex 

func (m *MockSource) AvailableIndex(ns namespace.Metadata, shardsTimeRanges result.ShardTimeRanges, opts RunOptions) (result.ShardTimeRanges, error)

AvailableIndex mocks base method

func (*MockSource) EXPECT 

func (m *MockSource) EXPECT() *MockSourceMockRecorder

EXPECT returns an object that allows the caller to indicate expected use

func (*MockSource) Read 

func (m *MockSource) Read(namespaces Namespaces) (NamespaceResults, error)

Read mocks base method

type MockSourceMockRecorder 

type MockSourceMockRecorder struct {
	// contains filtered or unexported fields
}

MockSourceMockRecorder is the mock recorder for MockSource

func (*MockSourceMockRecorder) AvailableData 

func (mr *MockSourceMockRecorder) AvailableData(ns, shardsTimeRanges, runOpts interface{}) *gomock.Call

AvailableData indicates an expected call of AvailableData

func (*MockSourceMockRecorder) AvailableIndex 

func (mr *MockSourceMockRecorder) AvailableIndex(ns, shardsTimeRanges, opts interface{}) *gomock.Call

AvailableIndex indicates an expected call of AvailableIndex

func (*MockSourceMockRecorder) Read 

func (mr *MockSourceMockRecorder) Read(namespaces interface{}) *gomock.Call

Read indicates an expected call of Read

type Namespace 

type Namespace struct {
	// Metadata of the namespace being bootstrapped.
	Metadata namespace.Metadata
	// Shards is the shards for the namespace being bootstrapped.
	Shards []uint32
	// DataAccumulator is the data accumulator for the shards.
	DataAccumulator NamespaceDataAccumulator
	// Hooks is a set of namespace bootstrap hooks.
	Hooks NamespaceHooks
	// DataTargetRange is the data target bootstrap range.
	DataTargetRange TargetRange
	// IndexTargetRange is the index target bootstrap range.
	IndexTargetRange TargetRange
	// DataRunOptions are the options for the data bootstrap for this
	// namespace.
	DataRunOptions NamespaceRunOptions
	// IndexRunOptions are the options for the index bootstrap for this
	// namespace.
	IndexRunOptions NamespaceRunOptions
}

Namespace is a namespace that is being bootstrapped.

type NamespaceDataAccumulator 

type NamespaceDataAccumulator interface {
	// CheckoutSeriesWithoutLock retrieves a series for writing to
	// and when the accumulator is closed it will ensure that the
	// series is released.
	//
	// If indexing is not enabled, tags is still required, simply pass
	// ident.EmptyTagIterator.
	//
	// Returns the result, whether the node owns the specified shard, along with
	// an error if any. This allows callers to handle unowned shards differently
	// than other errors. If owned == false, err should not be nil.
	//
	// Note: Without lock variant does not perform any locking and callers
	// must ensure non-parallel access themselves, this helps avoid
	// overhead of the lock for the commit log bootstrapper which reads
	// in a single threaded manner.
	CheckoutSeriesWithoutLock(
		shardID uint32,
		id ident.ID,
		tags ident.TagIterator,
	) (result CheckoutSeriesResult, owned bool, err error)

	// CheckoutSeriesWithLock is the "with lock" version of
	// CheckoutSeriesWithoutLock.
	// Note: With lock variant performs locking and callers do not need
	// to be concerned about parallel access.
	CheckoutSeriesWithLock(
		shardID uint32,
		id ident.ID,
		tags ident.TagIterator,
	) (result CheckoutSeriesResult, owned bool, err error)

	// Close will close the data accumulator and will release
	// all series read/write refs.
	Close() error
}

NamespaceDataAccumulator is the namespace data accumulator. TODO(r): Consider rename this to a better name.

type NamespaceHooks 

type NamespaceHooks struct {
	// contains filtered or unexported fields
}

NamespaceHooks is a set of namespace bootstrap hooks.

func NewNamespaceHooks 

func NewNamespaceHooks(opts NamespaceHooksOptions) NamespaceHooks

NewNamespaceHooks returns a new set of bootstrap hooks.

func (NamespaceHooks) BootstrapSourceBegin 

func (h NamespaceHooks) BootstrapSourceBegin() error

BootstrapSourceBegin is a hook to call when a bootstrap source starts.

func (NamespaceHooks) BootstrapSourceEnd 

func (h NamespaceHooks) BootstrapSourceEnd() error

BootstrapSourceEnd is a hook to call when a bootstrap source ends.

type NamespaceHooksOptions 

type NamespaceHooksOptions struct {
	BootstrapSourceBegin func() error
	BootstrapSourceEnd   func() error
}

NamespaceHooksOptions is a set of hooks options.

type NamespaceMatcher 

type NamespaceMatcher struct {
	// Namespaces are the expected namespaces.
	Namespaces Namespaces
}

NamespaceMatcher is a matcher for namespaces.

func (NamespaceMatcher) Matches 

func (m NamespaceMatcher) Matches(x interface{}) bool

Matches returns whether x is a match.

func (NamespaceMatcher) String 

func (m NamespaceMatcher) String() string

String describes what the matcher matches.

type NamespaceResult 

type NamespaceResult struct {
	Metadata    namespace.Metadata
	Shards      []uint32
	DataResult  result.DataBootstrapResult
	IndexResult result.IndexBootstrapResult
}

NamespaceResult is the result of a bootstrap process for a given namespace.

type NamespaceResults 

type NamespaceResults struct {
	// Results is the result of a bootstrap process.
	Results *NamespaceResultsMap
}

NamespaceResults is the result of a bootstrap process.

func MergeNamespaceResults 

func MergeNamespaceResults(a, b NamespaceResults) NamespaceResults

MergeNamespaceResults merges two namespace results, this will mutate both a and b and return a merged copy of them reusing one of the results.

func NewNamespaceResults 

func NewNamespaceResults(
	namespaces Namespaces,
) NamespaceResults

NewNamespaceResults creates a namespace results map with an entry for each namespace spoecified by a namespaces map.

type NamespaceResultsMap 

type NamespaceResultsMap struct {
	// contains filtered or unexported fields
}

NamespaceResultsMap uses the genny package to provide a generic hash map that can be specialized by running the following command from this root of the repository: ``` make hashmap-gen pkg=outpkg key_type=Type value_type=Type out_dir=/tmp ``` Or if you would like to use bytes or ident.ID as keys you can use the partially specialized maps to generate your own maps as well: ``` make byteshashmap-gen pkg=outpkg value_type=Type out_dir=/tmp make idhashmap-gen pkg=outpkg value_type=Type out_dir=/tmp ``` This will output to stdout the generated source file to use for your map. It uses linear probing by incrementing the number of the hash created when hashing the identifier if there is a collision. NamespaceResultsMap is a value type and not an interface to allow for less painful upgrades when adding/removing methods, it is not likely to need mocking so an interface would not be super useful either.

func NewNamespaceResultsMap 

func NewNamespaceResultsMap(opts NamespaceResultsMapOptions) *NamespaceResultsMap

NewNamespaceResultsMap returns a new byte keyed map.

func (*NamespaceResultsMap) Contains 

func (m *NamespaceResultsMap) Contains(k ident.ID) bool

Contains returns true if value exists for key, false otherwise, it is shorthand for a call to Get that doesn't return the value.

func (*NamespaceResultsMap) Delete 

func (m *NamespaceResultsMap) Delete(k ident.ID)

Delete will remove a value set in the map for the specified key.

func (*NamespaceResultsMap) Get 

func (m *NamespaceResultsMap) Get(k ident.ID) (NamespaceResult, bool)

Get returns a value in the map for an identifier if found.

func (*NamespaceResultsMap) Iter 

func (m *NamespaceResultsMap) Iter() map[NamespaceResultsMapHash]NamespaceResultsMapEntry

Iter provides the underlying map to allow for using a native Go for loop to iterate the map, however callers should only ever read and not write the map.

func (*NamespaceResultsMap) Len 

func (m *NamespaceResultsMap) Len() int

Len returns the number of map entries in the map.

func (*NamespaceResultsMap) Reallocate 

func (m *NamespaceResultsMap) Reallocate()

Reallocate will avoid deleting all keys and reallocate a new map, this is useful if you believe you have a large map and will not need to grow back to a similar size.

func (*NamespaceResultsMap) Reset 

func (m *NamespaceResultsMap) Reset()

Reset will reset the map by simply deleting all keys to avoid allocating a new map.

func (*NamespaceResultsMap) Set 

func (m *NamespaceResultsMap) Set(k ident.ID, v NamespaceResult)

Set will set the value for an identifier.

func (*NamespaceResultsMap) SetUnsafe 

func (m *NamespaceResultsMap) SetUnsafe(k ident.ID, v NamespaceResult, opts NamespaceResultsMapSetUnsafeOptions)

SetUnsafe will set the value for an identifier with unsafe options for how the map treats the key.

type NamespaceResultsMapCopyFn 

type NamespaceResultsMapCopyFn func(ident.ID) ident.ID

NamespaceResultsMapCopyFn is the copy key function to execute when copying the key.

type NamespaceResultsMapEntry 

type NamespaceResultsMapEntry struct {
	// contains filtered or unexported fields
}

NamespaceResultsMapEntry is an entry in the map, this is public to support iterating over the map using a native Go for loop.

func (NamespaceResultsMapEntry) Key 

func (e NamespaceResultsMapEntry) Key() ident.ID

Key returns the map entry key.

func (NamespaceResultsMapEntry) Value 

func (e NamespaceResultsMapEntry) Value() NamespaceResult

Value returns the map entry value.

type NamespaceResultsMapEqualsFn 

type NamespaceResultsMapEqualsFn func(ident.ID, ident.ID) bool

NamespaceResultsMapEqualsFn is the equals key function to execute when detecting equality of a key.

type NamespaceResultsMapFinalizeFn 

type NamespaceResultsMapFinalizeFn func(ident.ID)

NamespaceResultsMapFinalizeFn is the finalize key function to execute when finished with a key.

type NamespaceResultsMapHash 

type NamespaceResultsMapHash uint64

NamespaceResultsMapHash is the hash for a given map entry, this is public to support iterating over the map using a native Go for loop.

type NamespaceResultsMapHashFn 

type NamespaceResultsMapHashFn func(ident.ID) NamespaceResultsMapHash

NamespaceResultsMapHashFn is the hash function to execute when hashing a key.

type NamespaceResultsMapOptions 

type NamespaceResultsMapOptions struct {
	InitialSize int
	KeyCopyPool pool.BytesPool
}

NamespaceResultsMapOptions provides options used when created the map.

type NamespaceResultsMapSetUnsafeOptions 

type NamespaceResultsMapSetUnsafeOptions struct {
	NoCopyKey     bool
	NoFinalizeKey bool
}

NamespaceResultsMapSetUnsafeOptions is a set of options to use when setting a value with the SetUnsafe method.

type NamespaceRunOptions 

type NamespaceRunOptions struct {
	// ShardTimeRanges are the time ranges for the shards that should be fulfilled
	// by the bootstrapper.
	ShardTimeRanges result.ShardTimeRanges
	// RunOptions are the run options for the bootstrap run.
	RunOptions RunOptions
}

NamespaceRunOptions are the run options for a bootstrap process run.

type Namespaces 

type Namespaces struct {
	// Namespaces are the namespaces being bootstrapped.
	Namespaces *NamespacesMap
}

Namespaces are a set of namespaces being bootstrapped.

func NewNamespaces 

func NewNamespaces(
	namespaces []ProcessNamespace,
) Namespaces

NewNamespaces returns a new set of bootstrappable namespaces.

func (Namespaces) Hooks 

func (n Namespaces) Hooks() NamespacesHooks

Hooks returns namespaces hooks for the set of namespaces.

type NamespacesHooks 

type NamespacesHooks struct {
	// contains filtered or unexported fields
}

NamespacesHooks is a helper to run hooks for a set of namespaces.

func (NamespacesHooks) BootstrapSourceBegin 

func (h NamespacesHooks) BootstrapSourceBegin() error

BootstrapSourceBegin is a hook to call when a bootstrap source starts.

func (NamespacesHooks) BootstrapSourceEnd 

func (h NamespacesHooks) BootstrapSourceEnd() error

BootstrapSourceEnd is a hook to call when a bootstrap source starts.

type NamespacesMap 

type NamespacesMap struct {
	// contains filtered or unexported fields
}

NamespacesMap uses the genny package to provide a generic hash map that can be specialized by running the following command from this root of the repository: ``` make hashmap-gen pkg=outpkg key_type=Type value_type=Type out_dir=/tmp ``` Or if you would like to use bytes or ident.ID as keys you can use the partially specialized maps to generate your own maps as well: ``` make byteshashmap-gen pkg=outpkg value_type=Type out_dir=/tmp make idhashmap-gen pkg=outpkg value_type=Type out_dir=/tmp ``` This will output to stdout the generated source file to use for your map. It uses linear probing by incrementing the number of the hash created when hashing the identifier if there is a collision. NamespacesMap is a value type and not an interface to allow for less painful upgrades when adding/removing methods, it is not likely to need mocking so an interface would not be super useful either.

func NewNamespacesMap 

func NewNamespacesMap(opts NamespacesMapOptions) *NamespacesMap

NewNamespacesMap returns a new byte keyed map.

func (*NamespacesMap) Contains 

func (m *NamespacesMap) Contains(k ident.ID) bool

Contains returns true if value exists for key, false otherwise, it is shorthand for a call to Get that doesn't return the value.

func (*NamespacesMap) Delete 

func (m *NamespacesMap) Delete(k ident.ID)

Delete will remove a value set in the map for the specified key.

func (*NamespacesMap) Get 

func (m *NamespacesMap) Get(k ident.ID) (Namespace, bool)

Get returns a value in the map for an identifier if found.

func (*NamespacesMap) Iter 

func (m *NamespacesMap) Iter() map[NamespacesMapHash]NamespacesMapEntry

Iter provides the underlying map to allow for using a native Go for loop to iterate the map, however callers should only ever read and not write the map.

func (*NamespacesMap) Len 

func (m *NamespacesMap) Len() int

Len returns the number of map entries in the map.

func (*NamespacesMap) Reallocate 

func (m *NamespacesMap) Reallocate()

Reallocate will avoid deleting all keys and reallocate a new map, this is useful if you believe you have a large map and will not need to grow back to a similar size.

func (*NamespacesMap) Reset 

func (m *NamespacesMap) Reset()

Reset will reset the map by simply deleting all keys to avoid allocating a new map.

func (*NamespacesMap) Set 

func (m *NamespacesMap) Set(k ident.ID, v Namespace)

Set will set the value for an identifier.

func (*NamespacesMap) SetUnsafe 

func (m *NamespacesMap) SetUnsafe(k ident.ID, v Namespace, opts NamespacesMapSetUnsafeOptions)

SetUnsafe will set the value for an identifier with unsafe options for how the map treats the key.

type NamespacesMapCopyFn 

type NamespacesMapCopyFn func(ident.ID) ident.ID

NamespacesMapCopyFn is the copy key function to execute when copying the key.

type NamespacesMapEntry 

type NamespacesMapEntry struct {
	// contains filtered or unexported fields
}

NamespacesMapEntry is an entry in the map, this is public to support iterating over the map using a native Go for loop.

func (NamespacesMapEntry) Key 

func (e NamespacesMapEntry) Key() ident.ID

Key returns the map entry key.

func (NamespacesMapEntry) Value 

func (e NamespacesMapEntry) Value() Namespace

Value returns the map entry value.

type NamespacesMapEqualsFn 

type NamespacesMapEqualsFn func(ident.ID, ident.ID) bool

NamespacesMapEqualsFn is the equals key function to execute when detecting equality of a key.

type NamespacesMapFinalizeFn 

type NamespacesMapFinalizeFn func(ident.ID)

NamespacesMapFinalizeFn is the finalize key function to execute when finished with a key.

type NamespacesMapHash 

type NamespacesMapHash uint64

NamespacesMapHash is the hash for a given map entry, this is public to support iterating over the map using a native Go for loop.

type NamespacesMapHashFn 

type NamespacesMapHashFn func(ident.ID) NamespacesMapHash

NamespacesMapHashFn is the hash function to execute when hashing a key.

type NamespacesMapOptions 

type NamespacesMapOptions struct {
	InitialSize int
	KeyCopyPool pool.BytesPool
}

NamespacesMapOptions provides options used when created the map.

type NamespacesMapSetUnsafeOptions 

type NamespacesMapSetUnsafeOptions struct {
	NoCopyKey     bool
	NoFinalizeKey bool
}

NamespacesMapSetUnsafeOptions is a set of options to use when setting a value with the SetUnsafe method.

type NamespacesTester 

type NamespacesTester struct {

	// Accumulators are the accumulators which incoming blocks get loaded into.
	// One per namespace.
	Accumulators []*TestDataAccumulator

	// Namespaces are the namespaces for this tester.
	Namespaces Namespaces
	// Results are the namespace results after bootstrapping.
	Results NamespaceResults
	// contains filtered or unexported fields
}

NamespacesTester is a utility to assist testing bootstrapping.

func BuildNamespacesTester 

func BuildNamespacesTester(
	t require.TestingT,
	runOpts RunOptions,
	ranges result.ShardTimeRanges,
	mds ...namespace.Metadata,
) NamespacesTester

BuildNamespacesTester builds a NamespacesTester.

func BuildNamespacesTesterWithReaderIteratorPool 

func BuildNamespacesTesterWithReaderIteratorPool(
	t require.TestingT,
	runOpts RunOptions,
	ranges result.ShardTimeRanges,
	iterPool encoding.MultiReaderIteratorPool,
	mds ...namespace.Metadata,
) NamespacesTester

BuildNamespacesTesterWithReaderIteratorPool builds a NamespacesTester with a given MultiReaderIteratorPool.

func (*NamespacesTester) DumpLoadedBlocks 

func (nt *NamespacesTester) DumpLoadedBlocks() DecodedNamespaceMap

DumpLoadedBlocks dumps any loaded blocks as decoded series per namespace.

func (*NamespacesTester) DumpWrites 

func (nt *NamespacesTester) DumpWrites() DecodedNamespaceMap

DumpWrites dumps the writes encountered for all namespaces.

func (*NamespacesTester) EnsureDumpAllForNamespace 

func (nt *NamespacesTester) EnsureDumpAllForNamespace(
	md namespace.Metadata,
) (DecodedBlockMap, error)

EnsureDumpAllForNamespace dumps all results for a single namespace, and fails if the namespace is not found. The results are unsorted; if sorted order is important for verification, they should be sorted afterwards.

func (*NamespacesTester) EnsureDumpLoadedBlocksForNamespace 

func (nt *NamespacesTester) EnsureDumpLoadedBlocksForNamespace(
	md namespace.Metadata,
) DecodedBlockMap

EnsureDumpLoadedBlocksForNamespace dumps all loaded blocks as decoded series, and fails if the namespace is not found.

func (*NamespacesTester) EnsureDumpReadersForNamespace 

func (nt *NamespacesTester) EnsureDumpReadersForNamespace(
	md namespace.Metadata,
) ReaderMap

EnsureDumpReadersForNamespace dumps the readers and their start times for a given namespace, and fails if the namespace is not found.

func (*NamespacesTester) EnsureDumpWritesForNamespace 

func (nt *NamespacesTester) EnsureDumpWritesForNamespace(
	md namespace.Metadata,
) DecodedBlockMap

EnsureDumpWritesForNamespace dumps the writes encountered for the given namespace, and fails if the namespace is not found.

func (*NamespacesTester) EnsureNoLoadedBlocks 

func (nt *NamespacesTester) EnsureNoLoadedBlocks()

EnsureNoLoadedBlocks ensures that no blocks have been loaded into any of this testers accumulators.

func (*NamespacesTester) EnsureNoWrites 

func (nt *NamespacesTester) EnsureNoWrites()

EnsureNoWrites ensures that no writes have been written into any of this testers accumulators.

func (*NamespacesTester) Finish 

func (nt *NamespacesTester) Finish()

Finish closes the namespaceTester and tests mocks for completion.

func (*NamespacesTester) ResultForNamespace 

func (nt *NamespacesTester) ResultForNamespace(id ident.ID) NamespaceResult

ResultForNamespace gives the result for the given namespace, and fails if the namespace is not found.

func (*NamespacesTester) TestBootstrapWith 

func (nt *NamespacesTester) TestBootstrapWith(b Bootstrapper)

TestBootstrapWith bootstraps the current Namespaces with the provided bootstrapper.

func (*NamespacesTester) TestReadWith 

func (nt *NamespacesTester) TestReadWith(s Source)

TestReadWith reads the current Namespaces with the provided bootstrap source.

func (*NamespacesTester) TestUnfulfilledForIDIsEmpty 

func (nt *NamespacesTester) TestUnfulfilledForIDIsEmpty(
	id ident.ID,
	useIndex bool,
)

TestUnfulfilledForIDIsEmpty ensures the given id has an empty unfulfilled range.

func (*NamespacesTester) TestUnfulfilledForNamespace 

func (nt *NamespacesTester) TestUnfulfilledForNamespace(
	md namespace.Metadata,
	ex result.ShardTimeRanges,
	exIdx result.ShardTimeRanges,
)

TestUnfulfilledForNamespace ensures the given namespace has the expected range flagged as unfulfilled.

func (*NamespacesTester) TestUnfulfilledForNamespaceIsEmpty 

func (nt *NamespacesTester) TestUnfulfilledForNamespaceIsEmpty(
	md namespace.Metadata,
)

TestUnfulfilledForNamespaceIsEmpty ensures the given namespace has an empty unfulfilled range.

type PersistConfig added in v0.4.4 

type PersistConfig struct {
	// If enabled bootstrappers are allowed to write out bootstrapped data
	// to disk on their own instead of just returning result in-memory.
	Enabled bool
	// If enabled, what type of persistence files should be generated during
	// the process.
	FileSetType persist.FileSetType
}

PersistConfig is the configuration for a bootstrap with persistence.

type Process 

type Process interface {
	// Run runs the bootstrap process, returning the bootstrap result and any error encountered.
	Run(start time.Time, namespaces []ProcessNamespace) (NamespaceResults, error)
}

Process represents the bootstrap process. Note that a bootstrap process can and will be reused so it is important to not rely on state stored in the bootstrap itself with the mindset that it will always be set to default values from the constructor.

type ProcessNamespace 

type ProcessNamespace struct {
	// Metadata of the namespace being bootstrapped.
	Metadata namespace.Metadata
	// Shards is the shards to bootstrap for the bootstrap process.
	Shards []uint32
	// DataAccumulator is the data accumulator for the shards.
	DataAccumulator NamespaceDataAccumulator
	// Hooks is a set of namespace bootstrap hooks.
	Hooks NamespaceHooks
}

ProcessNamespace is a namespace to pass to the bootstrap process.

type ProcessOptions 

type ProcessOptions interface {
	// SetCacheSeriesMetadata sets whether bootstrappers created by this
	// provider should cache series metadata between runs.
	SetCacheSeriesMetadata(value bool) ProcessOptions

	// CacheSeriesMetadata returns whether bootstrappers created by this
	// provider should cache series metadata between runs.
	CacheSeriesMetadata() bool

	// SetTopologyMapProvider sets the TopologyMapProvider.
	SetTopologyMapProvider(value topology.MapProvider) ProcessOptions

	// TopologyMapProvider returns the TopologyMapProvider.
	TopologyMapProvider() topology.MapProvider

	// SetOrigin sets the origin.
	SetOrigin(value topology.Host) ProcessOptions

	// Origin returns the origin.
	Origin() topology.Host

	// Validate validates that the ProcessOptions are correct.
	Validate() error
}

ProcessOptions is a set of options for a bootstrap provider.

func NewProcessOptions 

func NewProcessOptions() ProcessOptions

NewProcessOptions creates new bootstrap run options

type ProcessProvider 

type ProcessProvider interface {
	// SetBootstrapper sets the bootstrapper provider to use when running the
	// process.
	SetBootstrapperProvider(bootstrapper BootstrapperProvider)

	// Bootstrapper returns the current bootstrappe provider to use when
	// running the process.
	BootstrapperProvider() BootstrapperProvider

	// Provide constructs a bootstrap process.
	Provide() (Process, error)
}

ProcessProvider constructs a bootstrap process that can execute a bootstrap run.

func NewNoOpProcessProvider 

func NewNoOpProcessProvider() ProcessProvider

NewNoOpProcessProvider creates a no-op bootstrap process proivder.

func NewProcessProvider 

func NewProcessProvider(
	bootstrapperProvider BootstrapperProvider,
	processOpts ProcessOptions,
	resultOpts result.Options,
) (ProcessProvider, error)

NewProcessProvider creates a new bootstrap process provider.

type ReaderAtTime 

type ReaderAtTime struct {
	// Start is the block start time.
	Start time.Time
	// Reader is the block segment reader.
	Reader xio.SegmentReader
	// Tags is the list of tags in a basic string map format.
	Tags map[string]string
}

ReaderAtTime captures incoming block loads, including their start times and tags.

type ReaderMap 

type ReaderMap map[string]ReadersForID

ReaderMap is a map containing all gathered block segment readers.

type ReadersForID 

type ReadersForID []ReaderAtTime

ReadersForID is a slice of readers that share a series ID.

type RunOptions 

type RunOptions interface {
	// SetPersistConfig sets persistence configuration for this bootstrap.
	SetPersistConfig(value PersistConfig) RunOptions

	// PersistConfig returns the persistence configuration for this bootstrap.
	PersistConfig() PersistConfig

	// SetCacheSeriesMetadata sets whether bootstrappers created by this
	// provider should cache series metadata between runs.
	SetCacheSeriesMetadata(value bool) RunOptions

	// CacheSeriesMetadata returns whether bootstrappers created by this
	// provider should cache series metadata between runs.
	CacheSeriesMetadata() bool

	// SetInitialTopologyState sets the initial topology state as it was
	// measured before the bootstrap process began.
	SetInitialTopologyState(value *topology.StateSnapshot) RunOptions

	// InitialTopologyState returns the initial topology as it was measured
	// before the bootstrap process began.
	InitialTopologyState() *topology.StateSnapshot
}

RunOptions is a set of options for a bootstrap run.

func NewRunOptions 

func NewRunOptions() RunOptions

NewRunOptions creates new bootstrap run options

type ShardTimeRangesMatcher 

type ShardTimeRangesMatcher struct {
	// Ranges are the expected ranges.
	Ranges result.ShardTimeRanges
}

ShardTimeRangesMatcher is a matcher for ShardTimeRanges.

func (ShardTimeRangesMatcher) Matches 

func (m ShardTimeRangesMatcher) Matches(x interface{}) bool

Matches returns whether x is a match.

func (ShardTimeRangesMatcher) String 

func (m ShardTimeRangesMatcher) String() string

String describes what the matcher matches.

type Source 

type Source interface {
	// AvailableData returns what time ranges are available for bootstrapping a given set of shards.
	AvailableData(
		ns namespace.Metadata,
		shardsTimeRanges result.ShardTimeRanges,
		runOpts RunOptions,
	) (result.ShardTimeRanges, error)

	// AvailableIndex returns what time ranges are available for bootstrapping.
	AvailableIndex(
		ns namespace.Metadata,
		shardsTimeRanges result.ShardTimeRanges,
		opts RunOptions,
	) (result.ShardTimeRanges, error)

	// Read returns series data and index metadata for a given set of shards
	// and specified time ranges and the time ranges it's unable to fulfill.
	// A bootstrapper source should only return an error should it want to
	// entirely cancel the bootstrapping of the node, i.e. non-recoverable
	// situation like not being able to read from the filesystem.
	Read(namespaces Namespaces) (NamespaceResults, error)
}

Source represents a bootstrap source. Note that a source can and will be reused so it is important to not rely on state stored in the source itself with the mindset that it will always be set to default values from the constructor.

type TargetRange 

type TargetRange struct {
	// Range is the time range to bootstrap for.
	Range xtime.Range

	// RunOptions is the bootstrap run options specific to the target range.
	RunOptions RunOptions
}

TargetRange is a bootstrap target range.

type TestDataAccumulator 

type TestDataAccumulator struct {
	sync.Mutex
	// contains filtered or unexported fields
}

TestDataAccumulator is a NamespaceDataAccumulator that captures any series inserts for examination.

func (*TestDataAccumulator) CheckoutSeriesWithLock 

func (a *TestDataAccumulator) CheckoutSeriesWithLock(
	shardID uint32,
	id ident.ID,
	tags ident.TagIterator,
) (CheckoutSeriesResult, bool, error)

CheckoutSeriesWithLock will retrieve a series for writing to, and when the accumulator is closed, it will ensure that the series is released (with lock).

func (*TestDataAccumulator) CheckoutSeriesWithoutLock 

func (a *TestDataAccumulator) CheckoutSeriesWithoutLock(
	shardID uint32,
	id ident.ID,
	tags ident.TagIterator,
) (CheckoutSeriesResult, bool, error)

CheckoutSeriesWithoutLock will retrieve a series for writing to, and when the accumulator is closed, it will ensure that the series is released (without lock).

func (*TestDataAccumulator) Close 

func (a *TestDataAccumulator) Close() error

Close is a no-op on the test accumulator.

func (*TestDataAccumulator) Release 

func (a *TestDataAccumulator) Release()

Release is a no-op on the test accumulator.

Source Files

View all Source files

Directories

Path Synopsis
commitlog
fs
peers
uninitialized

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.