repair

Documentation

Index

• type CopyFn
• type EqualsFn
• type FinalizeFn
• type HashFn
• type HostBlockMetadata
• type HostBlockMetadataSlice
• type HostBlockMetadataSlicePool
  • func NewHostBlockMetadataSlicePool(opts pool.ObjectPoolOptions, capacity int) HostBlockMetadataSlicePool
• type Map
  • func NewMap(opts MapOptions) *Map
  • func (m *Map) Contains(k ident.ID) bool
  • func (m *Map) Delete(k ident.ID)
  • func (m *Map) Get(k ident.ID) (ReplicaSeriesBlocksMetadata, bool)
  • func (m *Map) Iter() map[MapHash]MapEntry
  • func (m *Map) Len() int
  • func (m *Map) Reallocate()
  • func (m *Map) Reset()
  • func (m *Map) Set(k ident.ID, v ReplicaSeriesBlocksMetadata)
  • func (m *Map) SetUnsafe(k ident.ID, v ReplicaSeriesBlocksMetadata, opts SetUnsafeOptions)
• type MapEntry
  • func (e MapEntry) Key() ident.ID
  • func (e MapEntry) Value() ReplicaSeriesBlocksMetadata
• type MapHash
• type MapOptions
• type MetadataComparisonResult
• type Options
  • func NewOptions() Options
• type ReplicaBlockMetadata
  • func NewReplicaBlockMetadata(start time.Time, p HostBlockMetadataSlice) ReplicaBlockMetadata
• type ReplicaBlocksMetadata
  • func NewReplicaBlocksMetadata() ReplicaBlocksMetadata
• type ReplicaMetadataComparer
  • func NewReplicaMetadataComparer(replicas int, opts Options) ReplicaMetadataComparer
• type ReplicaSeriesBlocksMetadata
• type ReplicaSeriesMetadata
  • func NewReplicaSeriesMetadata() ReplicaSeriesMetadata
• type SetUnsafeOptions

Types

type CopyFn

type CopyFn func(ident.ID) ident.ID

CopyFn is the copy key function to execute when copying the key.

type EqualsFn

type EqualsFn func(ident.ID, ident.ID) bool

EqualsFn is the equals key function to execute when detecting equality of a key.

type FinalizeFn

type FinalizeFn func(ident.ID)

FinalizeFn is the finalize key function to execute when finished with a key.

type HashFn

type HashFn func(ident.ID) MapHash

HashFn is the hash function to execute when hashing a key.

type HostBlockMetadata

type HostBlockMetadata struct {
	Host     topology.Host
	Size     int64
	Checksum *uint32
}

HostBlockMetadata contains a host along with block metadata from that host

type HostBlockMetadataSlice

type HostBlockMetadataSlice interface {
	// Add adds the metadata to the slice
	Add(metadata HostBlockMetadata)

	// Metadata returns the metadata slice
	Metadata() []HostBlockMetadata

	// Reset resets the metadata slice
	Reset()

	// Close performs cleanup
	Close()
}

HostBlockMetadataSlice captures a slice of hostBlockMetadata

type HostBlockMetadataSlicePool

type HostBlockMetadataSlicePool interface {
	// Get returns a hostBlockMetadata slice
	Get() HostBlockMetadataSlice

	// Put puts a hostBlockMetadata slice back to pool
	Put(m HostBlockMetadataSlice)
}

HostBlockMetadataSlicePool provides a pool for hostBlockMetadata slices

func NewHostBlockMetadataSlicePool

func NewHostBlockMetadataSlicePool(opts pool.ObjectPoolOptions, capacity int) HostBlockMetadataSlicePool

NewHostBlockMetadataSlicePool creates a new hostBlockMetadataSlice pool

type Map

type Map struct {
	// contains filtered or unexported fields
}

Map uses the genny package to provide a generic hash map that can be specialized by running the following command from this root of the repository: ``` make hashmap-gen pkg=outpkg key_type=Type value_type=Type out_dir=/tmp ``` Or if you would like to use bytes or ident.ID as keys you can use the partially specialized maps to generate your own maps as well: ``` make byteshashmap-gen pkg=outpkg value_type=Type out_dir=/tmp make idhashmap-gen pkg=outpkg value_type=Type out_dir=/tmp ``` This will output to stdout the generated source file to use for your map. It uses linear probing by incrementing the number of the hash created when hashing the identifier if there is a collision. Map is a value type and not an interface to allow for less painful upgrades when adding/removing methods, it is not likely to need mocking so an interface would not be super useful either.

func NewMap

func NewMap(opts MapOptions) *Map

NewMap returns a new byte keyed map.

func (*Map) Contains

func (m *Map) Contains(k ident.ID) bool

Contains returns true if value exists for key, false otherwise, it is shorthand for a call to Get that doesn't return the value.

func (*Map) Delete

func (m *Map) Delete(k ident.ID)

Delete will remove a value set in the map for the specified key.

func (*Map) Get

func (m *Map) Get(k ident.ID) (ReplicaSeriesBlocksMetadata, bool)

Get returns a value in the map for an identifier if found.

func (*Map) Iter

func (m *Map) Iter() map[MapHash]MapEntry

Iter provides the underlying map to allow for using a native Go for loop to iterate the map, however callers should only ever read and not write the map.

func (*Map) Len

func (m *Map) Len() int

Len returns the number of map entries in the map.

func (*Map) Reallocate

func (m *Map) Reallocate()

Reallocate will avoid deleting all keys and reallocate a new map, this is useful if you believe you have a large map and will not need to grow back to a similar size.

func (*Map) Reset

func (m *Map) Reset()

Reset will reset the map by simply deleting all keys to avoid allocating a new map.

func (*Map) Set

func (m *Map) Set(k ident.ID, v ReplicaSeriesBlocksMetadata)

Set will set the value for an identifier.

func (*Map) SetUnsafe

func (m *Map) SetUnsafe(k ident.ID, v ReplicaSeriesBlocksMetadata, opts SetUnsafeOptions)

SetUnsafe will set the value for an identifier with unsafe options for how the map treats the key.

type MapEntry

type MapEntry struct {
	// contains filtered or unexported fields
}

MapEntry is an entry in the map, this is public to support iterating over the map using a native Go for loop.

func (MapEntry) Key

func (e MapEntry) Key() ident.ID

Key returns the map entry key.

func (MapEntry) Value

func (e MapEntry) Value() ReplicaSeriesBlocksMetadata

Value returns the map entry value.

type MapHash

type MapHash uint64

MapHash is the hash for a given map entry, this is public to support iterating over the map using a native Go for loop.

type MapOptions

type MapOptions struct {
	InitialSize int
	KeyCopyPool pool.BytesPool
}

MapOptions provides options used when created the map.

type MetadataComparisonResult

type MetadataComparisonResult struct {
	// NumSeries returns the total number of series
	NumSeries int64

	// NumBlocks returns the total number of blocks
	NumBlocks int64

	// SizeResult returns the size differences
	SizeDifferences ReplicaSeriesMetadata

	// ChecksumDifferences returns the checksum differences
	ChecksumDifferences ReplicaSeriesMetadata
}

MetadataComparisonResult captures metadata comparison results

type Options

type Options interface {
	// SetAdminClient sets the admin client
	SetAdminClient(value client.AdminClient) Options

	// AdminClient returns the admin client
	AdminClient() client.AdminClient

	// SetRepairConsistencyLevel sets the repair read level consistency
	// for which to repair shards with
	SetRepairConsistencyLevel(value topology.ReadConsistencyLevel) Options

	// RepairConsistencyLevel returns the repair read level consistency
	// for which to repair shards with
	RepairConsistencyLevel() topology.ReadConsistencyLevel

	// SetRepairShardConcurrency sets the concurrency in which to repair shards with
	SetRepairShardConcurrency(value int) Options

	// RepairShardConcurrency returns the concurrency in which to repair shards with
	RepairShardConcurrency() int

	// SetRepairInterval sets the repair interval
	SetRepairInterval(value time.Duration) Options

	// RepairInterval returns the repair interval
	RepairInterval() time.Duration

	// SetRepairTimeOffset sets the repair time offset
	SetRepairTimeOffset(value time.Duration) Options

	// RepairTimeOffset returns the repair time offset
	RepairTimeOffset() time.Duration

	// SetRepairJitter sets the repair time jitter
	SetRepairTimeJitter(value time.Duration) Options

	// RepairTimeJitter returns the repair time jitter
	RepairTimeJitter() time.Duration

	// SetRepairCheckInterval sets the repair check interval
	SetRepairCheckInterval(value time.Duration) Options

	// RepairCheckInterval returns the repair check interval
	RepairCheckInterval() time.Duration

	// SetRepairThrottle sets the repair throttle
	SetRepairThrottle(value time.Duration) Options

	// RepairThrottle returns the repair throttle
	RepairThrottle() time.Duration

	// SetRepairMaxRetries sets the max number of retries for a block start
	SetRepairMaxRetries(value int) Options

	// MaxRepairRetries returns the max number of retries for a block start
	RepairMaxRetries() int

	// SetHostBlockMetadataSlicePool sets the hostBlockMetadataSlice pool
	SetHostBlockMetadataSlicePool(value HostBlockMetadataSlicePool) Options

	// HostBlockMetadataSlicePool returns the hostBlockMetadataSlice pool
	HostBlockMetadataSlicePool() HostBlockMetadataSlicePool

	// Validate checks if the options are valid
	Validate() error
}

Options are the repair options

func NewOptions

func NewOptions() Options

NewOptions creates new bootstrap options

type ReplicaBlockMetadata

type ReplicaBlockMetadata interface {
	// Start is the start time of a block
	Start() time.Time

	// Metadata returns the metadata from all hosts
	Metadata() []HostBlockMetadata

	// Add adds a metadata from a host
	Add(metadata HostBlockMetadata)

	// Close performs cleanup
	Close()
}

ReplicaBlockMetadata captures the block metadata from hosts in a shard replica set

func NewReplicaBlockMetadata

func NewReplicaBlockMetadata(start time.Time, p HostBlockMetadataSlice) ReplicaBlockMetadata

NewReplicaBlockMetadata creates a new replica block metadata

type ReplicaBlocksMetadata

type ReplicaBlocksMetadata interface {
	// NumBlocks returns the total number of blocks
	NumBlocks() int64

	// Blocks returns the blocks metadata
	Blocks() map[xtime.UnixNano]ReplicaBlockMetadata

	// Add adds a block metadata
	Add(block ReplicaBlockMetadata)

	// GetOrAdd returns the blocks metadata for a start time, creating one if it doesn't exist
	GetOrAdd(start time.Time, p HostBlockMetadataSlicePool) ReplicaBlockMetadata

	// Close performs cleanup
	Close()
}

ReplicaBlocksMetadata captures the blocks metadata from hosts in a shard replica set

func NewReplicaBlocksMetadata

func NewReplicaBlocksMetadata() ReplicaBlocksMetadata

NewReplicaBlocksMetadata creates a new replica blocks metadata

type ReplicaMetadataComparer

type ReplicaMetadataComparer interface {
	// AddLocalMetadata adds metadata from local host
	AddLocalMetadata(origin topology.Host, localIter block.FilteredBlocksMetadataIter) error

	// AddPeerMetadata adds metadata from peers
	AddPeerMetadata(peerIter client.PeerBlockMetadataIter) error

	// Compare returns the metadata differences between local host and peers
	Compare() MetadataComparisonResult

	// Finalize performs cleanup during close
	Finalize()
}

ReplicaMetadataComparer compares metadata from hosts in a replica set

func NewReplicaMetadataComparer

func NewReplicaMetadataComparer(replicas int, opts Options) ReplicaMetadataComparer

NewReplicaMetadataComparer creates a new replica metadata comparer

type ReplicaSeriesBlocksMetadata

type ReplicaSeriesBlocksMetadata struct {
	ID       ident.ID
	Metadata ReplicaBlocksMetadata
}

ReplicaSeriesBlocksMetadata represents series metadata and an associated ID.

type ReplicaSeriesMetadata

type ReplicaSeriesMetadata interface {
	// NumSeries returns the total number of series
	NumSeries() int64

	// NumBlocks returns the total number of blocks
	NumBlocks() int64

	// Series returns the series metadata
	Series() *Map

	// GetOrAdd returns the series metadata for an id, creating one if it doesn't exist
	GetOrAdd(id ident.ID) ReplicaBlocksMetadata

	// Close performs cleanup
	Close()
}

ReplicaSeriesMetadata captures the metadata for a list of series from hosts in a shard replica set

func NewReplicaSeriesMetadata

func NewReplicaSeriesMetadata() ReplicaSeriesMetadata

NewReplicaSeriesMetadata creates a new replica series metadata

type SetUnsafeOptions

type SetUnsafeOptions struct {
	NoCopyKey     bool
	NoFinalizeKey bool
}

SetUnsafeOptions is a set of options to use when setting a value with the SetUnsafe method.