schedule

package
v2.1.15+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 17, 2019 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// LeaderOperatorWaitTime is the duration that when a leader operator lives
	// longer than it, the operator will be considered timeout.
	LeaderOperatorWaitTime = 10 * time.Second
	// RegionOperatorWaitTime is the duration that when a region operator lives
	// longer than it, the operator will be considered timeout.
	RegionOperatorWaitTime = 10 * time.Minute
)
View Source
const (
	// RegionHeartBeatReportInterval is the heartbeat report interval of a region
	RegionHeartBeatReportInterval = 60
)
View Source
const (
	// RejectLeader is the label property type that sugguests a store should not
	// have any region leaders.
	RejectLeader = "reject-leader"
)

Variables

View Source
var Simulating bool

Simulating is an option to overpass the impact of accelerated time. Should only turned on by the simulator.

Functions

func CheckOperatorValid

func CheckOperatorValid(op *Operator) bool

CheckOperatorValid checks if the operator is valid.

func CreateMergeRegionOperator

func CreateMergeRegionOperator(desc string, cluster Cluster, source *core.RegionInfo, target *core.RegionInfo, kind OperatorKind) (*Operator, *Operator, error)

CreateMergeRegionOperator creates an Operator that merge two region into one

func DistinctScore

func DistinctScore(labels []string, stores []*core.StoreInfo, other *core.StoreInfo) float64

DistinctScore returns the score that the other is distinct from the stores. A higher score means the other store is more different from the existed stores.

func FilterSource

func FilterSource(opt Options, store *core.StoreInfo, filters []Filter) bool

FilterSource checks if store can pass all Filters as source store.

func FilterTarget

func FilterTarget(opt Options, store *core.StoreInfo, filters []Filter) bool

FilterTarget checks if store can pass all Filters as target store.

func RegisterScheduler

func RegisterScheduler(name string, createFn CreateSchedulerFunc)

RegisterScheduler binds a scheduler creator. It should be called in init() func of a package.

Types

type AddLearner

type AddLearner struct {
	ToStore, PeerID uint64
}

AddLearner is an OperatorStep that adds a region learner peer.

func (AddLearner) Influence

func (al AddLearner) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make

func (AddLearner) IsFinish

func (al AddLearner) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (AddLearner) String

func (al AddLearner) String() string

type AddPeer

type AddPeer struct {
	ToStore, PeerID uint64
}

AddPeer is an OperatorStep that adds a region peer.

func (AddPeer) Influence

func (ap AddPeer) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make

func (AddPeer) IsFinish

func (ap AddPeer) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (AddPeer) String

func (ap AddPeer) String() string

type BalanceSelector

type BalanceSelector struct {
	// contains filtered or unexported fields
}

BalanceSelector selects source/target from store candidates based on their resource scores.

func NewBalanceSelector

func NewBalanceSelector(kind core.ResourceKind, filters []Filter) *BalanceSelector

NewBalanceSelector creates a BalanceSelector instance.

func (*BalanceSelector) SelectSource

func (s *BalanceSelector) SelectSource(opt Options, stores []*core.StoreInfo) *core.StoreInfo

SelectSource selects the store that can pass all filters and has the minimal resource score.

func (*BalanceSelector) SelectTarget

func (s *BalanceSelector) SelectTarget(opt Options, stores []*core.StoreInfo, filters ...Filter) *core.StoreInfo

SelectTarget selects the store that can pass all filters and has the maximal resource score.

type BasicCluster

type BasicCluster struct {
	Stores   *core.StoresInfo
	Regions  *core.RegionsInfo
	HotCache *HotSpotCache
}

BasicCluster provides basic data member and interface for a tikv cluster.

func NewBasicCluster

func NewBasicCluster() *BasicCluster

NewBasicCluster creates a BasicCluster.

func (*BasicCluster) BlockStore

func (bc *BasicCluster) BlockStore(storeID uint64) error

BlockStore stops balancer from selecting the store.

func (*BasicCluster) CheckReadStatus

func (bc *BasicCluster) CheckReadStatus(region *core.RegionInfo) (bool, *core.RegionStat)

CheckReadStatus checks the read status, returns whether need update statistics and item.

func (*BasicCluster) CheckWriteStatus

func (bc *BasicCluster) CheckWriteStatus(region *core.RegionInfo) (bool, *core.RegionStat)

CheckWriteStatus checks the write status, returns whether need update statistics and item.

func (*BasicCluster) GetAdjacentRegions

func (bc *BasicCluster) GetAdjacentRegions(region *core.RegionInfo) (*core.RegionInfo, *core.RegionInfo)

GetAdjacentRegions returns region's info that is adjacent with specific region

func (*BasicCluster) GetAverageRegionSize

func (bc *BasicCluster) GetAverageRegionSize() int64

GetAverageRegionSize returns the average region approximate size.

func (*BasicCluster) GetFollowerStores

func (bc *BasicCluster) GetFollowerStores(region *core.RegionInfo) []*core.StoreInfo

GetFollowerStores returns all Stores that contains the region's follower peer.

func (*BasicCluster) GetLeaderStore

func (bc *BasicCluster) GetLeaderStore(region *core.RegionInfo) *core.StoreInfo

GetLeaderStore returns all Stores that contains the region's leader peer.

func (*BasicCluster) GetRegion

func (bc *BasicCluster) GetRegion(regionID uint64) *core.RegionInfo

GetRegion searches for a region by ID.

func (*BasicCluster) GetRegionStores

func (bc *BasicCluster) GetRegionStores(region *core.RegionInfo) []*core.StoreInfo

GetRegionStores returns all Stores that contains the region's peer.

func (*BasicCluster) GetStore

func (bc *BasicCluster) GetStore(storeID uint64) *core.StoreInfo

GetStore searches for a store by ID.

func (*BasicCluster) GetStores

func (bc *BasicCluster) GetStores() []*core.StoreInfo

GetStores returns all Stores in the cluster.

func (*BasicCluster) IsRegionHot

func (bc *BasicCluster) IsRegionHot(id uint64, hotThreshold int) bool

IsRegionHot checks if a region is in hot state.

func (*BasicCluster) PutRegion

func (bc *BasicCluster) PutRegion(region *core.RegionInfo) error

PutRegion put a region

func (*BasicCluster) PutStore

func (bc *BasicCluster) PutStore(store *core.StoreInfo) error

PutStore put a store

func (*BasicCluster) RandFollowerRegion

func (bc *BasicCluster) RandFollowerRegion(storeID uint64, opts ...core.RegionOption) *core.RegionInfo

RandFollowerRegion returns a random region that has a follower on the store.

func (*BasicCluster) RandLeaderRegion

func (bc *BasicCluster) RandLeaderRegion(storeID uint64, opts ...core.RegionOption) *core.RegionInfo

RandLeaderRegion returns a random region that has leader on the store.

func (*BasicCluster) RegionReadStats

func (bc *BasicCluster) RegionReadStats() []*core.RegionStat

RegionReadStats returns hot region's read stats.

func (*BasicCluster) RegionWriteStats

func (bc *BasicCluster) RegionWriteStats() []*core.RegionStat

RegionWriteStats returns hot region's write stats.

func (*BasicCluster) UnblockStore

func (bc *BasicCluster) UnblockStore(storeID uint64)

UnblockStore allows balancer to select the store.

type Cluster

type Cluster interface {
	RandFollowerRegion(storeID uint64, opts ...core.RegionOption) *core.RegionInfo
	RandLeaderRegion(storeID uint64, opts ...core.RegionOption) *core.RegionInfo
	GetAverageRegionSize() int64

	GetStores() []*core.StoreInfo
	GetStore(id uint64) *core.StoreInfo
	GetRegion(id uint64) *core.RegionInfo
	GetRegionStores(region *core.RegionInfo) []*core.StoreInfo
	GetFollowerStores(region *core.RegionInfo) []*core.StoreInfo
	GetLeaderStore(region *core.RegionInfo) *core.StoreInfo
	GetAdjacentRegions(region *core.RegionInfo) (*core.RegionInfo, *core.RegionInfo)
	ScanRegions(startKey []byte, limit int) []*core.RegionInfo

	BlockStore(id uint64) error
	UnblockStore(id uint64)

	IsRegionHot(id uint64) bool
	RegionWriteStats() []*core.RegionStat
	RegionReadStats() []*core.RegionStat
	RandHotRegionFromStore(store uint64, kind FlowKind) *core.RegionInfo

	// get config methods
	GetOpt() NamespaceOptions
	Options

	// TODO: it should be removed. Schedulers don't need to know anything
	// about peers.
	AllocPeer(storeID uint64) (*metapb.Peer, error)
}

Cluster provides an overview of a cluster's regions distribution.

type CreateSchedulerFunc

type CreateSchedulerFunc func(limiter *Limiter, args []string) (Scheduler, error)

CreateSchedulerFunc is for creating scheudler.

type Filter

type Filter interface {
	Type() string
	// Return true if the store should not be used as a source store.
	FilterSource(opt Options, store *core.StoreInfo) bool
	// Return true if the store should not be used as a target store.
	FilterTarget(opt Options, store *core.StoreInfo) bool
}

Filter is an interface to filter source and target store.

func NewBlockFilter

func NewBlockFilter() Filter

NewBlockFilter creates a Filter that filters all stores that are blocked from balance.

func NewCacheFilter

func NewCacheFilter(cache *cache.TTLUint64) Filter

NewCacheFilter creates a Filter that filters all stores that are in the cache.

func NewDisconnectFilter

func NewDisconnectFilter() Filter

NewDisconnectFilter creates a Filter that filters all stores that are disconnected.

func NewDistinctScoreFilter

func NewDistinctScoreFilter(labels []string, stores []*core.StoreInfo, source *core.StoreInfo) Filter

NewDistinctScoreFilter creates a filter that filters all stores that have lower distinct score than specified store.

func NewExcludedFilter

func NewExcludedFilter(sources, targets map[uint64]struct{}) Filter

NewExcludedFilter creates a Filter that filters all specified stores.

func NewHealthFilter

func NewHealthFilter() Filter

NewHealthFilter creates a Filter that filters all stores that are Busy or Down.

func NewNamespaceFilter

func NewNamespaceFilter(classifier namespace.Classifier, namespace string) Filter

NewNamespaceFilter creates a Filter that filters all stores that are not belong to a namespace.

func NewPendingPeerCountFilter

func NewPendingPeerCountFilter() Filter

NewPendingPeerCountFilter creates a Filter that filters all stores that are currently handling too many pending peers.

func NewRejectLeaderFilter

func NewRejectLeaderFilter() Filter

NewRejectLeaderFilter creates a Filter that filters stores that marked as rejectLeader from being the target of leader transfer.

func NewSnapshotCountFilter

func NewSnapshotCountFilter() Filter

NewSnapshotCountFilter creates a Filter that filters all stores that are currently handling too many snapshots.

func NewStateFilter

func NewStateFilter() Filter

NewStateFilter creates a Filter that filters all stores that are not UP.

func NewStorageThresholdFilter

func NewStorageThresholdFilter() Filter

NewStorageThresholdFilter creates a Filter that filters all stores that are almost full.

type FlowKind

type FlowKind uint32

FlowKind is a identify Flow types.

const (
	WriteFlow FlowKind = iota
	ReadFlow
)

Flags for flow.

type HotSpotCache

type HotSpotCache struct {
	// contains filtered or unexported fields
}

HotSpotCache is a cache hold hot regions.

func (*HotSpotCache) CheckRead

func (w *HotSpotCache) CheckRead(region *core.RegionInfo, stores *core.StoresInfo) (bool, *core.RegionStat)

CheckRead checks the read status, returns whether need update statistics and item.

func (*HotSpotCache) CheckWrite

func (w *HotSpotCache) CheckWrite(region *core.RegionInfo, stores *core.StoresInfo) (bool, *core.RegionStat)

CheckWrite checks the write status, returns whether need update statistics and item.

func (*HotSpotCache) CollectMetrics

func (w *HotSpotCache) CollectMetrics(stores *core.StoresInfo)

CollectMetrics collect the hot cache metrics

func (*HotSpotCache) RandHotRegionFromStore

func (w *HotSpotCache) RandHotRegionFromStore(storeID uint64, kind FlowKind, hotThreshold int) *core.RegionStat

RandHotRegionFromStore random picks a hot region in specify store.

func (*HotSpotCache) RegionStats

func (w *HotSpotCache) RegionStats(kind FlowKind) []*core.RegionStat

RegionStats returns hot items according to kind

func (*HotSpotCache) Update

func (w *HotSpotCache) Update(key uint64, item *core.RegionStat, kind FlowKind)

Update updates the cache.

type Limiter

type Limiter struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Limiter a counter that limits the number of operators

func NewLimiter

func NewLimiter() *Limiter

NewLimiter create a schedule limiter

func (*Limiter) OperatorCount

func (l *Limiter) OperatorCount(mask OperatorKind) uint64

OperatorCount gets the count of operators filtered by mask.

func (*Limiter) UpdateCounts

func (l *Limiter) UpdateCounts(operators map[uint64]*Operator)

UpdateCounts updates resouce counts using current pending operators.

type MergeChecker

type MergeChecker struct {
	// contains filtered or unexported fields
}

MergeChecker ensures region to merge with adjacent region when size is small

func NewMergeChecker

func NewMergeChecker(cluster Cluster, classifier namespace.Classifier) *MergeChecker

NewMergeChecker creates a merge checker.

func (*MergeChecker) Check

func (m *MergeChecker) Check(region *core.RegionInfo) (*Operator, *Operator)

Check verifies a region's replicas, creating an Operator if need.

func (*MergeChecker) RecordRegionSplit

func (m *MergeChecker) RecordRegionSplit(regionID uint64)

RecordRegionSplit put the recently splitted region into cache. MergeChecker will skip check it for a while.

type MergeRegion

type MergeRegion struct {
	FromRegion *metapb.Region
	ToRegion   *metapb.Region
	// there are two regions involved in merge process,
	// so to keep them from other scheduler,
	// both of them should add MerRegion operatorStep.
	// But actually, tikv just need the region want to be merged to get the merge request,
	// thus use a IsPssive mark to indicate that
	// this region doesn't need to send merge request to tikv.
	IsPassive bool
}

MergeRegion is an OperatorStep that merge two regions.

func (MergeRegion) Influence

func (mr MergeRegion) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make

func (MergeRegion) IsFinish

func (mr MergeRegion) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished

func (MergeRegion) String

func (mr MergeRegion) String() string

type MockCluster

type MockCluster struct {
	*BasicCluster

	*MockSchedulerOptions
	// contains filtered or unexported fields
}

MockCluster is used to mock clusterInfo for test use

func NewMockCluster

func NewMockCluster(opt *MockSchedulerOptions) *MockCluster

NewMockCluster creates a new MockCluster

func (*MockCluster) AddLabelsStore

func (mc *MockCluster) AddLabelsStore(storeID uint64, regionCount int, labels map[string]string)

AddLabelsStore adds store with specified count of region and labels.

func (*MockCluster) AddLeaderRegion

func (mc *MockCluster) AddLeaderRegion(regionID uint64, leaderID uint64, followerIds ...uint64)

AddLeaderRegion adds region with specified leader and followers.

func (*MockCluster) AddLeaderRegionWithRange

func (mc *MockCluster) AddLeaderRegionWithRange(regionID uint64, startKey string, endKey string, leaderID uint64, followerIds ...uint64)

AddLeaderRegionWithRange adds region with specified leader, followers and key range.

func (*MockCluster) AddLeaderRegionWithReadInfo

func (mc *MockCluster) AddLeaderRegionWithReadInfo(regionID uint64, leaderID uint64, readBytes uint64, followerIds ...uint64)

AddLeaderRegionWithReadInfo adds region with specified leader, followers and read info.

func (*MockCluster) AddLeaderRegionWithWriteInfo

func (mc *MockCluster) AddLeaderRegionWithWriteInfo(regionID uint64, leaderID uint64, writtenBytes uint64, followerIds ...uint64)

AddLeaderRegionWithWriteInfo adds region with specified leader, followers and write info.

func (*MockCluster) AddLeaderStore

func (mc *MockCluster) AddLeaderStore(storeID uint64, leaderCount int)

AddLeaderStore adds store with specified count of leader.

func (*MockCluster) AddRegionStore

func (mc *MockCluster) AddRegionStore(storeID uint64, regionCount int)

AddRegionStore adds store with specified count of region.

func (*MockCluster) AllocPeer

func (mc *MockCluster) AllocPeer(storeID uint64) (*metapb.Peer, error)

AllocPeer allocs a new peer on a store.

func (*MockCluster) ApplyOperator

func (mc *MockCluster) ApplyOperator(op *Operator)

ApplyOperator mocks apply operator.

func (*MockCluster) ApplyOperatorStep

func (mc *MockCluster) ApplyOperatorStep(region *core.RegionInfo, op *Operator) *core.RegionInfo

ApplyOperatorStep mocks apply operator step.

func (*MockCluster) CheckLabelProperty

func (mc *MockCluster) CheckLabelProperty(typ string, labels []*metapb.StoreLabel) bool

CheckLabelProperty checks label property.

func (*MockCluster) GetHotRegionScheduleLimit

func (mc *MockCluster) GetHotRegionScheduleLimit() uint64

GetHotRegionScheduleLimit mocks method.

func (*MockCluster) GetLeaderScheduleLimit

func (mc *MockCluster) GetLeaderScheduleLimit() uint64

GetLeaderScheduleLimit mocks method.

func (*MockCluster) GetMaxReplicas

func (mc *MockCluster) GetMaxReplicas() int

GetMaxReplicas mocks method.

func (*MockCluster) GetMergeScheduleLimit

func (mc *MockCluster) GetMergeScheduleLimit() uint64

GetMergeScheduleLimit mocks method.

func (*MockCluster) GetOpt

func (mc *MockCluster) GetOpt() NamespaceOptions

GetOpt mocks method.

func (*MockCluster) GetRegionScheduleLimit

func (mc *MockCluster) GetRegionScheduleLimit() uint64

GetRegionScheduleLimit mocks method.

func (*MockCluster) GetReplicaScheduleLimit

func (mc *MockCluster) GetReplicaScheduleLimit() uint64

GetReplicaScheduleLimit mocks method.

func (*MockCluster) IsRegionHot

func (mc *MockCluster) IsRegionHot(id uint64) bool

IsRegionHot checks if the region is hot

func (*MockCluster) LoadRegion

func (mc *MockCluster) LoadRegion(regionID uint64, followerIds ...uint64)

LoadRegion put region info without leader

func (*MockCluster) RandHotRegionFromStore

func (mc *MockCluster) RandHotRegionFromStore(store uint64, kind FlowKind) *core.RegionInfo

RandHotRegionFromStore random picks a hot region in specify store.

func (*MockCluster) ScanRegions

func (mc *MockCluster) ScanRegions(startKey []byte, limit int) []*core.RegionInfo

ScanRegions scan region with start key, until number greater than limit.

func (*MockCluster) SetStoreBusy

func (mc *MockCluster) SetStoreBusy(storeID uint64, busy bool)

SetStoreBusy sets store busy.

func (*MockCluster) SetStoreDisconnect

func (mc *MockCluster) SetStoreDisconnect(storeID uint64)

SetStoreDisconnect changes a store's state to disconnected.

func (*MockCluster) SetStoreDown

func (mc *MockCluster) SetStoreDown(storeID uint64)

SetStoreDown sets store down.

func (*MockCluster) SetStoreOffline

func (mc *MockCluster) SetStoreOffline(storeID uint64)

SetStoreOffline sets store state to be offline.

func (*MockCluster) SetStoreUp

func (mc *MockCluster) SetStoreUp(storeID uint64)

SetStoreUp sets store state to be up.

func (*MockCluster) UpdateLeaderCount

func (mc *MockCluster) UpdateLeaderCount(storeID uint64, leaderCount int)

UpdateLeaderCount updates store leader count.

func (*MockCluster) UpdatePendingPeerCount

func (mc *MockCluster) UpdatePendingPeerCount(storeID uint64, pendingPeerCount int)

UpdatePendingPeerCount updates store pending peer count.

func (*MockCluster) UpdateRegionCount

func (mc *MockCluster) UpdateRegionCount(storeID uint64, regionCount int)

UpdateRegionCount updates store region count.

func (*MockCluster) UpdateSnapshotCount

func (mc *MockCluster) UpdateSnapshotCount(storeID uint64, snapshotCount int)

UpdateSnapshotCount updates store snapshot count.

func (*MockCluster) UpdateStorageRatio

func (mc *MockCluster) UpdateStorageRatio(storeID uint64, usedRatio, availableRatio float64)

UpdateStorageRatio updates store storage ratio count.

func (*MockCluster) UpdateStorageReadBytes

func (mc *MockCluster) UpdateStorageReadBytes(storeID uint64, BytesRead uint64)

UpdateStorageReadBytes updates store read bytes.

func (*MockCluster) UpdateStorageWrittenBytes

func (mc *MockCluster) UpdateStorageWrittenBytes(storeID uint64, BytesWritten uint64)

UpdateStorageWrittenBytes updates store written bytes.

func (*MockCluster) UpdateStoreLeaderSize

func (mc *MockCluster) UpdateStoreLeaderSize(storeID uint64, size int64)

UpdateStoreLeaderSize updates store leader size.

func (*MockCluster) UpdateStoreLeaderWeight

func (mc *MockCluster) UpdateStoreLeaderWeight(storeID uint64, weight float64)

UpdateStoreLeaderWeight updates store leader weight.

func (*MockCluster) UpdateStoreRegionSize

func (mc *MockCluster) UpdateStoreRegionSize(storeID uint64, size int64)

UpdateStoreRegionSize updates store region size.

func (*MockCluster) UpdateStoreRegionWeight

func (mc *MockCluster) UpdateStoreRegionWeight(storeID uint64, weight float64)

UpdateStoreRegionWeight updates store region weight.

func (*MockCluster) UpdateStoreStatus

func (mc *MockCluster) UpdateStoreStatus(id uint64)

UpdateStoreStatus updates store status.

type MockSchedulerOptions

type MockSchedulerOptions struct {
	RegionScheduleLimit          uint64
	LeaderScheduleLimit          uint64
	ReplicaScheduleLimit         uint64
	MergeScheduleLimit           uint64
	HotRegionScheduleLimit       uint64
	MaxSnapshotCount             uint64
	MaxPendingPeerCount          uint64
	MaxMergeRegionSize           uint64
	MaxMergeRegionKeys           uint64
	SplitMergeInterval           time.Duration
	MaxStoreDownTime             time.Duration
	MaxReplicas                  int
	LocationLabels               []string
	HotRegionCacheHitsThreshold  int
	TolerantSizeRatio            float64
	LowSpaceRatio                float64
	HighSpaceRatio               float64
	DisableLearner               bool
	DisableRemoveDownReplica     bool
	DisableReplaceOfflineReplica bool
	DisableMakeUpReplica         bool
	DisableRemoveExtraReplica    bool
	DisableLocationReplacement   bool
	DisableNamespaceRelocation   bool
	LabelProperties              map[string][]*metapb.StoreLabel
}

MockSchedulerOptions is a mock of SchedulerOptions which implements Options interface

func NewMockSchedulerOptions

func NewMockSchedulerOptions() *MockSchedulerOptions

NewMockSchedulerOptions creates a mock schedule option.

func (*MockSchedulerOptions) GetHighSpaceRatio

func (mso *MockSchedulerOptions) GetHighSpaceRatio() float64

GetHighSpaceRatio mock method

func (*MockSchedulerOptions) GetHotRegionCacheHitsThreshold

func (mso *MockSchedulerOptions) GetHotRegionCacheHitsThreshold() int

GetHotRegionCacheHitsThreshold mock method

func (*MockSchedulerOptions) GetHotRegionScheduleLimit

func (mso *MockSchedulerOptions) GetHotRegionScheduleLimit(name string) uint64

GetHotRegionScheduleLimit mock method

func (*MockSchedulerOptions) GetLeaderScheduleLimit

func (mso *MockSchedulerOptions) GetLeaderScheduleLimit(name string) uint64

GetLeaderScheduleLimit mock method

func (*MockSchedulerOptions) GetLocationLabels

func (mso *MockSchedulerOptions) GetLocationLabels() []string

GetLocationLabels mock method

func (*MockSchedulerOptions) GetLowSpaceRatio

func (mso *MockSchedulerOptions) GetLowSpaceRatio() float64

GetLowSpaceRatio mock method

func (*MockSchedulerOptions) GetMaxMergeRegionKeys

func (mso *MockSchedulerOptions) GetMaxMergeRegionKeys() uint64

GetMaxMergeRegionKeys mock method

func (*MockSchedulerOptions) GetMaxMergeRegionSize

func (mso *MockSchedulerOptions) GetMaxMergeRegionSize() uint64

GetMaxMergeRegionSize mock method

func (*MockSchedulerOptions) GetMaxPendingPeerCount

func (mso *MockSchedulerOptions) GetMaxPendingPeerCount() uint64

GetMaxPendingPeerCount mock method

func (*MockSchedulerOptions) GetMaxReplicas

func (mso *MockSchedulerOptions) GetMaxReplicas(name string) int

GetMaxReplicas mock method

func (*MockSchedulerOptions) GetMaxSnapshotCount

func (mso *MockSchedulerOptions) GetMaxSnapshotCount() uint64

GetMaxSnapshotCount mock method

func (*MockSchedulerOptions) GetMaxStoreDownTime

func (mso *MockSchedulerOptions) GetMaxStoreDownTime() time.Duration

GetMaxStoreDownTime mock method

func (*MockSchedulerOptions) GetMergeScheduleLimit

func (mso *MockSchedulerOptions) GetMergeScheduleLimit(name string) uint64

GetMergeScheduleLimit mock method

func (*MockSchedulerOptions) GetRegionScheduleLimit

func (mso *MockSchedulerOptions) GetRegionScheduleLimit(name string) uint64

GetRegionScheduleLimit mock method

func (*MockSchedulerOptions) GetReplicaScheduleLimit

func (mso *MockSchedulerOptions) GetReplicaScheduleLimit(name string) uint64

GetReplicaScheduleLimit mock method

func (*MockSchedulerOptions) GetSplitMergeInterval

func (mso *MockSchedulerOptions) GetSplitMergeInterval() time.Duration

GetSplitMergeInterval mock method

func (*MockSchedulerOptions) GetTolerantSizeRatio

func (mso *MockSchedulerOptions) GetTolerantSizeRatio() float64

GetTolerantSizeRatio mock method

func (*MockSchedulerOptions) IsLocationReplacementEnabled

func (mso *MockSchedulerOptions) IsLocationReplacementEnabled() bool

IsLocationReplacementEnabled mock method.

func (*MockSchedulerOptions) IsMakeUpReplicaEnabled

func (mso *MockSchedulerOptions) IsMakeUpReplicaEnabled() bool

IsMakeUpReplicaEnabled mock method.

func (*MockSchedulerOptions) IsNamespaceRelocationEnabled

func (mso *MockSchedulerOptions) IsNamespaceRelocationEnabled() bool

IsNamespaceRelocationEnabled mock method.

func (*MockSchedulerOptions) IsRaftLearnerEnabled

func (mso *MockSchedulerOptions) IsRaftLearnerEnabled() bool

IsRaftLearnerEnabled mock method

func (*MockSchedulerOptions) IsRemoveDownReplicaEnabled

func (mso *MockSchedulerOptions) IsRemoveDownReplicaEnabled() bool

IsRemoveDownReplicaEnabled mock method.

func (*MockSchedulerOptions) IsRemoveExtraReplicaEnabled

func (mso *MockSchedulerOptions) IsRemoveExtraReplicaEnabled() bool

IsRemoveExtraReplicaEnabled mock method.

func (*MockSchedulerOptions) IsReplaceOfflineReplicaEnabled

func (mso *MockSchedulerOptions) IsReplaceOfflineReplicaEnabled() bool

IsReplaceOfflineReplicaEnabled mock method.

func (*MockSchedulerOptions) SetMaxReplicas

func (mso *MockSchedulerOptions) SetMaxReplicas(replicas int)

SetMaxReplicas mock method

type NamespaceChecker

type NamespaceChecker struct {
	// contains filtered or unexported fields
}

NamespaceChecker ensures region to go to the right place.

func NewNamespaceChecker

func NewNamespaceChecker(cluster Cluster, classifier namespace.Classifier) *NamespaceChecker

NewNamespaceChecker creates a namespace checker.

func (*NamespaceChecker) Check

func (n *NamespaceChecker) Check(region *core.RegionInfo) *Operator

Check verifies a region's namespace, creating an Operator if need.

func (*NamespaceChecker) SelectBestPeerToRelocate

func (n *NamespaceChecker) SelectBestPeerToRelocate(region *core.RegionInfo, targets []*core.StoreInfo) *metapb.Peer

SelectBestPeerToRelocate return a new peer that to be used to move a region

func (*NamespaceChecker) SelectBestStoreToRelocate

func (n *NamespaceChecker) SelectBestStoreToRelocate(region *core.RegionInfo, targets []*core.StoreInfo) uint64

SelectBestStoreToRelocate randomly returns the store to relocate

type NamespaceOptions

type NamespaceOptions interface {
	GetLeaderScheduleLimit(name string) uint64
	GetRegionScheduleLimit(name string) uint64
	GetReplicaScheduleLimit(name string) uint64
	GetMergeScheduleLimit(name string) uint64
	GetMaxReplicas(name string) int
}

NamespaceOptions for namespace cluster.

type OpInfluence

type OpInfluence struct {
	// contains filtered or unexported fields
}

OpInfluence records the influence of the cluster.

func NewOpInfluence

func NewOpInfluence(operators []*Operator, cluster Cluster) OpInfluence

NewOpInfluence creates a OpInfluence.

func (OpInfluence) GetRegionsInfluence

func (m OpInfluence) GetRegionsInfluence() map[uint64]*Operator

GetRegionsInfluence gets regionInfluence of specific region.

func (OpInfluence) GetStoreInfluence

func (m OpInfluence) GetStoreInfluence(id uint64) *StoreInfluence

GetStoreInfluence get storeInfluence of specific store.

type Operator

type Operator struct {
	// contains filtered or unexported fields
}

Operator contains execution steps generated by scheduler.

func CreateMovePeerOperator

func CreateMovePeerOperator(desc string, cluster Cluster, region *core.RegionInfo, kind OperatorKind, oldStore, newStore uint64, peerID uint64) (*Operator, error)

CreateMovePeerOperator creates an Operator that replaces an old peer with a new peer.

func CreateRemovePeerOperator

func CreateRemovePeerOperator(desc string, cluster Cluster, kind OperatorKind, region *core.RegionInfo, storeID uint64) (*Operator, error)

CreateRemovePeerOperator creates an Operator that removes a peer from region.

func NewOperator

func NewOperator(desc string, regionID uint64, regionEpoch *metapb.RegionEpoch, kind OperatorKind, steps ...OperatorStep) *Operator

NewOperator creates a new operator.

func (*Operator) AttachKind

func (o *Operator) AttachKind(kind OperatorKind)

AttachKind attaches an operator kind for the operator.

func (*Operator) Check

func (o *Operator) Check(region *core.RegionInfo) OperatorStep

Check checks if current step is finished, returns next step to take action. It's safe to be called by multiple goroutine concurrently.

func (*Operator) Desc

func (o *Operator) Desc() string

Desc returns the operator's short description.

func (*Operator) ElapsedTime

func (o *Operator) ElapsedTime() time.Duration

ElapsedTime returns duration since it was created.

func (*Operator) GetPriorityLevel

func (o *Operator) GetPriorityLevel() core.PriorityLevel

GetPriorityLevel get the priority level

func (*Operator) History

func (o *Operator) History() []OperatorHistory

History transfers the operator's steps to operator histories.

func (*Operator) Influence

func (o *Operator) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference which unfinished operator steps make

func (*Operator) IsFinish

func (o *Operator) IsFinish() bool

IsFinish checks if all steps are finished.

func (*Operator) IsTimeout

func (o *Operator) IsTimeout() bool

IsTimeout checks the operator's create time and determines if it is timeout.

func (*Operator) Kind

func (o *Operator) Kind() OperatorKind

Kind returns operator's kind.

func (*Operator) Len

func (o *Operator) Len() int

Len returns the operator's steps count.

func (*Operator) MarshalJSON

func (o *Operator) MarshalJSON() ([]byte, error)

MarshalJSON serialize custom types to JSON

func (*Operator) RegionEpoch

func (o *Operator) RegionEpoch() *metapb.RegionEpoch

RegionEpoch returns the region's epoch that is attched to the operator.

func (*Operator) RegionID

func (o *Operator) RegionID() uint64

RegionID returns the region that operator is targeted.

func (*Operator) SetDesc

func (o *Operator) SetDesc(desc string)

SetDesc sets the description for the operator.

func (*Operator) SetPriorityLevel

func (o *Operator) SetPriorityLevel(level core.PriorityLevel)

SetPriorityLevel set the priority level for operator

func (*Operator) Step

func (o *Operator) Step(i int) OperatorStep

Step returns the i-th step.

func (*Operator) String

func (o *Operator) String() string

type OperatorHistory

type OperatorHistory struct {
	FinishTime time.Time
	From, To   uint64
	Kind       core.ResourceKind
}

OperatorHistory is used to log and visualize completed operators.

type OperatorKind

type OperatorKind uint32

OperatorKind is a bit field to identify operator types.

const (
	OpLeader    OperatorKind = 1 << iota // Include leader transfer.
	OpRegion                             // Include peer movement.
	OpAdmin                              // Initiated by admin.
	OpHotRegion                          // Initiated by hot region scheduler.
	OpAdjacent                           // Initiated by adjacent region scheduler.
	OpReplica                            // Initiated by replica checkers.
	OpBalance                            // Initiated by balancers.
	OpMerge                              // Initiated by merge checkers or merge schedulers.
	OpRange                              // Initiated by range scheduler.

)

Flags for operators.

func ParseOperatorKind

func ParseOperatorKind(str string) (OperatorKind, error)

ParseOperatorKind converts string (flag name list concat by ',') to OperatorKind.

func (OperatorKind) String

func (k OperatorKind) String() string

type OperatorStep

type OperatorStep interface {
	fmt.Stringer
	IsFinish(region *core.RegionInfo) bool
	Influence(opInfluence OpInfluence, region *core.RegionInfo)
}

OperatorStep describes the basic scheduling steps that can not be subdivided.

func CreateAddPeerSteps

func CreateAddPeerSteps(newStore uint64, peerID uint64, cluster Cluster) []OperatorStep

CreateAddPeerSteps creates an OperatorStep list that add a new Peer.

type Options

type Options interface {
	GetLeaderScheduleLimit() uint64
	GetRegionScheduleLimit() uint64
	GetReplicaScheduleLimit() uint64
	GetMergeScheduleLimit() uint64
	GetHotRegionScheduleLimit() uint64

	GetMaxSnapshotCount() uint64
	GetMaxPendingPeerCount() uint64
	GetMaxStoreDownTime() time.Duration
	GetMaxMergeRegionSize() uint64
	GetMaxMergeRegionKeys() uint64
	GetSplitMergeInterval() time.Duration

	GetMaxReplicas() int
	GetLocationLabels() []string

	GetHotRegionCacheHitsThreshold() int
	GetTolerantSizeRatio() float64
	GetLowSpaceRatio() float64
	GetHighSpaceRatio() float64

	IsRaftLearnerEnabled() bool

	IsRemoveDownReplicaEnabled() bool
	IsReplaceOfflineReplicaEnabled() bool
	IsMakeUpReplicaEnabled() bool
	IsRemoveExtraReplicaEnabled() bool
	IsLocationReplacementEnabled() bool
	IsNamespaceRelocationEnabled() bool

	CheckLabelProperty(typ string, labels []*metapb.StoreLabel) bool
}

Options for schedulers.

type PromoteLearner

type PromoteLearner struct {
	ToStore, PeerID uint64
}

PromoteLearner is an OperatorStep that promotes a region learner peer to normal voter.

func (PromoteLearner) Influence

func (pl PromoteLearner) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make

func (PromoteLearner) IsFinish

func (pl PromoteLearner) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (PromoteLearner) String

func (pl PromoteLearner) String() string

type RandomSelector

type RandomSelector struct {
	// contains filtered or unexported fields
}

RandomSelector selects source/target store randomly.

func NewRandomSelector

func NewRandomSelector(filters []Filter) *RandomSelector

NewRandomSelector creates a RandomSelector instance.

func (*RandomSelector) SelectSource

func (s *RandomSelector) SelectSource(opt Options, stores []*core.StoreInfo) *core.StoreInfo

SelectSource randomly selects a source store from those can pass all filters.

func (*RandomSelector) SelectTarget

func (s *RandomSelector) SelectTarget(opt Options, stores []*core.StoreInfo, filters ...Filter) *core.StoreInfo

SelectTarget randomly selects a target store from those can pass all filters.

type RangeCluster

type RangeCluster struct {
	Cluster
	// contains filtered or unexported fields
}

RangeCluster isolates the cluster by range.

func GenRangeCluster

func GenRangeCluster(cluster Cluster, startKey, endKey []byte) *RangeCluster

GenRangeCluster gets a range cluster by specifying start key and end key.

func (*RangeCluster) GetAverageRegionSize

func (r *RangeCluster) GetAverageRegionSize() int64

GetAverageRegionSize returns the average region approximate size.

func (*RangeCluster) GetFollowerStores

func (r *RangeCluster) GetFollowerStores(region *core.RegionInfo) []*core.StoreInfo

GetFollowerStores returns all stores that contains the region's follower peer.

func (*RangeCluster) GetLeaderStore

func (r *RangeCluster) GetLeaderStore(region *core.RegionInfo) *core.StoreInfo

GetLeaderStore returns all stores that contains the region's leader peer.

func (*RangeCluster) GetRegionStores

func (r *RangeCluster) GetRegionStores(region *core.RegionInfo) []*core.StoreInfo

GetRegionStores returns all stores that contains the region's peer.

func (*RangeCluster) GetStore

func (r *RangeCluster) GetStore(id uint64) *core.StoreInfo

GetStore searches for a store by ID.

func (*RangeCluster) GetStores

func (r *RangeCluster) GetStores() []*core.StoreInfo

GetStores returns all Stores in the cluster.

func (*RangeCluster) GetTolerantSizeRatio

func (r *RangeCluster) GetTolerantSizeRatio() float64

GetTolerantSizeRatio gets the tolerant size ratio.

func (*RangeCluster) RandFollowerRegion

func (r *RangeCluster) RandFollowerRegion(storeID uint64, opts ...core.RegionOption) *core.RegionInfo

RandFollowerRegion returns a random region that has a follower on the store.

func (*RangeCluster) RandLeaderRegion

func (r *RangeCluster) RandLeaderRegion(storeID uint64, opts ...core.RegionOption) *core.RegionInfo

RandLeaderRegion returns a random region that has leader on the store.

func (*RangeCluster) SetTolerantSizeRatio

func (r *RangeCluster) SetTolerantSizeRatio(ratio float64)

SetTolerantSizeRatio sets the tolerant size ratio.

type RegionScatterer

type RegionScatterer struct {
	// contains filtered or unexported fields
}

RegionScatterer scatters regions.

func NewRegionScatterer

func NewRegionScatterer(cluster Cluster, classifier namespace.Classifier) *RegionScatterer

NewRegionScatterer creates a region scatterer.

func (*RegionScatterer) Scatter

func (r *RegionScatterer) Scatter(region *core.RegionInfo) (*Operator, error)

Scatter relocates the region.

type RemovePeer

type RemovePeer struct {
	FromStore uint64
}

RemovePeer is an OperatorStep that removes a region peer.

func (RemovePeer) Influence

func (rp RemovePeer) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make

func (RemovePeer) IsFinish

func (rp RemovePeer) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (RemovePeer) String

func (rp RemovePeer) String() string

type ReplicaChecker

type ReplicaChecker struct {
	// contains filtered or unexported fields
}

ReplicaChecker ensures region has the best replicas.

func NewReplicaChecker

func NewReplicaChecker(cluster Cluster, classifier namespace.Classifier) *ReplicaChecker

NewReplicaChecker creates a replica checker.

func (*ReplicaChecker) Check

func (r *ReplicaChecker) Check(region *core.RegionInfo) *Operator

Check verifies a region's replicas, creating an Operator if need.

func (*ReplicaChecker) SelectBestReplacementStore

func (r *ReplicaChecker) SelectBestReplacementStore(region *core.RegionInfo, oldPeer *metapb.Peer, filters ...Filter) (uint64, float64)

SelectBestReplacementStore returns a store id that to be used to replace the old peer and distinct score.

type ReplicaSelector

type ReplicaSelector struct {
	// contains filtered or unexported fields
}

ReplicaSelector selects source/target store candidates based on their distinct scores based on a region's peer stores.

func NewReplicaSelector

func NewReplicaSelector(regionStores []*core.StoreInfo, labels []string, filters ...Filter) *ReplicaSelector

NewReplicaSelector creates a ReplicaSelector instance.

func (*ReplicaSelector) SelectSource

func (s *ReplicaSelector) SelectSource(opt Options, stores []*core.StoreInfo) *core.StoreInfo

SelectSource selects the store that can pass all filters and has the minimal distinct score.

func (*ReplicaSelector) SelectTarget

func (s *ReplicaSelector) SelectTarget(opt Options, stores []*core.StoreInfo, filters ...Filter) *core.StoreInfo

SelectTarget selects the store that can pass all filters and has the maximal distinct score.

type Scheduler

type Scheduler interface {
	GetName() string
	// GetType should in accordance with the name passing to schedule.RegisterScheduler()
	GetType() string
	GetMinInterval() time.Duration
	GetNextInterval(interval time.Duration) time.Duration
	Prepare(cluster Cluster) error
	Cleanup(cluster Cluster)
	Schedule(cluster Cluster, opInfluence OpInfluence) []*Operator
	IsScheduleAllowed(cluster Cluster) bool
}

Scheduler is an interface to schedule resources.

func CreateScheduler

func CreateScheduler(name string, limiter *Limiter, args ...string) (Scheduler, error)

CreateScheduler creates a scheduler with registered creator func.

type SplitRegion

type SplitRegion struct {
	StartKey, EndKey []byte
	Policy           pdpb.CheckPolicy
}

SplitRegion is an OperatorStep that splits a region.

func (SplitRegion) Influence

func (sr SplitRegion) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make.

func (SplitRegion) IsFinish

func (sr SplitRegion) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (SplitRegion) String

func (sr SplitRegion) String() string

type StoreInfluence

type StoreInfluence struct {
	RegionSize  int64
	RegionCount int64
	LeaderSize  int64
	LeaderCount int64
}

StoreInfluence records influences that pending operators will make.

func (StoreInfluence) ResourceSize

func (s StoreInfluence) ResourceSize(kind core.ResourceKind) int64

ResourceSize returns delta size of leader/region by influence.

type StoreStateFilter

type StoreStateFilter struct {
	// Set true if the schedule involves any transfer leader operation.
	TransferLeader bool
	// Set true if the schedule involves any move region operation.
	MoveRegion bool
}

StoreStateFilter is used to determine whether a store can be selected as the source or target of the schedule based on the store's state.

func (StoreStateFilter) FilterSource

func (f StoreStateFilter) FilterSource(opt Options, store *core.StoreInfo) bool

FilterSource returns true when the store cannot be selected as the schedule source.

func (StoreStateFilter) FilterTarget

func (f StoreStateFilter) FilterTarget(opt Options, store *core.StoreInfo) bool

FilterTarget returns true when the store cannot be selected as the schedule target.

func (StoreStateFilter) Type

func (f StoreStateFilter) Type() string

Type returns the type of the Filter.

type TransferLeader

type TransferLeader struct {
	FromStore, ToStore uint64
}

TransferLeader is an OperatorStep that transfers a region's leader.

func (TransferLeader) Influence

func (tl TransferLeader) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make

func (TransferLeader) IsFinish

func (tl TransferLeader) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (TransferLeader) String

func (tl TransferLeader) String() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL