schedule

package
v2.0.0-rc.3+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 22, 2018 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

View Source 
const MaxOperatorWaitTime = 10 * time.Minute

MaxOperatorWaitTime is the duration that if an operator lives longer that it, the operator is considered timeout.

View Source 
const (
	// RegionHeartBeatReportInterval is the heartbeat report interval of a region
	RegionHeartBeatReportInterval = 60
)
View Source 
const (
	// RejectLeader is the label property type that sugguests a store should not
	// have any region leaders.
	RejectLeader = "reject-leader"
)

Variables

View Source 
var (
	// HotRegionLowThreshold is the low threadshold of hot region
	HotRegionLowThreshold = 3
)
View Source 
var Simulating bool

Simulating is an option to overpass the impact of accelerated time. Should only turned on by the simulator.

Functions

func CreateMergeRegionOperator 

func CreateMergeRegionOperator(desc string, cluster Cluster, source *core.RegionInfo, target *core.RegionInfo, kind OperatorKind) (*Operator, *Operator, error)

CreateMergeRegionOperator creates an Operator that merge two region into one

func DistinctScore 

func DistinctScore(labels []string, stores []*core.StoreInfo, other *core.StoreInfo) float64

DistinctScore returns the score that the other is distinct from the stores. A higher score means the other store is more different from the existed stores.

func FilterSource 

func FilterSource(opt Options, store *core.StoreInfo, filters []Filter) bool

FilterSource checks if store can pass all Filters as source store.

func FilterTarget 

func FilterTarget(opt Options, store *core.StoreInfo, filters []Filter) bool

FilterTarget checks if store can pass all Filters as target store.

func RegisterScheduler 

func RegisterScheduler(name string, createFn CreateSchedulerFunc)

RegisterScheduler binds a scheduler creator. It should be called in init() func of a package.

Types

type AddPeer 

type AddPeer struct {
	ToStore, PeerID uint64
}

AddPeer is an OperatorStep that adds a region peer.

func (AddPeer) Influence 

func (ap AddPeer) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make

func (AddPeer) IsFinish 

func (ap AddPeer) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (AddPeer) String 

func (ap AddPeer) String() string

type BasicCluster 

type BasicCluster struct {
	Stores   *core.StoresInfo
	Regions  *core.RegionsInfo
	HotCache *HotSpotCache
}

BasicCluster provides basic data member and interface for a tikv cluster.

func NewBasicCluster 

func NewBasicCluster() *BasicCluster

NewBasicCluster creates a BasicCluster.

func (*BasicCluster) BlockStore 

func (bc *BasicCluster) BlockStore(storeID uint64) error

BlockStore stops balancer from selecting the store.

func (*BasicCluster) CheckReadStatus 

func (bc *BasicCluster) CheckReadStatus(region *core.RegionInfo) (bool, *core.RegionStat)

CheckReadStatus checks the read status, returns whether need update statistics and item.

func (*BasicCluster) CheckWriteStatus 

func (bc *BasicCluster) CheckWriteStatus(region *core.RegionInfo) (bool, *core.RegionStat)

CheckWriteStatus checks the write status, returns whether need update statistics and item.

func (*BasicCluster) GetAdjacentRegions 

func (bc *BasicCluster) GetAdjacentRegions(region *core.RegionInfo) (*core.RegionInfo, *core.RegionInfo)

GetAdjacentRegions returns region's info that is adjacent with specific region

func (*BasicCluster) GetFollowerStores 

func (bc *BasicCluster) GetFollowerStores(region *core.RegionInfo) []*core.StoreInfo

GetFollowerStores returns all Stores that contains the region's follower peer.

func (*BasicCluster) GetLeaderStore 

func (bc *BasicCluster) GetLeaderStore(region *core.RegionInfo) *core.StoreInfo

GetLeaderStore returns all Stores that contains the region's leader peer.

func (*BasicCluster) GetRegion 

func (bc *BasicCluster) GetRegion(regionID uint64) *core.RegionInfo

GetRegion searches for a region by ID.

func (*BasicCluster) GetRegionStores 

func (bc *BasicCluster) GetRegionStores(region *core.RegionInfo) []*core.StoreInfo

GetRegionStores returns all Stores that contains the region's peer.

func (*BasicCluster) GetStore 

func (bc *BasicCluster) GetStore(storeID uint64) *core.StoreInfo

GetStore searches for a store by ID.

func (*BasicCluster) GetStores 

func (bc *BasicCluster) GetStores() []*core.StoreInfo

GetStores returns all Stores in the cluster.

func (*BasicCluster) IsRegionHot 

func (bc *BasicCluster) IsRegionHot(id uint64, hotThreshold int) bool

IsRegionHot checks if a region is in hot state.

func (*BasicCluster) PutRegion 

func (bc *BasicCluster) PutRegion(region *core.RegionInfo) error

PutRegion put a region

func (*BasicCluster) PutStore 

func (bc *BasicCluster) PutStore(store *core.StoreInfo) error

PutStore put a store

func (*BasicCluster) RandFollowerRegion 

func (bc *BasicCluster) RandFollowerRegion(storeID uint64) *core.RegionInfo

RandFollowerRegion returns a random region that has a follower on the store.

func (*BasicCluster) RandLeaderRegion 

func (bc *BasicCluster) RandLeaderRegion(storeID uint64) *core.RegionInfo

RandLeaderRegion returns a random region that has leader on the store.

func (*BasicCluster) RegionReadStats 

func (bc *BasicCluster) RegionReadStats() []*core.RegionStat

RegionReadStats returns hot region's read stats.

func (*BasicCluster) RegionWriteStats 

func (bc *BasicCluster) RegionWriteStats() []*core.RegionStat

RegionWriteStats returns hot region's write stats.

func (*BasicCluster) UnblockStore 

func (bc *BasicCluster) UnblockStore(storeID uint64)

UnblockStore allows balancer to select the store.

type Cluster 

type Cluster interface {
	RandFollowerRegion(storeID uint64) *core.RegionInfo
	RandLeaderRegion(storeID uint64) *core.RegionInfo

	GetStores() []*core.StoreInfo
	GetStore(id uint64) *core.StoreInfo
	GetRegion(id uint64) *core.RegionInfo
	GetRegionStores(region *core.RegionInfo) []*core.StoreInfo
	GetFollowerStores(region *core.RegionInfo) []*core.StoreInfo
	GetLeaderStore(region *core.RegionInfo) *core.StoreInfo
	GetAdjacentRegions(region *core.RegionInfo) (*core.RegionInfo, *core.RegionInfo)
	GetStoresAverageScore(kind core.ResourceKind, filters ...Filter) float64
	ScanRegions(startKey []byte, limit int) []*core.RegionInfo

	BlockStore(id uint64) error
	UnblockStore(id uint64)

	IsRegionHot(id uint64) bool
	RegionWriteStats() []*core.RegionStat
	RegionReadStats() []*core.RegionStat
	RandHotRegionFromStore(store uint64, kind FlowKind) *core.RegionInfo

	// get config methods
	GetOpt() NamespaceOptions
	Options

	// TODO: it should be removed. Schedulers don't need to know anything
	// about peers.
	AllocPeer(storeID uint64) (*metapb.Peer, error)
}

Cluster provides an overview of a cluster's regions distribution.

type CreateSchedulerFunc 

type CreateSchedulerFunc func(limiter *Limiter, args []string) (Scheduler, error)

CreateSchedulerFunc is for creating scheudler.

type Filter 

type Filter interface {
	// Return true if the store should not be used as a source store.
	FilterSource(opt Options, store *core.StoreInfo) bool
	// Return true if the store should not be used as a target store.
	FilterTarget(opt Options, store *core.StoreInfo) bool
}

Filter is an interface to filter source and target store.

func NewBlockFilter 

func NewBlockFilter() Filter

NewBlockFilter creates a Filter that filters all stores that are blocked from balance.

func NewCacheFilter 

func NewCacheFilter(cache *cache.TTLUint64) Filter

NewCacheFilter creates a Filter that filters all stores that are in the cache.

func NewDistinctScoreFilter 

func NewDistinctScoreFilter(labels []string, stores []*core.StoreInfo, source *core.StoreInfo) Filter

NewDistinctScoreFilter creates a filter that filters all stores that have lower distinct score than specified store.

func NewExcludedFilter 

func NewExcludedFilter(sources, targets map[uint64]struct{}) Filter

NewExcludedFilter creates a Filter that filters all specified stores.

func NewHealthFilter 

func NewHealthFilter() Filter

NewHealthFilter creates a Filter that filters all stores that are Busy or Down.

func NewNamespaceFilter 

func NewNamespaceFilter(classifier namespace.Classifier, namespace string) Filter

NewNamespaceFilter creates a Filter that filters all stores that are not belong to a namespace.

func NewPendingPeerCountFilter 

func NewPendingPeerCountFilter() Filter

NewPendingPeerCountFilter creates a Filter that filters all stores that are currently handling too many pengding peers.

func NewRejectLeaderFilter 

func NewRejectLeaderFilter() Filter

NewRejectLeaderFilter creates a Filter that filters stores that marked as rejectLeader from being the target of leader transfer.

func NewSnapshotCountFilter 

func NewSnapshotCountFilter() Filter

NewSnapshotCountFilter creates a Filter that filters all stores that are currently handling too many snapshots.

func NewStateFilter 

func NewStateFilter() Filter

NewStateFilter creates a Filter that filters all stores that are not UP.

func NewStorageThresholdFilter 

func NewStorageThresholdFilter() Filter

NewStorageThresholdFilter creates a Filter that filters all stores that are almost full.

type FlowKind 

type FlowKind uint32

FlowKind is a identify Flow types. 

const (
	WriteFlow FlowKind = iota
	ReadFlow
)

Flags for flow.

type HotSpotCache 

type HotSpotCache struct {
	// contains filtered or unexported fields
}

HotSpotCache is a cache hold hot regions.

func (*HotSpotCache) CheckRead 

func (w *HotSpotCache) CheckRead(region *core.RegionInfo, stores *core.StoresInfo) (bool, *core.RegionStat)

CheckRead checks the read status, returns whether need update statistics and item.

func (*HotSpotCache) CheckWrite 

func (w *HotSpotCache) CheckWrite(region *core.RegionInfo, stores *core.StoresInfo) (bool, *core.RegionStat)

CheckWrite checks the write status, returns whether need update statistics and item.

func (*HotSpotCache) RandHotRegionFromStore 

func (w *HotSpotCache) RandHotRegionFromStore(storeID uint64, kind FlowKind, hotThreshold int) *core.RegionStat

RandHotRegionFromStore random picks a hot region in specify store.

func (*HotSpotCache) RegionStats 

func (w *HotSpotCache) RegionStats(kind FlowKind) []*core.RegionStat

RegionStats returns hot items according to kind

func (*HotSpotCache) Update 

func (w *HotSpotCache) Update(key uint64, item *core.RegionStat, kind FlowKind)

Update updates the cache.

type Limiter 

type Limiter struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Limiter a counter that limits the number of operators

func NewLimiter 

func NewLimiter() *Limiter

NewLimiter create a schedule limiter

func (*Limiter) OperatorCount 

func (l *Limiter) OperatorCount(mask OperatorKind) uint64

OperatorCount gets the count of operators filtered by mask.

func (*Limiter) UpdateCounts 

func (l *Limiter) UpdateCounts(operators map[uint64]*Operator)

UpdateCounts updates resouce counts using current pending operators.

type MergeChecker 

type MergeChecker struct {
	// contains filtered or unexported fields
}

MergeChecker ensures region to merge with adjacent region when size is small

func NewMergeChecker 

func NewMergeChecker(cluster Cluster, classifier namespace.Classifier) *MergeChecker

NewMergeChecker creates a merge checker.

func (*MergeChecker) Check 

func (m *MergeChecker) Check(region *core.RegionInfo) (*Operator, *Operator)

Check verifies a region's replicas, creating an Operator if need.

type MergeRegion 

type MergeRegion struct {
	FromRegion *metapb.Region
	ToRegion   *metapb.Region
	// there are two regions involved in merge process,
	// so to keep them from other scheduler,
	// both of them should add MerRegion operatorStep.
	// But actually, tikv just need the region want to be merged to get the merge request,
	// thus use a IsPssive mark to indicate that
	// this region doesn't need to send merge request to tikv.
	IsPassive bool
}

MergeRegion is an OperatorStep that merge two regions.

func (MergeRegion) Influence 

func (mr MergeRegion) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make

func (MergeRegion) IsFinish 

func (mr MergeRegion) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished

func (MergeRegion) String 

func (mr MergeRegion) String() string

type NamespaceChecker 

type NamespaceChecker struct {
	// contains filtered or unexported fields
}

NamespaceChecker ensures region to go to the right place.

func NewNamespaceChecker 

func NewNamespaceChecker(cluster Cluster, classifier namespace.Classifier) *NamespaceChecker

NewNamespaceChecker creates a namespace checker.

func (*NamespaceChecker) Check 

func (n *NamespaceChecker) Check(region *core.RegionInfo) *Operator

Check verifies a region's namespace, creating an Operator if need.

func (*NamespaceChecker) SelectBestPeerToRelocate 

func (n *NamespaceChecker) SelectBestPeerToRelocate(region *core.RegionInfo, targets []*core.StoreInfo, filters ...Filter) *metapb.Peer

SelectBestPeerToRelocate return a new peer that to be used to move a region

func (*NamespaceChecker) SelectBestStoreToRelocate 

func (n *NamespaceChecker) SelectBestStoreToRelocate(region *core.RegionInfo, targets []*core.StoreInfo, filters ...Filter) uint64

SelectBestStoreToRelocate randomly returns the store to relocate

type NamespaceOptions 

type NamespaceOptions interface {
	GetLeaderScheduleLimit(name string) uint64
	GetRegionScheduleLimit(name string) uint64
	GetReplicaScheduleLimit(name string) uint64
	GetMergeScheduleLimit(name string) uint64
	GetMaxReplicas(name string) int
}

NamespaceOptions for namespace cluster.

type OpInfluence 

type OpInfluence map[uint64]*StoreInfluence

OpInfluence is a map of StoreInfluence.

func NewOpInfluence 

func NewOpInfluence(operators []*Operator, cluster Cluster) OpInfluence

NewOpInfluence creates a OpInfluence.

func (OpInfluence) GetStoreInfluence 

func (m OpInfluence) GetStoreInfluence(id uint64) *StoreInfluence

GetStoreInfluence get storeInfluence of specific store.

type Operator 

type Operator struct {
	// contains filtered or unexported fields
}

Operator contains execution steps generated by scheduler.

func CreateMovePeerOperator 

func CreateMovePeerOperator(desc string, cluster Cluster, region *core.RegionInfo, kind OperatorKind, oldStore, newStore uint64, peerID uint64) *Operator

CreateMovePeerOperator creates an Operator that replaces an old peer with a new peer

func CreateRemovePeerOperator 

func CreateRemovePeerOperator(desc string, cluster Cluster, kind OperatorKind, region *core.RegionInfo, storeID uint64) *Operator

CreateRemovePeerOperator creates an Operator that removes a peer from region.

func NewOperator 

func NewOperator(desc string, regionID uint64, kind OperatorKind, steps ...OperatorStep) *Operator

NewOperator creates a new operator.

func (*Operator) Check 

func (o *Operator) Check(region *core.RegionInfo) OperatorStep

Check checks if current step is finished, returns next step to take action. It's safe to be called by multiple goroutine concurrently.

func (*Operator) Desc 

func (o *Operator) Desc() string

Desc returns the operator's short description.

func (*Operator) ElapsedTime 

func (o *Operator) ElapsedTime() time.Duration

ElapsedTime returns duration since it was created.

func (*Operator) GetPriorityLevel 

func (o *Operator) GetPriorityLevel() core.PriorityLevel

GetPriorityLevel get the priority level

func (*Operator) History 

func (o *Operator) History() []OperatorHistory

History transfers the operator's steps to operator histories.

func (*Operator) Influence 

func (o *Operator) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference which unfinished operator steps make

func (*Operator) IsFinish 

func (o *Operator) IsFinish() bool

IsFinish checks if all steps are finished.

func (*Operator) IsTimeout 

func (o *Operator) IsTimeout() bool

IsTimeout checks the operator's create time and determines if it is timeout.

func (*Operator) Kind 

func (o *Operator) Kind() OperatorKind

Kind returns operator's kind.

func (*Operator) Len 

func (o *Operator) Len() int

Len returns the operator's steps count.

func (*Operator) MarshalJSON 

func (o *Operator) MarshalJSON() ([]byte, error)

MarshalJSON serialize custom types to JSON

func (*Operator) RegionID 

func (o *Operator) RegionID() uint64

RegionID returns the region that operator is targeted.

func (*Operator) SetPriorityLevel 

func (o *Operator) SetPriorityLevel(level core.PriorityLevel)

SetPriorityLevel set the priority level for operator

func (*Operator) Step 

func (o *Operator) Step(i int) OperatorStep

Step returns the i-th step.

func (*Operator) String 

func (o *Operator) String() string

type OperatorHistory 

type OperatorHistory struct {
	FinishTime time.Time
	From, To   uint64
	Kind       core.ResourceKind
}

OperatorHistory is used to log and visualize completed operators.

type OperatorKind 

type OperatorKind uint32

OperatorKind is a bit field to identify operator types. 

const (
	OpLeader    OperatorKind = 1 << iota // Include leader transfer.
	OpRegion                             // Include peer movement.
	OpAdmin                              // Initiated by admin.
	OpHotRegion                          // Initiated by hot region scheduler.
	OpAdjacent                           // Initiated by adjacent region scheduler.
	OpReplica                            // Initiated by replica checkers.
	OpBalance                            // Initiated by balancers.
	OpMerge                              // Initiated by merge checkers.

)

Flags for operators.

func ParseOperatorKind 

func ParseOperatorKind(str string) (OperatorKind, error)

ParseOperatorKind converts string (flag name list concat by ',') to OperatorKind.

func (OperatorKind) String 

func (k OperatorKind) String() string

type OperatorStep 

type OperatorStep interface {
	fmt.Stringer
	IsFinish(region *core.RegionInfo) bool
	Influence(opInfluence OpInfluence, region *core.RegionInfo)
}

OperatorStep describes the basic scheduling steps that can not be subdivided.

type Options 

type Options interface {
	GetLeaderScheduleLimit() uint64
	GetRegionScheduleLimit() uint64
	GetReplicaScheduleLimit() uint64
	GetMergeScheduleLimit() uint64

	GetMaxSnapshotCount() uint64
	GetMaxPendingPeerCount() uint64
	GetMaxStoreDownTime() time.Duration
	GetMaxMergeRegionSize() uint64

	GetMaxReplicas() int
	GetLocationLabels() []string

	GetHotRegionLowThreshold() int
	GetTolerantSizeRatio() float64

	CheckLabelProperty(typ string, labels []*metapb.StoreLabel) bool
}

Options for schedulers.

type RegionScatterer 

type RegionScatterer struct {
	// contains filtered or unexported fields
}

RegionScatterer scatters regions.

func NewRegionScatterer 

func NewRegionScatterer(cluster Cluster, classifier namespace.Classifier) *RegionScatterer

NewRegionScatterer creates a region scatterer.

func (*RegionScatterer) Scatter 

func (r *RegionScatterer) Scatter(region *core.RegionInfo) *Operator

Scatter relocates the region.

type RemovePeer 

type RemovePeer struct {
	FromStore uint64
}

RemovePeer is an OperatorStep that removes a region peer.

func (RemovePeer) Influence 

func (rp RemovePeer) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make

func (RemovePeer) IsFinish 

func (rp RemovePeer) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (RemovePeer) String 

func (rp RemovePeer) String() string

type ReplicaChecker 

type ReplicaChecker struct {
	// contains filtered or unexported fields
}

ReplicaChecker ensures region has the best replicas.

func NewReplicaChecker 

func NewReplicaChecker(cluster Cluster, classifier namespace.Classifier) *ReplicaChecker

NewReplicaChecker creates a replica checker.

func (*ReplicaChecker) Check 

func (r *ReplicaChecker) Check(region *core.RegionInfo) *Operator

Check verifies a region's replicas, creating an Operator if need.

func (*ReplicaChecker) SelectBestReplacedPeerToAddReplica 

func (r *ReplicaChecker) SelectBestReplacedPeerToAddReplica(region *core.RegionInfo, oldPeer *metapb.Peer, filters ...Filter) *metapb.Peer

SelectBestReplacedPeerToAddReplica returns a new peer that to be used to replace the old peer and distinct score.

type Scheduler 

type Scheduler interface {
	GetName() string
	// GetType should in accordance with the name passing to schedule.RegisterScheduler()
	GetType() string
	GetMinInterval() time.Duration
	GetNextInterval(interval time.Duration) time.Duration
	Prepare(cluster Cluster) error
	Cleanup(cluster Cluster)
	Schedule(cluster Cluster, opInfluence OpInfluence) []*Operator
	IsScheduleAllowed(cluster Cluster) bool
}

Scheduler is an interface to schedule resources.

func CreateScheduler 

func CreateScheduler(name string, limiter *Limiter, args ...string) (Scheduler, error)

CreateScheduler creates a scheduler with registered creator func.

type Selector 

type Selector interface {
	SelectSource(opt Options, stores []*core.StoreInfo, filters ...Filter) *core.StoreInfo
	SelectTarget(opt Options, stores []*core.StoreInfo, filters ...Filter) *core.StoreInfo
	GetFilters() []Filter
}

Selector is an interface to select source and target store to schedule.

func NewBalanceSelector 

func NewBalanceSelector(kind core.ResourceKind, filters []Filter) Selector

NewBalanceSelector creates a Selector that select source/target store by their resource scores.

func NewRandomSelector 

func NewRandomSelector(filters []Filter) Selector

NewRandomSelector creates a selector that select store randomly.

func NewReplicaSelector 

func NewReplicaSelector(regionStores []*core.StoreInfo, labels []string, filters ...Filter) Selector

NewReplicaSelector creates a Selector that select source/target store by their distinct scores based on a region's peer stores.

type StoreInfluence 

type StoreInfluence struct {
	RegionSize  int
	RegionCount int
	LeaderSize  int
	LeaderCount int
}

StoreInfluence records influences that pending operators will make.

type TransferLeader 

type TransferLeader struct {
	FromStore, ToStore uint64
}

TransferLeader is an OperatorStep that transfers a region's leader.

func (TransferLeader) Influence 

func (tl TransferLeader) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step make

func (TransferLeader) IsFinish 

func (tl TransferLeader) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (TransferLeader) String 

func (tl TransferLeader) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.