schedule

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 15, 2023 License: Apache-2.0 Imports: 43 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	DispatchFromHeartBeat     = "heartbeat"
	DispatchFromNotifierQueue = "active push"
	DispatchFromCreate        = "create"
)

The source of dispatched region.

Variables

View Source 
var (

	// PushOperatorTickInterval is the interval try to push the operator.
	PushOperatorTickInterval = 500 * time.Millisecond
	// StoreBalanceBaseTime represents the base time of balance rate.
	StoreBalanceBaseTime float64 = 60
	// FastOperatorFinishTime min finish time, if finish duration less than it,op will be pushed to fast operator queue
	FastOperatorFinishTime = 10 * time.Second
)
View Source 
var (

	// LabelerEventCounter is a counter of the scheduler labeler system.
	LabelerEventCounter = prometheus.NewCounterVec(
		prometheus.CounterOpts{
			Namespace: "pd",
			Subsystem: "schedule",
			Name:      "labeler_event_counter",
			Help:      "Counter of the scheduler label.",
		}, []string{"type", "event"})
)
View Source 
var PriorityWeight = []float64{1.0, 4.0, 9.0, 16.0}

PriorityWeight is used to represent the weight of different priorities of operators.

Functions

func AddOpInfluence 

func AddOpInfluence(op *operator.Operator, influence operator.OpInfluence, cluster Cluster)

AddOpInfluence add operator influence for cluster

func ApplyOperator 

func ApplyOperator(mc *mockcluster.Cluster, op *operator.Operator)

ApplyOperator applies operator. Only for test purpose.

func ApplyOperatorStep 

func ApplyOperatorStep(region *core.RegionInfo, op *operator.Operator) *core.RegionInfo

ApplyOperatorStep applies operator step. Only for test purpose.

func DecodeConfig 

func DecodeConfig(data []byte, v interface{}) error

DecodeConfig decode the custom config for each scheduler.

func EncodeConfig 

func EncodeConfig(v interface{}) ([]byte, error)

EncodeConfig encode the custom config for each scheduler.

func FindSchedulerTypeByName 

func FindSchedulerTypeByName(name string) string

FindSchedulerTypeByName finds the type of the specified name.

func NewTotalOpInfluence 

func NewTotalOpInfluence(operators []*operator.Operator, cluster Cluster) operator.OpInfluence

NewTotalOpInfluence creates a OpInfluence.

func RegisterScheduler 

func RegisterScheduler(typ string, createFn CreateSchedulerFunc)

RegisterScheduler binds a scheduler creator. It should be called in init() func of a package.

func RegisterSliceDecoderBuilder 

func RegisterSliceDecoderBuilder(typ string, builder ConfigSliceDecoderBuilder)

RegisterSliceDecoderBuilder convert arguments to config. It should be called in init() func of package.

Types

type Bucket 

type Bucket struct {
	// contains filtered or unexported fields
}

Bucket is used to maintain the operators created by a specific scheduler.

type Cluster 

type Cluster interface {
	core.RegionSetInformer
	core.StoreSetInformer
	core.StoreSetController

	statistics.RegionStatInformer
	statistics.StoreStatInformer
	buckets.BucketStatInformer

	operator.ClusterInformer

	RemoveScheduler(name string) error
	AddSuspectRegions(ids ...uint64)
	SetHotPendingInfluenceMetrics(storeLabel, rwTy, dim string, load float64)
	RecordOpStepWithTTL(regionID uint64)
}

Cluster provides an overview of a cluster's regions distribution.

type ConfigDecoder 

type ConfigDecoder func(v interface{}) error

ConfigDecoder used to decode the config.

func ConfigJSONDecoder 

func ConfigJSONDecoder(data []byte) ConfigDecoder

ConfigJSONDecoder used to build a json decoder of the config.

func ConfigSliceDecoder 

func ConfigSliceDecoder(name string, args []string) ConfigDecoder

ConfigSliceDecoder the default decode for the config.

type ConfigSliceDecoderBuilder 

type ConfigSliceDecoderBuilder func([]string) ConfigDecoder

ConfigSliceDecoderBuilder used to build slice decoder of the config.

type CreateSchedulerFunc 

type CreateSchedulerFunc func(opController *OperatorController, storage endpoint.ConfigStorage, dec ConfigDecoder) (Scheduler, error)

CreateSchedulerFunc is for creating scheduler.

type OperatorController 

type OperatorController struct {
	syncutil.RWMutex
	// contains filtered or unexported fields
}

OperatorController is used to limit the speed of scheduling.

func NewOperatorController 

func NewOperatorController(ctx context.Context, cluster Cluster, hbStreams *hbstream.HeartbeatStreams) *OperatorController

NewOperatorController creates a OperatorController.

func (*OperatorController) AddOperator 

func (oc *OperatorController) AddOperator(ops ...*operator.Operator) bool

AddOperator adds operators to the running operators.

func (*OperatorController) AddWaitingOperator 

func (oc *OperatorController) AddWaitingOperator(ops ...*operator.Operator) int

AddWaitingOperator adds operators to waiting operators.

func (*OperatorController) Ctx 

func (oc *OperatorController) Ctx() context.Context

Ctx returns a context which will be canceled once RaftCluster is stopped. For now, it is only used to control the lifetime of TTL cache in schedulers.

func (*OperatorController) Dispatch 

func (oc *OperatorController) Dispatch(region *core.RegionInfo, source string)

Dispatch is used to dispatch the operator of a region.

func (*OperatorController) ExceedStoreLimit 

func (oc *OperatorController) ExceedStoreLimit(ops ...*operator.Operator) bool

ExceedStoreLimit returns true if the store exceeds the cost limit after adding the operator. Otherwise, returns false.

func (*OperatorController) GetCluster 

func (oc *OperatorController) GetCluster() Cluster

GetCluster exports cluster to evict-scheduler for check store status.

func (*OperatorController) GetFastOpInfluence 

func (oc *OperatorController) GetFastOpInfluence(cluster Cluster, influence operator.OpInfluence)

GetFastOpInfluence get fast finish operator influence

func (*OperatorController) GetHistory 

func (oc *OperatorController) GetHistory(start time.Time) []operator.OpHistory

GetHistory gets operators' history.

func (*OperatorController) GetOpInfluence 

func (oc *OperatorController) GetOpInfluence(cluster Cluster) operator.OpInfluence

GetOpInfluence gets OpInfluence.

func (*OperatorController) GetOperator 

func (oc *OperatorController) GetOperator(regionID uint64) *operator.Operator

GetOperator gets a operator from the given region.

func (*OperatorController) GetOperatorStatus 

func (oc *OperatorController) GetOperatorStatus(id uint64) *OperatorWithStatus

GetOperatorStatus gets the operator and its status with the specify id.

func (*OperatorController) GetOperators 

func (oc *OperatorController) GetOperators() []*operator.Operator

GetOperators gets operators from the running operators.

func (*OperatorController) GetRecords 

func (oc *OperatorController) GetRecords(from time.Time) []*operator.OpRecord

GetRecords gets operators' records.

func (*OperatorController) GetWaitingOperators 

func (oc *OperatorController) GetWaitingOperators() []*operator.Operator

GetWaitingOperators gets operators from the waiting operators.

func (*OperatorController) OperatorCount 

func (oc *OperatorController) OperatorCount(kind operator.OpKind) uint64

OperatorCount gets the count of operators filtered by kind. kind only has one OpKind.

func (*OperatorController) PromoteWaitingOperator 

func (oc *OperatorController) PromoteWaitingOperator()

PromoteWaitingOperator promotes operators from waiting operators.

func (*OperatorController) PushOperators 

func (oc *OperatorController) PushOperators()

PushOperators periodically pushes the unfinished operator to the executor(TiKV).

func (*OperatorController) RemoveOperator 

func (oc *OperatorController) RemoveOperator(op *operator.Operator, extraFields ...zap.Field) bool

RemoveOperator removes a operator from the running operators.

func (*OperatorController) SendScheduleCommand 

func (oc *OperatorController) SendScheduleCommand(region *core.RegionInfo, step operator.OpStep, source string)

SendScheduleCommand sends a command to the region.

func (*OperatorController) SetOperator 

func (oc *OperatorController) SetOperator(op *operator.Operator)

SetOperator is only used for test.

type OperatorRecords 

type OperatorRecords struct {
	// contains filtered or unexported fields
}

OperatorRecords remains the operator and its status for a while.

func NewOperatorRecords 

func NewOperatorRecords(ctx context.Context) *OperatorRecords

NewOperatorRecords returns a OperatorRecords.

func (*OperatorRecords) Get 

func (o *OperatorRecords) Get(id uint64) *OperatorWithStatus

Get gets the operator and its status.

func (*OperatorRecords) Put 

func (o *OperatorRecords) Put(op *operator.Operator)

Put puts the operator and its status.

type OperatorWithStatus 

type OperatorWithStatus struct {
	*operator.Operator
	Status     pdpb.OperatorStatus
	FinishTime time.Time
}

OperatorWithStatus records the operator and its status.

func NewOperatorWithStatus 

func NewOperatorWithStatus(op *operator.Operator) *OperatorWithStatus

NewOperatorWithStatus creates an OperatorStatus from an operator.

func (*OperatorWithStatus) MarshalJSON 

func (o *OperatorWithStatus) MarshalJSON() ([]byte, error)

MarshalJSON returns the status of operator as a JSON string

type PluginInterface 

type PluginInterface struct {
	// contains filtered or unexported fields
}

PluginInterface is used to manage all plugin.

func NewPluginInterface 

func NewPluginInterface() *PluginInterface

NewPluginInterface create a plugin interface

func (*PluginInterface) GetFunction 

func (p *PluginInterface) GetFunction(path string, funcName string) (plugin.Symbol, error)

GetFunction gets func by funcName from plugin(.so)

type RandBuckets 

type RandBuckets struct {
	// contains filtered or unexported fields
}

RandBuckets is an implementation of waiting operators

func NewRandBuckets 

func NewRandBuckets() *RandBuckets

NewRandBuckets creates a random buckets.

func (*RandBuckets) GetOperator 

func (b *RandBuckets) GetOperator() []*operator.Operator

GetOperator gets an operator from the random buckets.

func (*RandBuckets) ListOperator 

func (b *RandBuckets) ListOperator() []*operator.Operator

ListOperator lists all operator in the random buckets.

func (*RandBuckets) PutOperator 

func (b *RandBuckets) PutOperator(op *operator.Operator)

PutOperator puts an operator into the random buckets.

type RangeCluster 

type RangeCluster struct {
	Cluster
	// contains filtered or unexported fields
}

RangeCluster isolates the cluster by range.

func GenRangeCluster 

func GenRangeCluster(cluster Cluster, startKey, endKey []byte) *RangeCluster

GenRangeCluster gets a range cluster by specifying start key and end key. The cluster can only know the regions within [startKey, endKey].

func (*RangeCluster) GetAverageRegionSize 

func (r *RangeCluster) GetAverageRegionSize() int64

GetAverageRegionSize returns the average region approximate size.

func (*RangeCluster) GetFollowerStores 

func (r *RangeCluster) GetFollowerStores(region *core.RegionInfo) []*core.StoreInfo

GetFollowerStores returns all stores that contains the region's follower peer.

func (*RangeCluster) GetLeaderStore 

func (r *RangeCluster) GetLeaderStore(region *core.RegionInfo) *core.StoreInfo

GetLeaderStore returns all stores that contains the region's leader peer.

func (*RangeCluster) GetRegionStores 

func (r *RangeCluster) GetRegionStores(region *core.RegionInfo) []*core.StoreInfo

GetRegionStores returns all stores that contains the region's peer.

func (*RangeCluster) GetStore 

func (r *RangeCluster) GetStore(id uint64) *core.StoreInfo

GetStore searches for a store by ID.

func (*RangeCluster) GetStores 

func (r *RangeCluster) GetStores() []*core.StoreInfo

GetStores returns all Stores in the cluster.

func (*RangeCluster) GetTolerantSizeRatio 

func (r *RangeCluster) GetTolerantSizeRatio() float64

GetTolerantSizeRatio gets the tolerant size ratio.

func (*RangeCluster) RandFollowerRegions 

func (r *RangeCluster) RandFollowerRegions(storeID uint64, ranges []core.KeyRange) []*core.RegionInfo

RandFollowerRegions returns a random region that has a follower on the store.

func (*RangeCluster) RandLeaderRegions 

func (r *RangeCluster) RandLeaderRegions(storeID uint64, ranges []core.KeyRange) []*core.RegionInfo

RandLeaderRegions returns a random region that has leader on the store.

func (*RangeCluster) SetTolerantSizeRatio 

func (r *RangeCluster) SetTolerantSizeRatio(ratio float64)

SetTolerantSizeRatio sets the tolerant size ratio.

type RegionScatterer 

type RegionScatterer struct {
	// contains filtered or unexported fields
}

RegionScatterer scatters regions.

func NewRegionScatterer 

func NewRegionScatterer(ctx context.Context, cluster Cluster, opController *OperatorController) *RegionScatterer

NewRegionScatterer creates a region scatterer. RegionScatter is used for the `Lightning`, it will scatter the specified regions before import data.

func (*RegionScatterer) Put 

func (r *RegionScatterer) Put(peers map[uint64]*metapb.Peer, leaderStoreID uint64, group string)

Put put the final distribution in the context no matter the operator was created

func (*RegionScatterer) Scatter 

func (r *RegionScatterer) Scatter(region *core.RegionInfo, group string) (*operator.Operator, error)

Scatter relocates the region. If the group is defined, the regions' leader with the same group would be scattered in a group level instead of cluster level.

func (*RegionScatterer) ScatterRegionsByID 

func (r *RegionScatterer) ScatterRegionsByID(regionsID []uint64, group string, retryLimit int) (int, map[uint64]error, error)

ScatterRegionsByID directly scatter regions by ScatterRegions

func (*RegionScatterer) ScatterRegionsByRange 

func (r *RegionScatterer) ScatterRegionsByRange(startKey, endKey []byte, group string, retryLimit int) (int, map[uint64]error, error)

ScatterRegionsByRange directly scatter regions by ScatterRegions

type RegionSplitter 

type RegionSplitter struct {
	// contains filtered or unexported fields
}

RegionSplitter handles split regions

func NewRegionSplitter 

func NewRegionSplitter(cluster Cluster, handler SplitRegionsHandler) *RegionSplitter

NewRegionSplitter return a region splitter

func (*RegionSplitter) SplitRegions 

func (r *RegionSplitter) SplitRegions(ctx context.Context, splitKeys [][]byte, retryLimit int) (int, []uint64)

SplitRegions support splitRegions by given split keys.

type Scheduler 

type Scheduler interface {
	http.Handler
	GetName() string
	// GetType should in accordance with the name passing to schedule.RegisterScheduler()
	GetType() string
	EncodeConfig() ([]byte, error)
	GetMinInterval() time.Duration
	GetNextInterval(interval time.Duration) time.Duration
	Prepare(cluster Cluster) error
	Cleanup(cluster Cluster)
	Schedule(cluster Cluster, dryRun bool) ([]*operator.Operator, []plan.Plan)
	IsScheduleAllowed(cluster Cluster) bool
}

Scheduler is an interface to schedule resources.

func CreateScheduler 

func CreateScheduler(typ string, opController *OperatorController, storage endpoint.ConfigStorage, dec ConfigDecoder) (Scheduler, error)

CreateScheduler creates a scheduler with registered creator func.

type SplitRegionsHandler 

type SplitRegionsHandler interface {
	SplitRegionByKeys(region *core.RegionInfo, splitKeys [][]byte) error
	ScanRegionsByKeyRange(groupKeys *regionGroupKeys, results *splitKeyResults)
}

SplitRegionsHandler used to handle region splitting

func NewSplitRegionsHandler 

func NewSplitRegionsHandler(cluster Cluster, oc *OperatorController) SplitRegionsHandler

NewSplitRegionsHandler return SplitRegionsHandler

type WaitingOperator 

type WaitingOperator interface {
	PutOperator(op *operator.Operator)
	GetOperator() []*operator.Operator
	ListOperator() []*operator.Operator
}

WaitingOperator is an interface of waiting operators.

type WaitingOperatorStatus 

type WaitingOperatorStatus struct {
	// contains filtered or unexported fields
}

WaitingOperatorStatus is used to limit the count of each kind of operators.

func NewWaitingOperatorStatus 

func NewWaitingOperatorStatus() *WaitingOperatorStatus

NewWaitingOperatorStatus creates a new WaitingOperatorStatus.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.