schedule

package
v4.0.0-beta.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 18, 2020 License: Apache-2.0 Imports: 31 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	DispatchFromHeartBeat     = "heartbeat"
	DispatchFromNotifierQueue = "active push"
	DispatchFromCreate        = "create"
)

The source of dispatched region.

Variables

View Source 
var (

	// PushOperatorTickInterval is the interval try to push the operator.
	PushOperatorTickInterval = 500 * time.Millisecond
	// StoreBalanceBaseTime represents the base time of balance rate.
	StoreBalanceBaseTime float64 = 60
)
View Source 
var PriorityWeight = []float64{1.0, 4.0, 9.0}

PriorityWeight is used to represent the weight of different priorities of operators.

Functions

func ApplyOperator 

func ApplyOperator(mc *mockcluster.Cluster, op *operator.Operator)

ApplyOperator applies operator. Only for test purpose.

func ApplyOperatorStep 

func ApplyOperatorStep(region *core.RegionInfo, op *operator.Operator) *core.RegionInfo

ApplyOperatorStep applies operator step. Only for test purpose.

func DecodeConfig 

func DecodeConfig(data []byte, v interface{}) error

DecodeConfig decode the custom config for each scheduler.

func EncodeConfig 

func EncodeConfig(v interface{}) ([]byte, error)

EncodeConfig encode the custom config for each scheduler.

func FindSchedulerTypeByName 

func FindSchedulerTypeByName(name string) string

FindSchedulerTypeByName finds the type of the specified name.

func IsSchedulerRegistered 

func IsSchedulerRegistered(name string) bool

IsSchedulerRegistered check where the named scheduler type is registered.

func NewTotalOpInfluence 

func NewTotalOpInfluence(operators []*operator.Operator, cluster opt.Cluster) operator.OpInfluence

NewTotalOpInfluence creates a OpInfluence.

func RegisterScheduler 

func RegisterScheduler(typ string, createFn CreateSchedulerFunc)

RegisterScheduler binds a scheduler creator. It should be called in init() func of a package.

func RegisterSliceDecoderBuilder 

func RegisterSliceDecoderBuilder(typ string, builder ConfigSliceDecoderBuilder)

RegisterSliceDecoderBuilder convert arguments to config. It should be called in init() func of package.

Types

type Bucket 

type Bucket struct {
	// contains filtered or unexported fields
}

Bucket is used to maintain the operators created by a specific scheduler.

type CheckerController 

type CheckerController struct {
	// contains filtered or unexported fields
}

CheckerController is used to manage all checkers.

func NewCheckerController 

func NewCheckerController(ctx context.Context, cluster opt.Cluster, ruleManager *placement.RuleManager, opController *OperatorController) *CheckerController

NewCheckerController create a new CheckerController. TODO: isSupportMerge should be removed.

func (*CheckerController) CheckRegion 

func (c *CheckerController) CheckRegion(region *core.RegionInfo) (bool, []*operator.Operator)

CheckRegion will check the region and add a new operator if needed.

func (*CheckerController) GetMergeChecker 

func (c *CheckerController) GetMergeChecker() *checker.MergeChecker

GetMergeChecker returns the merge checker.

type ConfigDecoder 

type ConfigDecoder func(v interface{}) error

ConfigDecoder used to decode the config.

func ConfigJSONDecoder 

func ConfigJSONDecoder(data []byte) ConfigDecoder

ConfigJSONDecoder used to build a json decoder of the config.

func ConfigSliceDecoder 

func ConfigSliceDecoder(name string, args []string) ConfigDecoder

ConfigSliceDecoder the default decode for the config.

type ConfigSliceDecoderBuilder 

type ConfigSliceDecoderBuilder func([]string) ConfigDecoder

ConfigSliceDecoderBuilder used to build slice decoder of the config.

type CreateSchedulerFunc 

type CreateSchedulerFunc func(opController *OperatorController, storage *core.Storage, dec ConfigDecoder) (Scheduler, error)

CreateSchedulerFunc is for creating scheduler.

type OperatorController 

type OperatorController struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

OperatorController is used to limit the speed of scheduling.

func NewOperatorController 

func NewOperatorController(ctx context.Context, cluster opt.Cluster, hbStreams opt.HeartbeatStreams) *OperatorController

NewOperatorController creates a OperatorController.

func (*OperatorController) AddOperator 

func (oc *OperatorController) AddOperator(ops ...*operator.Operator) bool

AddOperator adds operators to the running operators.

func (*OperatorController) AddWaitingOperator 

func (oc *OperatorController) AddWaitingOperator(ops ...*operator.Operator) int

AddWaitingOperator adds operators to waiting operators.

func (*OperatorController) Ctx 

func (oc *OperatorController) Ctx() context.Context

Ctx returns a context which will be canceled once RaftCluster is stopped. For now, it is only used to control the lifetime of TTL cache in schedulers.

func (*OperatorController) Dispatch 

func (oc *OperatorController) Dispatch(region *core.RegionInfo, source string)

Dispatch is used to dispatch the operator of a region.

func (*OperatorController) GetAllStoresLimit 

func (oc *OperatorController) GetAllStoresLimit() map[uint64]*StoreLimit

GetAllStoresLimit is used to get limit of all stores.

func (*OperatorController) GetCluster 

func (oc *OperatorController) GetCluster() opt.Cluster

GetCluster exports cluster to evict-scheduler for check sctore status.

func (*OperatorController) GetHistory 

func (oc *OperatorController) GetHistory(start time.Time) []operator.OpHistory

GetHistory gets operators' history.

func (*OperatorController) GetLeaderSchedulePolicy 

func (oc *OperatorController) GetLeaderSchedulePolicy() core.SchedulePolicy

GetLeaderSchedulePolicy is to get leader schedule policy.

func (*OperatorController) GetOpInfluence 

func (oc *OperatorController) GetOpInfluence(cluster opt.Cluster) operator.OpInfluence

GetOpInfluence gets OpInfluence.

func (*OperatorController) GetOperator 

func (oc *OperatorController) GetOperator(regionID uint64) *operator.Operator

GetOperator gets a operator from the given region.

func (*OperatorController) GetOperatorStatus 

func (oc *OperatorController) GetOperatorStatus(id uint64) *OperatorWithStatus

GetOperatorStatus gets the operator and its status with the specify id.

func (*OperatorController) GetOperators 

func (oc *OperatorController) GetOperators() []*operator.Operator

GetOperators gets operators from the running operators.

func (*OperatorController) GetWaitingOperators 

func (oc *OperatorController) GetWaitingOperators() []*operator.Operator

GetWaitingOperators gets operators from the waiting operators.

func (*OperatorController) OperatorCount 

func (oc *OperatorController) OperatorCount(mask operator.OpKind) uint64

OperatorCount gets the count of operators filtered by mask.

func (*OperatorController) PromoteWaitingOperator 

func (oc *OperatorController) PromoteWaitingOperator()

PromoteWaitingOperator promotes operators from waiting operators.

func (*OperatorController) PruneHistory 

func (oc *OperatorController) PruneHistory()

PruneHistory prunes a part of operators' history.

func (*OperatorController) PushOperators 

func (oc *OperatorController) PushOperators()

PushOperators periodically pushes the unfinished operator to the executor(TiKV).

func (*OperatorController) RemoveOperator 

func (oc *OperatorController) RemoveOperator(op *operator.Operator) bool

RemoveOperator removes a operator from the running operators.

func (*OperatorController) RemoveStoreLimit 

func (oc *OperatorController) RemoveStoreLimit(storeID uint64)

RemoveStoreLimit removes the store limit for a given store ID.

func (*OperatorController) SendScheduleCommand 

func (oc *OperatorController) SendScheduleCommand(region *core.RegionInfo, step operator.OpStep, source string)

SendScheduleCommand sends a command to the region.

func (*OperatorController) SetAllStoresLimit 

func (oc *OperatorController) SetAllStoresLimit(rate float64, mode StoreLimitMode)

SetAllStoresLimit is used to set limit of all stores.

func (*OperatorController) SetAllStoresLimitAuto 

func (oc *OperatorController) SetAllStoresLimitAuto(rate float64)

SetAllStoresLimitAuto updates the store limit in StoreLimitAuto mode

func (*OperatorController) SetOperator 

func (oc *OperatorController) SetOperator(op *operator.Operator)

SetOperator is only used for test.

func (*OperatorController) SetStoreLimit 

func (oc *OperatorController) SetStoreLimit(storeID uint64, rate float64, mode StoreLimitMode)

SetStoreLimit is used to set the limit of a store.

type OperatorRecords 

type OperatorRecords struct {
	// contains filtered or unexported fields
}

OperatorRecords remains the operator and its status for a while.

func NewOperatorRecords 

func NewOperatorRecords(ctx context.Context) *OperatorRecords

NewOperatorRecords returns a OperatorRecords.

func (*OperatorRecords) Get 

func (o *OperatorRecords) Get(id uint64) *OperatorWithStatus

Get gets the operator and its status.

func (*OperatorRecords) Put 

func (o *OperatorRecords) Put(op *operator.Operator)

Put puts the operator and its status.

type OperatorWithStatus 

type OperatorWithStatus struct {
	Op     *operator.Operator
	Status pdpb.OperatorStatus
}

OperatorWithStatus records the operator and its status.

func NewOperatorWithStatus 

func NewOperatorWithStatus(op *operator.Operator) *OperatorWithStatus

NewOperatorWithStatus creates an OperatorStatus from an operator.

func (*OperatorWithStatus) MarshalJSON 

func (o *OperatorWithStatus) MarshalJSON() ([]byte, error)

MarshalJSON returns the status of operator as a JSON string

type PluginInterface 

type PluginInterface struct {
	// contains filtered or unexported fields
}

PluginInterface is used to manage all plugin.

func NewPluginInterface 

func NewPluginInterface() *PluginInterface

NewPluginInterface create a plugin interface

func (*PluginInterface) GetFunction 

func (p *PluginInterface) GetFunction(path string, funcName string) (plugin.Symbol, error)

GetFunction gets func by funcName from plugin(.so)

type RandBuckets 

type RandBuckets struct {
	// contains filtered or unexported fields
}

RandBuckets is an implementation of waiting operators

func NewRandBuckets 

func NewRandBuckets() *RandBuckets

NewRandBuckets creates a random buckets.

func (*RandBuckets) GetOperator 

func (b *RandBuckets) GetOperator() []*operator.Operator

GetOperator gets an operator from the random buckets.

func (*RandBuckets) ListOperator 

func (b *RandBuckets) ListOperator() []*operator.Operator

ListOperator lists all operator in the random buckets.

func (*RandBuckets) PutOperator 

func (b *RandBuckets) PutOperator(op *operator.Operator)

PutOperator puts an operator into the random buckets.

type RangeCluster 

type RangeCluster struct {
	opt.Cluster
	// contains filtered or unexported fields
}

RangeCluster isolates the cluster by range.

func GenRangeCluster 

func GenRangeCluster(cluster opt.Cluster, startKey, endKey []byte) *RangeCluster

GenRangeCluster gets a range cluster by specifying start key and end key. The cluster can only know the regions within [startKey, endKey].

func (*RangeCluster) GetAverageRegionSize 

func (r *RangeCluster) GetAverageRegionSize() int64

GetAverageRegionSize returns the average region approximate size.

func (*RangeCluster) GetFollowerStores 

func (r *RangeCluster) GetFollowerStores(region *core.RegionInfo) []*core.StoreInfo

GetFollowerStores returns all stores that contains the region's follower peer.

func (*RangeCluster) GetLeaderStore 

func (r *RangeCluster) GetLeaderStore(region *core.RegionInfo) *core.StoreInfo

GetLeaderStore returns all stores that contains the region's leader peer.

func (*RangeCluster) GetRegionStores 

func (r *RangeCluster) GetRegionStores(region *core.RegionInfo) []*core.StoreInfo

GetRegionStores returns all stores that contains the region's peer.

func (*RangeCluster) GetStore 

func (r *RangeCluster) GetStore(id uint64) *core.StoreInfo

GetStore searches for a store by ID.

func (*RangeCluster) GetStores 

func (r *RangeCluster) GetStores() []*core.StoreInfo

GetStores returns all Stores in the cluster.

func (*RangeCluster) GetTolerantSizeRatio 

func (r *RangeCluster) GetTolerantSizeRatio() float64

GetTolerantSizeRatio gets the tolerant size ratio.

func (*RangeCluster) RandFollowerRegion 

func (r *RangeCluster) RandFollowerRegion(storeID uint64, ranges []core.KeyRange, opts ...core.RegionOption) *core.RegionInfo

RandFollowerRegion returns a random region that has a follower on the store.

func (*RangeCluster) RandLeaderRegion 

func (r *RangeCluster) RandLeaderRegion(storeID uint64, ranges []core.KeyRange, opts ...core.RegionOption) *core.RegionInfo

RandLeaderRegion returns a random region that has leader on the store.

func (*RangeCluster) SetTolerantSizeRatio 

func (r *RangeCluster) SetTolerantSizeRatio(ratio float64)

SetTolerantSizeRatio sets the tolerant size ratio.

type RegionScatterer 

type RegionScatterer struct {
	// contains filtered or unexported fields
}

RegionScatterer scatters regions.

func NewRegionScatterer 

func NewRegionScatterer(cluster opt.Cluster) *RegionScatterer

NewRegionScatterer creates a region scatterer. RegionScatter is used for the `Lightning`, it will scatter the specified regions before import data.

func (*RegionScatterer) Scatter 

func (r *RegionScatterer) Scatter(region *core.RegionInfo) (*operator.Operator, error)

Scatter relocates the region.

type Scheduler 

type Scheduler interface {
	http.Handler
	GetName() string
	// GetType should in accordance with the name passing to schedule.RegisterScheduler()
	GetType() string
	EncodeConfig() ([]byte, error)
	GetMinInterval() time.Duration
	GetNextInterval(interval time.Duration) time.Duration
	Prepare(cluster opt.Cluster) error
	Cleanup(cluster opt.Cluster)
	Schedule(cluster opt.Cluster) []*operator.Operator
	IsScheduleAllowed(cluster opt.Cluster) bool
}

Scheduler is an interface to schedule resources.

func CreateScheduler 

func CreateScheduler(typ string, opController *OperatorController, storage *core.Storage, dec ConfigDecoder) (Scheduler, error)

CreateScheduler creates a scheduler with registered creator func.

type StoreLimit 

type StoreLimit struct {
	// contains filtered or unexported fields
}

StoreLimit limits the operators of a store

func NewStoreLimit 

func NewStoreLimit(rate float64, mode StoreLimitMode) *StoreLimit

NewStoreLimit returns a StoreLimit object

func (*StoreLimit) Available 

func (l *StoreLimit) Available() int64

Available returns the number of available tokens

func (*StoreLimit) Mode 

func (l *StoreLimit) Mode() StoreLimitMode

Mode returns the store limit mode

func (*StoreLimit) Rate 

func (l *StoreLimit) Rate() float64

Rate returns the fill rate of the bucket, in tokens per second.

func (*StoreLimit) Take 

func (l *StoreLimit) Take(count int64) time.Duration

Take takes count tokens from the bucket without blocking.

type StoreLimitMode 

type StoreLimitMode int

StoreLimitMode indicates the strategy to set store limit 

const (
	StoreLimitAuto StoreLimitMode = iota
	StoreLimitManual
)

There are two modes supported now, "auto" indicates the value is set by PD itself. "manual" means it is set by the user. An auto set value can be overwrite by a manual set value.

func (StoreLimitMode) String 

func (m StoreLimitMode) String() string

String returns the representation of the StoreLimitMode

type StoreLimitScene 

type StoreLimitScene struct {
	Idle   int
	Low    int
	Normal int
	High   int
}

StoreLimitScene defines the store limitation on difference scenes Idle/Low/Normal/High indicates the load of the cluster, it is defined in cluster.State. See the details there for how to calculate the load. The values here defines the store-limit for each load. For example: Idle = 60, means that change the store-limit to 60 when the cluster is idle.

func DefaultStoreLimitScene 

func DefaultStoreLimitScene() *StoreLimitScene

DefaultStoreLimitScene returns StoreLimitScene object with default values

type WaitingOperator 

type WaitingOperator interface {
	PutOperator(op *operator.Operator)
	GetOperator() []*operator.Operator
	ListOperator() []*operator.Operator
}

WaitingOperator is an interface of waiting operators.

type WaitingOperatorStatus 

type WaitingOperatorStatus struct {
	// contains filtered or unexported fields
}

WaitingOperatorStatus is used to limit the count of each kind of operators.

func NewWaitingOperatorStatus 

func NewWaitingOperatorStatus() *WaitingOperatorStatus

NewWaitingOperatorStatus creates a new WaitingOperatorStatus.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.