operator

package
v3.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 10, 2020 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// OperatorExpireTime is the duration that when an operator is not started
	// after it, the operator will be considered expired.
	OperatorExpireTime = 3 * time.Second
	// LeaderOperatorWaitTime is the duration that when a leader operator runs
	// longer than it, the operator will be considered timeout.
	LeaderOperatorWaitTime = 10 * time.Second
	// RegionOperatorWaitTime is the duration that when a region operator runs
	// longer than it, the operator will be considered timeout.
	RegionOperatorWaitTime = 10 * time.Minute
)
View Source 
const (
	// RegionInfluence represents the influence of a operator step, which is used by ratelimit.
	RegionInfluence int64 = 1000
)

Variables

This section is empty.

Functions

func IsEndStatus 

func IsEndStatus(s OpStatus) bool

IsEndStatus checks whether s is an end status.

func OpStatusToPDPB 

func OpStatusToPDPB(s OpStatus) pdpb.OperatorStatus

OpStatusToPDPB converts OpStatus to pdpb.OperatorStatus.

func OpStatusToString 

func OpStatusToString(s OpStatus) string

OpStatusToString converts Status to string.

func SetOperatorStatusReachTime 

func SetOperatorStatusReachTime(op *Operator, st OpStatus, t time.Time)

SetOperatorStatusReachTime sets the reach time of the operator. NOTE: Should only use in test.

Types

type AddLearner 

type AddLearner struct {
	ToStore, PeerID uint64
}

AddLearner is an OpStep that adds a region learner peer.

func (AddLearner) ConfVerChanged 

func (al AddLearner) ConfVerChanged(region *core.RegionInfo) bool

ConfVerChanged returns true if the conf version has been changed by this step

func (AddLearner) Influence 

func (al AddLearner) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step makes.

func (AddLearner) IsFinish 

func (al AddLearner) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (AddLearner) String 

func (al AddLearner) String() string

type AddLightLearner 

type AddLightLearner struct {
	ToStore, PeerID uint64
}

AddLightLearner is an OpStep that adds a region learner peer without considering the influence.

func (AddLightLearner) ConfVerChanged 

func (al AddLightLearner) ConfVerChanged(region *core.RegionInfo) bool

ConfVerChanged returns true if the conf version has been changed by this step

func (AddLightLearner) Influence 

func (al AddLightLearner) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step makes.

func (AddLightLearner) IsFinish 

func (al AddLightLearner) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (AddLightLearner) String 

func (al AddLightLearner) String() string

type AddLightPeer 

type AddLightPeer struct {
	ToStore, PeerID uint64
}

AddLightPeer is an OpStep that adds a region peer without considering the influence.

func (AddLightPeer) ConfVerChanged 

func (ap AddLightPeer) ConfVerChanged(region *core.RegionInfo) bool

ConfVerChanged returns true if the conf version has been changed by this step

func (AddLightPeer) Influence 

func (ap AddLightPeer) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step makes.

func (AddLightPeer) IsFinish 

func (ap AddLightPeer) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (AddLightPeer) String 

func (ap AddLightPeer) String() string

type AddPeer 

type AddPeer struct {
	ToStore, PeerID uint64
}

AddPeer is an OpStep that adds a region peer.

func (AddPeer) ConfVerChanged 

func (ap AddPeer) ConfVerChanged(region *core.RegionInfo) bool

ConfVerChanged returns true if the conf version has been changed by this step

func (AddPeer) Influence 

func (ap AddPeer) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step makes.

func (AddPeer) IsFinish 

func (ap AddPeer) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (AddPeer) String 

func (ap AddPeer) String() string

type Builder 

type Builder struct {
	// contains filtered or unexported fields
}

Builder is used to create operators. Usage: 

op, err := NewBuilder(desc, cluster, region).
            RemovePeer(store1).
            AddPeer(peer1).
            SetLeader(store2).
            Build(kind)

The generated Operator will choose the most appropriate execution order according to various constraints.

func NewBuilder 

func NewBuilder(desc string, cluster Cluster, region *core.RegionInfo) *Builder

NewBuilder creates a Builder.

func (*Builder) AddPeer 

func (b *Builder) AddPeer(p *metapb.Peer) *Builder

AddPeer records an add Peer operation in Builder. If p.Id is 0, the builder will allocate a new peer ID later.

func (*Builder) Build 

func (b *Builder) Build(kind OpKind) (*Operator, error)

Build creates the Operator.

func (*Builder) PromoteLearner 

func (b *Builder) PromoteLearner(storeID uint64) *Builder

PromoteLearner records a promote learner operation in Builder.

func (*Builder) RemovePeer 

func (b *Builder) RemovePeer(storeID uint64) *Builder

RemovePeer records a remove peer operation in Builder.

func (*Builder) SetLeader 

func (b *Builder) SetLeader(storeID uint64) *Builder

SetLeader records the target leader in Builder.

func (*Builder) SetLightWeight 

func (b *Builder) SetLightWeight() *Builder

SetLightWeight marks the region as light weight. It is used for scatter regions.

func (*Builder) SetPeers 

func (b *Builder) SetPeers(peers map[uint64]*metapb.Peer) *Builder

SetPeers resets the target peer list.

If peer's ID is 0, the builder will allocate a new ID later. If current target leader does not exist in peers, it will be reset.

type Cluster 

type Cluster interface {
	opt.Options
	GetStore(id uint64) *core.StoreInfo
	AllocID() (uint64, error)
	FitRegion(region *core.RegionInfo) *placement.RegionFit
}

Cluster provides an overview of a cluster's regions distribution.

type MergeRegion 

type MergeRegion struct {
	FromRegion *metapb.Region
	ToRegion   *metapb.Region
	// there are two regions involved in merge process,
	// so to keep them from other scheduler,
	// both of them should add MerRegion operatorStep.
	// But actually, TiKV just needs the region want to be merged to get the merge request,
	// thus use a IsPassive mark to indicate that
	// this region doesn't need to send merge request to TiKV.
	IsPassive bool
}

MergeRegion is an OpStep that merge two regions.

func (MergeRegion) ConfVerChanged 

func (mr MergeRegion) ConfVerChanged(region *core.RegionInfo) bool

ConfVerChanged returns true if the conf version has been changed by this step

func (MergeRegion) Influence 

func (mr MergeRegion) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step makes.

func (MergeRegion) IsFinish 

func (mr MergeRegion) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (MergeRegion) String 

func (mr MergeRegion) String() string

type OpHistory 

type OpHistory struct {
	FinishTime time.Time
	From, To   uint64
	Kind       core.ResourceKind
}

OpHistory is used to log and visualize completed operators.

type OpInfluence 

type OpInfluence struct {
	StoresInfluence map[uint64]*StoreInfluence
}

OpInfluence records the influence of the cluster.

func (OpInfluence) GetStoreInfluence 

func (m OpInfluence) GetStoreInfluence(id uint64) *StoreInfluence

GetStoreInfluence get storeInfluence of specific store.

type OpKind 

type OpKind uint32

OpKind is a bit field to identify operator types. 

const (
	OpLeader    OpKind = 1 << iota // Include leader transfer.
	OpRegion                       // Include peer movement.
	OpSplit                        // Include region split.
	OpAdmin                        // Initiated by admin.
	OpHotRegion                    // Initiated by hot region scheduler.
	OpAdjacent                     // Initiated by adjacent region scheduler.
	OpReplica                      // Initiated by replica checkers.
	OpBalance                      // Initiated by balancers.
	OpMerge                        // Initiated by merge checkers or merge schedulers.
	OpRange                        // Initiated by range scheduler.

)

Flags for operators.

func ParseOperatorKind 

func ParseOperatorKind(str string) (OpKind, error)

ParseOperatorKind converts string (flag name list concat by ',') to OpKind.

func (OpKind) String 

func (k OpKind) String() string

type OpStatus 

type OpStatus = uint32

OpStatus represents the status of an Operator. 

const (
	// Status list
	// Just created. Next status: {RUNNING, CANCELED, EXPIRED}.
	CREATED OpStatus = iota
	// Started and not finished. Next status: {SUCCESS, CANCELED, REPLACED, TIMEOUT}.
	STARTED
	// Followings are end status, i.e. no next status.
	SUCCESS  // Finished successfully
	CANCELED // Canceled due to some reason
	REPLACED // Replaced by an higher priority operator
	EXPIRED  // Didn't start to run for too long
	TIMEOUT  // Running for too long

)

Status list

type OpStatusTracker 

type OpStatusTracker struct {
	// contains filtered or unexported fields
}

OpStatusTracker represents the status of an operator.

func NewOpStatusTracker 

func NewOpStatusTracker() OpStatusTracker

NewOpStatusTracker creates an OpStatus.

func (*OpStatusTracker) CheckExpired 

func (trk *OpStatusTracker) CheckExpired(exp time.Duration) bool

CheckExpired checks if expired, and update the current status.

func (*OpStatusTracker) CheckTimeout 

func (trk *OpStatusTracker) CheckTimeout(wait time.Duration) bool

CheckTimeout checks if timeout, and update the current status.

func (*OpStatusTracker) IsEnd 

func (trk *OpStatusTracker) IsEnd() bool

IsEnd checks whether the current status is an end status.

func (*OpStatusTracker) ReachTime 

func (trk *OpStatusTracker) ReachTime() time.Time

ReachTime returns the reach time of current status.

func (*OpStatusTracker) ReachTimeOf 

func (trk *OpStatusTracker) ReachTimeOf(s OpStatus) time.Time

ReachTimeOf returns the time when reached given status. If didn't reatched the given status, return zero.

func (*OpStatusTracker) Status 

func (trk *OpStatusTracker) Status() OpStatus

Status returns current status.

func (*OpStatusTracker) String 

func (trk *OpStatusTracker) String() string

String implements fmt.Stringer.

func (*OpStatusTracker) To 

func (trk *OpStatusTracker) To(dst OpStatus) bool

To transfer the current status to dst if this transition is valid, returns whether transferred.

type OpStep 

type OpStep interface {
	fmt.Stringer
	ConfVerChanged(region *core.RegionInfo) bool
	IsFinish(region *core.RegionInfo) bool
	Influence(opInfluence OpInfluence, region *core.RegionInfo)
}

OpStep describes the basic scheduling steps that can not be subdivided.

type Operator 

type Operator struct {
	Counters []prometheus.Counter
	// contains filtered or unexported fields
}

Operator contains execution steps generated by scheduler.

func CreateAddPeerOperator 

func CreateAddPeerOperator(desc string, cluster Cluster, region *core.RegionInfo, peer *metapb.Peer, kind OpKind) (*Operator, error)

CreateAddPeerOperator creates an operator that adds a new peer.

func CreateMergeRegionOperator 

func CreateMergeRegionOperator(desc string, cluster Cluster, source *core.RegionInfo, target *core.RegionInfo, kind OpKind) ([]*Operator, error)

CreateMergeRegionOperator creates an operator that merge two region into one.

func CreateMoveLeaderOperator 

func CreateMoveLeaderOperator(desc string, cluster Cluster, region *core.RegionInfo, kind OpKind, oldStore uint64, peer *metapb.Peer) (*Operator, error)

CreateMoveLeaderOperator creates an operator that replaces an old leader with a new leader.

func CreateMovePeerOperator 

func CreateMovePeerOperator(desc string, cluster Cluster, region *core.RegionInfo, kind OpKind, oldStore uint64, peer *metapb.Peer) (*Operator, error)

CreateMovePeerOperator creates an operator that replaces an old peer with a new peer.

func CreateMoveRegionOperator 

func CreateMoveRegionOperator(desc string, cluster Cluster, region *core.RegionInfo, kind OpKind, peers map[uint64]*metapb.Peer) (*Operator, error)

CreateMoveRegionOperator creates an operator that moves a region to specified stores.

func CreatePromoteLearnerOperator 

func CreatePromoteLearnerOperator(desc string, cluster Cluster, region *core.RegionInfo, peer *metapb.Peer) (*Operator, error)

CreatePromoteLearnerOperator creates an operator that promotes a learner.

func CreateRemovePeerOperator 

func CreateRemovePeerOperator(desc string, cluster Cluster, kind OpKind, region *core.RegionInfo, storeID uint64) (*Operator, error)

CreateRemovePeerOperator creates an operator that removes a peer from region.

func CreateScatterRegionOperator 

func CreateScatterRegionOperator(desc string, cluster Cluster, origin *core.RegionInfo, targetPeers map[uint64]*metapb.Peer) (*Operator, error)

CreateScatterRegionOperator creates an operator that scatters the specified region.

func CreateSplitRegionOperator 

func CreateSplitRegionOperator(desc string, region *core.RegionInfo, kind OpKind, policy pdpb.CheckPolicy, keys [][]byte) *Operator

CreateSplitRegionOperator creates an operator that splits a region.

func CreateTransferLeaderOperator 

func CreateTransferLeaderOperator(desc string, cluster Cluster, region *core.RegionInfo, sourceStoreID uint64, targetStoreID uint64, kind OpKind) (*Operator, error)

CreateTransferLeaderOperator creates an operator that transfers the leader from a source store to a target store.

func NewOperator 

func NewOperator(desc, brief string, regionID uint64, regionEpoch *metapb.RegionEpoch, kind OpKind, steps ...OpStep) *Operator

NewOperator creates a new operator.

func (*Operator) AttachKind 

func (o *Operator) AttachKind(kind OpKind)

AttachKind attaches an operator kind for the operator.

func (*Operator) Cancel 

func (o *Operator) Cancel() bool

Cancel marks the operator canceled.

func (*Operator) Check 

func (o *Operator) Check(region *core.RegionInfo) OpStep

Check checks if current step is finished, returns next step to take action. If operator is at an end status, check returns nil. It's safe to be called by multiple goroutine concurrently. FIXME: metrics is not thread-safe

func (*Operator) CheckExpired 

func (o *Operator) CheckExpired() bool

CheckExpired checks if the operator is expired, and update the status.

func (*Operator) CheckSuccess 

func (o *Operator) CheckSuccess() bool

CheckSuccess checks if all steps are finished, and update the status.

func (*Operator) CheckTimeout 

func (o *Operator) CheckTimeout() bool

CheckTimeout checks if the operator is timeout, and update the status.

func (*Operator) ConfVerChanged 

func (o *Operator) ConfVerChanged(region *core.RegionInfo) int

ConfVerChanged returns the number of confver has consumed by steps

func (*Operator) Desc 

func (o *Operator) Desc() string

Desc returns the operator's short description.

func (*Operator) ElapsedTime 

func (o *Operator) ElapsedTime() time.Duration

ElapsedTime returns duration since it was created.

func (*Operator) GetCreateTime 

func (o *Operator) GetCreateTime() time.Time

GetCreateTime gets the create time of operator.

func (*Operator) GetPriorityLevel 

func (o *Operator) GetPriorityLevel() core.PriorityLevel

GetPriorityLevel gets the priority level.

func (*Operator) GetReachTimeOf 

func (o *Operator) GetReachTimeOf(st OpStatus) time.Time

GetReachTimeOf returns the time when operator reaches the given status.

func (*Operator) GetStartTime 

func (o *Operator) GetStartTime() time.Time

GetStartTime gets the start time of operator.

func (*Operator) HasStarted 

func (o *Operator) HasStarted() bool

HasStarted returns whether operator has started.

func (*Operator) History 

func (o *Operator) History() []OpHistory

History transfers the operator's steps to operator histories.

func (*Operator) IsEnd 

func (o *Operator) IsEnd() bool

IsEnd checks if the operator is at and end status.

func (*Operator) Kind 

func (o *Operator) Kind() OpKind

Kind returns operator's kind.

func (*Operator) Len 

func (o *Operator) Len() int

Len returns the operator's steps count.

func (*Operator) MarshalJSON 

func (o *Operator) MarshalJSON() ([]byte, error)

MarshalJSON serializes custom types to JSON.

func (*Operator) RegionEpoch 

func (o *Operator) RegionEpoch() *metapb.RegionEpoch

RegionEpoch returns the region's epoch that is attached to the operator.

func (*Operator) RegionID 

func (o *Operator) RegionID() uint64

RegionID returns the region that operator is targeted.

func (*Operator) Replace 

func (o *Operator) Replace() bool

Replace marks the operator replaced.

func (*Operator) RunningTime 

func (o *Operator) RunningTime() time.Duration

RunningTime returns duration since it started.

func (*Operator) SetDesc 

func (o *Operator) SetDesc(desc string)

SetDesc sets the description for the operator.

func (*Operator) SetPriorityLevel 

func (o *Operator) SetPriorityLevel(level core.PriorityLevel)

SetPriorityLevel sets the priority level for operator.

func (*Operator) Start 

func (o *Operator) Start() bool

Start sets the operator to STARTED status, returns whether succeeded.

func (*Operator) Status 

func (o *Operator) Status() OpStatus

Status returns operator status.

func (*Operator) Step 

func (o *Operator) Step(i int) OpStep

Step returns the i-th step.

func (*Operator) String 

func (o *Operator) String() string

func (*Operator) TotalInfluence 

func (o *Operator) TotalInfluence(opInfluence OpInfluence, region *core.RegionInfo)

TotalInfluence calculates the store difference which whole operator steps make.

func (*Operator) UnfinishedInfluence 

func (o *Operator) UnfinishedInfluence(opInfluence OpInfluence, region *core.RegionInfo)

UnfinishedInfluence calculates the store difference which unfinished operator steps make.

type PromoteLearner 

type PromoteLearner struct {
	ToStore, PeerID uint64
}

PromoteLearner is an OpStep that promotes a region learner peer to normal voter.

func (PromoteLearner) ConfVerChanged 

func (pl PromoteLearner) ConfVerChanged(region *core.RegionInfo) bool

ConfVerChanged returns true if the conf version has been changed by this step

func (PromoteLearner) Influence 

func (pl PromoteLearner) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step makes.

func (PromoteLearner) IsFinish 

func (pl PromoteLearner) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (PromoteLearner) String 

func (pl PromoteLearner) String() string

type RemovePeer 

type RemovePeer struct {
	FromStore uint64
}

RemovePeer is an OpStep that removes a region peer.

func (RemovePeer) ConfVerChanged 

func (rp RemovePeer) ConfVerChanged(region *core.RegionInfo) bool

ConfVerChanged returns true if the conf version has been changed by this step

func (RemovePeer) Influence 

func (rp RemovePeer) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step makes.

func (RemovePeer) IsFinish 

func (rp RemovePeer) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (RemovePeer) String 

func (rp RemovePeer) String() string

type SplitRegion 

type SplitRegion struct {
	StartKey, EndKey []byte
	Policy           pdpb.CheckPolicy
	SplitKeys        [][]byte
}

SplitRegion is an OpStep that splits a region.

func (SplitRegion) ConfVerChanged 

func (sr SplitRegion) ConfVerChanged(region *core.RegionInfo) bool

ConfVerChanged returns true if the conf version has been changed by this step

func (SplitRegion) Influence 

func (sr SplitRegion) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step makes.

func (SplitRegion) IsFinish 

func (sr SplitRegion) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (SplitRegion) String 

func (sr SplitRegion) String() string

type StoreInfluence 

type StoreInfluence struct {
	RegionSize  int64
	RegionCount int64
	LeaderSize  int64
	LeaderCount int64
	StepCost    int64
}

StoreInfluence records influences that pending operators will make.

func (StoreInfluence) ResourceProperty 

func (s StoreInfluence) ResourceProperty(kind core.ScheduleKind) int64

ResourceProperty returns delta size of leader/region by influence.

type TransferLeader 

type TransferLeader struct {
	FromStore, ToStore uint64
}

TransferLeader is an OpStep that transfers a region's leader.

func (TransferLeader) ConfVerChanged 

func (tl TransferLeader) ConfVerChanged(region *core.RegionInfo) bool

ConfVerChanged returns true if the conf version has been changed by this step

func (TransferLeader) Influence 

func (tl TransferLeader) Influence(opInfluence OpInfluence, region *core.RegionInfo)

Influence calculates the store difference that current step makes.

func (TransferLeader) IsFinish 

func (tl TransferLeader) IsFinish(region *core.RegionInfo) bool

IsFinish checks if current step is finished.

func (TransferLeader) String 

func (tl TransferLeader) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.