data

Documentation

Overview

Copyright 2022 PingCAP, Inc. Licensed under Apache-2.0.

Index

• func CheckConsistencyAndValidPeer(regionInfos []*RecoverRegionInfo) (map[uint64]struct{}, error)
• func PrefixEndKey(key []byte) []byte
• func PrefixStartKey(key []byte) []byte
• func RecoverData(ctx context.Context, resolveTS uint64, allStores []*metapb.Store, ...) (int, error)
• type RecoverRegion
  • func LeaderCandidates(peers []*RecoverRegion) ([]*RecoverRegion, error)
  • func SelectRegionLeader(storeBalanceScore map[uint64]int, peers []*RecoverRegion) *RecoverRegion
• type RecoverRegionInfo
  • func SortRecoverRegions(regions map[uint64][]*RecoverRegion) []*RecoverRegionInfo
• type Recovery
  • func NewRecovery(allStores []*metapb.Store, mgr *conn.Mgr, progress glue.Progress, ...) Recovery
  • func (recovery *Recovery) FlashbackToVersion(ctx context.Context, resolveTS uint64, commitTS uint64) (err error)
  • func (recovery *Recovery) GetTotalRegions() int
  • func (recovery *Recovery) MakeRecoveryPlan() error
  • func (recovery *Recovery) PrepareFlashbackToVersion(ctx context.Context, resolveTS uint64, startTS uint64) (err error)
  • func (recovery *Recovery) ReadRegionMeta(ctx context.Context) error
  • func (recovery *Recovery) RecoverRegionOfStore(ctx context.Context, storeID uint64, plan []*recovpb.RecoverRegionRequest) error
  • func (recovery *Recovery) RecoverRegions(ctx context.Context) (err error)
  • func (recovery *Recovery) SpawnTiKVShutDownWatchers(ctx context.Context)
• type RecoveryStage
  • func FailedAt(err error) RecoveryStage
  • func (s RecoveryStage) String() string
• type StoreMeta
  • func NewStoreMeta(storeId uint64) StoreMeta

Functions

func CheckConsistencyAndValidPeer

func CheckConsistencyAndValidPeer(regionInfos []*RecoverRegionInfo) (map[uint64]struct{}, error)

func PrefixEndKey

func PrefixEndKey(key []byte) []byte

func PrefixStartKey

func PrefixStartKey(key []byte) []byte

func RecoverData

func RecoverData(ctx context.Context, resolveTS uint64, allStores []*metapb.Store, mgr *conn.Mgr, progress glue.Progress, restoreTS uint64, concurrency uint32) (int, error)

RecoverData recover the tikv cluster 1. read all meta data from tikvs 2. make recovery plan and then recovery max allocate ID firstly 3. send the recover plan and the wait tikv to apply, in waitapply, all assigned region leader will check apply log to the last log 4. ensure all region apply to last log 5. prepare the flashback 6. flashback to resolveTS

Types

type RecoverRegion

type RecoverRegion struct {
	*recovpb.RegionMeta
	StoreId uint64
}

func LeaderCandidates

func LeaderCandidates(peers []*RecoverRegion) ([]*RecoverRegion, error)

in cloud, since iops and bandwidth limitation, write operator in raft is slow, so raft state (logterm, lastlog, commitlog...) are the same among the peers LeaderCandidates select all peers can be select as a leader during the restore

func SelectRegionLeader

func SelectRegionLeader(storeBalanceScore map[uint64]int, peers []*RecoverRegion) *RecoverRegion

for region A, has candidate leader x, y, z peer x on store 1 with storeBalanceScore 3 peer y on store 3 with storeBalanceScore 2 peer z on store 4 with storeBalanceScore 1 result: peer z will be select as leader on store 4

type RecoverRegionInfo

type RecoverRegionInfo struct {
	RegionId      uint64
	RegionVersion uint64
	StartKey      []byte
	EndKey        []byte
	TombStone     bool
}

func SortRecoverRegions

func SortRecoverRegions(regions map[uint64][]*RecoverRegion) []*RecoverRegionInfo

type Recovery

type Recovery struct {
	StoreMetas   []StoreMeta
	RecoveryPlan map[uint64][]*recovpb.RecoverRegionRequest
	MaxAllocID   uint64
	// contains filtered or unexported fields
}

for test

func NewRecovery

func NewRecovery(allStores []*metapb.Store, mgr *conn.Mgr, progress glue.Progress, concurrency uint32) Recovery

func (*Recovery) FlashbackToVersion

func (recovery *Recovery) FlashbackToVersion(ctx context.Context, resolveTS uint64, commitTS uint64) (err error)

flashback the region data to version resolveTS

func (*Recovery) GetTotalRegions

func (recovery *Recovery) GetTotalRegions() int

func (*Recovery) MakeRecoveryPlan

func (recovery *Recovery) MakeRecoveryPlan() error

generate the related the recovery plan to tikvs: 1. check overlap the region, make a recovery decision 2. build a leader list for all region during the tikv startup 3. get max allocate id

func (*Recovery) PrepareFlashbackToVersion

func (recovery *Recovery) PrepareFlashbackToVersion(ctx context.Context, resolveTS uint64, startTS uint64) (err error)

prepare the region for flashback the data, the purpose is to stop region service, put region in flashback state

func (*Recovery) ReadRegionMeta

func (recovery *Recovery) ReadRegionMeta(ctx context.Context) error

ReadRegionMeta read all region meta from tikvs

func (*Recovery) RecoverRegionOfStore

func (recovery *Recovery) RecoverRegionOfStore(ctx context.Context, storeID uint64, plan []*recovpb.RecoverRegionRequest) error

func (*Recovery) RecoverRegions

func (recovery *Recovery) RecoverRegions(ctx context.Context) (err error)

RecoverRegions send the recovery plan to recovery region (force leader etc) only tikvs have regions whose have to recover be sent

func (*Recovery) SpawnTiKVShutDownWatchers

func (recovery *Recovery) SpawnTiKVShutDownWatchers(ctx context.Context)

type RecoveryStage

type RecoveryStage int

const (
	StageUnknown RecoveryStage = iota
	StageCollectingMeta
	StageMakingRecoveryPlan
	StageResetPDAllocateID
	StageRecovering
	StageFlashback
)

func FailedAt

func FailedAt(err error) RecoveryStage

func (RecoveryStage) String

func (s RecoveryStage) String() string

type StoreMeta

type StoreMeta struct {
	StoreId     uint64
	RegionMetas []*recovpb.RegionMeta
}

func NewStoreMeta

func NewStoreMeta(storeId uint64) StoreMeta