utils

package
v0.0.0-...-8586071 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 22, 2020 License: Apache-2.0 Imports: 42 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// LockFile represents file name,
	LockFile = "backup.lock"
	// MetaFile represents file name
	MetaFile = "backupmeta"
	// MetaJSONFile represents backup meta json file name
	MetaJSONFile = "backupmeta.json"
	// SavedMetaFile represents saved meta file name for recovering later
	SavedMetaFile = "backupmeta.bak"
)
View Source 
const (
	// B is number of bytes in one byte.
	B = uint64(1) << (iota * 10)
	// KB is number of bytes in one kibibyte.
	KB
	// MB is number of bytes in one mebibyte.
	MB
	// GB is number of bytes in one gibibyte.
	GB
	// TB is number of bytes in one tebibyte.
	TB
)
View Source 
const (

	// DefaultBRGCSafePointTTL means PD keep safePoint limit at least 5min
	DefaultBRGCSafePointTTL = 5 * 60
)

Variables

View Source 
var (
	BRReleaseVersion = "None"
	BRBuildTS        = "None"
	BRGitHash        = "None"
	BRGitBranch      = "None"

	VersionHash = regexp.MustCompile("-[0-9]+-g[0-9a-f]{7,}")
)

Version information.

Functions

func ArchiveSize 

func ArchiveSize(meta *backup.BackupMeta) uint64

ArchiveSize returns the total size of the backup archive.

func BRInfo 

func BRInfo() string

BRInfo returns version information about BR.

func CheckClusterVersion 

func CheckClusterVersion(ctx context.Context, client pd.Client) error

CheckClusterVersion check TiKV version.

func CheckGCSafePoint 

func CheckGCSafePoint(ctx context.Context, pdClient pd.Client, ts uint64) error

CheckGCSafePoint checks whether the ts is older than GC safepoint. Note: It ignores errors other than exceed GC safepoint.

func ClampInt 

func ClampInt(n, min, max int) int

ClampInt restrict a value to a certain interval.

func CompareEndKey 

func CompareEndKey(a, b []byte) int

CompareEndKey compared two keys that BOTH represent the EXCLUSIVE ending of some range. An empty end key is the very end, so an empty key is greater than any other keys. Please note that this function is not applicable if any one argument is not an EXCLUSIVE ending of a range.

func EncloseName 

func EncloseName(name string) string

EncloseName formats name in sql.

func GetPlacementRules 

func GetPlacementRules(pdAddr string, tlsConf *tls.Config) ([]placement.Rule, error)

GetPlacementRules return the current placement rules.

func IsTiFlash 

func IsTiFlash(store *metapb.Store) bool

IsTiFlash tests whether the store is based on tiflash engine.

func LoadBackupTables 

func LoadBackupTables(meta *backup.BackupMeta) (map[string]*Database, error)

LoadBackupTables loads schemas from BackupMeta.

func LogBRInfo 

func LogBRInfo()

LogBRInfo logs version information about BR.

func MakeSafePointID 

func MakeSafePointID() string

MakeSafePointID makes a unique safe point ID, for reduce name conflict.

func MaxInt 

func MaxInt(x int, xs ...int) int

MaxInt choice biggest integer from its arguments.

func MinInt 

func MinInt(x int, xs ...int) int

MinInt choice smallest integer from its arguments.

func MinInt64 

func MinInt64(x int64, xs ...int64) int64

MinInt64 choice smallest integer from its arguments.

func NeedAutoID 

func NeedAutoID(tblInfo *model.TableInfo) bool

NeedAutoID checks whether the table needs backing up with an autoid.

func ParseKey 

func ParseKey(format, key string) ([]byte, error)

ParseKey parse key by given format.

func ResetTS 

func ResetTS(pdAddr string, ts uint64, tlsConf *tls.Config) error

ResetTS resets the timestamp of PD to a bigger value.

func SearchPlacementRule 

func SearchPlacementRule(tableID int64, placementRules []placement.Rule, role placement.PeerRoleType) *placement.Rule

SearchPlacementRule returns the placement rule matched to the table or nil.

func StartDynamicPProfListener 

func StartDynamicPProfListener()

StartDynamicPProfListener starts the listener that will enable pprof when received `startPProfSignal`.

func StartPProfListener 

func StartPProfListener(statusAddr string)

StartPProfListener forks a new goroutine listening on specified port and provide pprof info.

func StartServiceSafePointKeeper 

func StartServiceSafePointKeeper(
	ctx context.Context,
	pdClient pd.Client,
	sp BRServiceSafePoint,
)

StartServiceSafePointKeeper will run UpdateServiceSafePoint periodicity hence keeping service safepoint won't lose.

func UpdateServiceSafePoint 

func UpdateServiceSafePoint(ctx context.Context, pdClient pd.Client, sp BRServiceSafePoint) error

UpdateServiceSafePoint register BackupTS to PD, to lock down BackupTS as safePoint with TTL seconds.

func WithRetry 

func WithRetry(
	ctx context.Context,
	retryableFunc RetryableFunc,
	backoffer Backoffer,
) error

WithRetry retries a given operation with a backoff policy.

Returns nil if `retryableFunc` succeeded at least once. Otherwise, returns a multierr containing all errors encountered.

func WrapKey 

func WrapKey(key []byte) fmt.Stringer

WrapKey wrap a key as a Stringer that can print proper upper hex format.

func WrapKeys 

func WrapKeys(keys [][]byte) zapcore.ArrayMarshaler

WrapKeys wrap keys as an ArrayMarshaler that can print proper upper hex format.

func ZapFile 

func ZapFile(file *backup.File) zapcore.Field

ZapFile make the zap fields for a file.

func ZapFiles 

func ZapFiles(fs []*backup.File) zapcore.Field

ZapFiles make the zap field for a set of file.

func ZapRegion 

func ZapRegion(region *metapb.Region) zapcore.Field

ZapRegion make the zap fields for a region.

func ZapRewriteRule 

func ZapRewriteRule(rewriteRule *import_sstpb.RewriteRule) zapcore.Field

ZapRewriteRule make the zap fields for a rewrite rule.

func ZapSSTMeta 

func ZapSSTMeta(sstMeta *import_sstpb.SSTMeta) zapcore.Field

ZapSSTMeta make the zap fields for a SST meta.

Types

type BRServiceSafePoint 

type BRServiceSafePoint struct {
	ID       string
	TTL      int64
	BackupTS uint64
}

BRServiceSafePoint is metadata of service safe point from a BR 'instance'.

func (BRServiceSafePoint) MarshalLogObject 

func (sp BRServiceSafePoint) MarshalLogObject(encoder zapcore.ObjectEncoder) error

MarshalLogObject implements zapcore.ObjectMarshaler.

type Backoffer 

type Backoffer interface {
	// NextBackoff returns a duration to wait before retrying again
	NextBackoff(err error) time.Duration
	// Attempt returns the remain attempt times
	Attempt() int
}

Backoffer implements a backoff policy for retrying operations.

type BytesBuffer 

type BytesBuffer struct {
	// contains filtered or unexported fields
}

BytesBuffer represents the reuse buffer.

func NewBytesBuffer 

func NewBytesBuffer() *BytesBuffer

NewBytesBuffer creates the BytesBuffer.

func (*BytesBuffer) AddBuf 

func (b *BytesBuffer) AddBuf()

AddBuf adds buffer to BytesBuffer.

func (*BytesBuffer) AddBytes 

func (b *BytesBuffer) AddBytes(bytes []byte) []byte

AddBytes add the bytes into this BytesBuffer.

func (*BytesBuffer) Destroy 

func (b *BytesBuffer) Destroy()

Destroy free all buffer.

func (*BytesBuffer) Reset 

func (b *BytesBuffer) Reset()

Reset reset the buffer.

func (*BytesBuffer) TotalSize 

func (b *BytesBuffer) TotalSize() int64

TotalSize represents the total memory size of this BytesBuffer.

type Database 

type Database struct {
	Info   *model.DBInfo
	Tables []*Table
}

Database wraps the schema and tables of a database.

func (*Database) GetTable 

func (db *Database) GetTable(name string) *Table

GetTable returns a table of the database by name.

type ProgressPrinter 

type ProgressPrinter struct {
	// contains filtered or unexported fields
}

ProgressPrinter prints a progress bar.

func NewProgressPrinter 

func NewProgressPrinter(
	name string,
	total int64,
	redirectLog bool,
) *ProgressPrinter

NewProgressPrinter returns a new progress printer.

func StartProgress 

func StartProgress(
	ctx context.Context,
	name string,
	total int64,
	redirectLog bool,
	log logFunc,
) *ProgressPrinter

StartProgress starts progress bar.

func (*ProgressPrinter) Close 

func (pp *ProgressPrinter) Close()

Close closes the current progress bar.

func (*ProgressPrinter) Inc 

func (pp *ProgressPrinter) Inc()

Inc increases the current progress bar.

type RetryableFunc 

type RetryableFunc func() error

RetryableFunc presents a retryable operation.

type Table 

type Table struct {
	DB              *model.DBInfo
	Info            *model.TableInfo
	Crc64Xor        uint64
	TotalKvs        uint64
	TotalBytes      uint64
	Files           []*backup.File
	TiFlashReplicas int
}

Table wraps the schema and files of a table.

func (*Table) NoChecksum 

func (tbl *Table) NoChecksum() bool

NoChecksum checks whether the table has a calculated checksum.

type UndoFunc 

type UndoFunc func(context.Context) error

UndoFunc is a 'undo' operation of some undoable command. (e.g. RemoveSchedulers). 

var Nop UndoFunc = func(context.Context) error { return nil }

Nop is the 'zero value' of undo func.

type Worker 

type Worker struct {
	ID uint64
}

Worker identified by ID.

type WorkerPool 

type WorkerPool struct {
	// contains filtered or unexported fields
}

WorkerPool contains a pool of workers.

func NewWorkerPool 

func NewWorkerPool(limit uint, name string) *WorkerPool

NewWorkerPool returns a WorkPool.

func (*WorkerPool) Apply 

func (pool *WorkerPool) Apply(fn taskFunc)

Apply executes a task.

func (*WorkerPool) ApplyOnErrorGroup 

func (pool *WorkerPool) ApplyOnErrorGroup(eg *errgroup.Group, fn func() error)

ApplyOnErrorGroup executes a task in an errorgroup.

func (*WorkerPool) ApplyWithID 

func (pool *WorkerPool) ApplyWithID(fn identifiedTaskFunc)

ApplyWithID execute a task and provides it with the worker ID.

func (*WorkerPool) ApplyWithIDInErrorGroup 

func (pool *WorkerPool) ApplyWithIDInErrorGroup(eg *errgroup.Group, fn func(id uint64) error)

ApplyWithIDInErrorGroup executes a task in an errorgroup and provides it with the worker ID.

func (*WorkerPool) HasWorker 

func (pool *WorkerPool) HasWorker() bool

HasWorker checks if the pool has unallocated workers.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.