utils

package
v5.1.0-alpha+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 21, 2021 License: Apache-2.0 Imports: 37 Imported by: 138

Documentation

Index

Constants

View Source 
const (
	// LockFile represents file name
	LockFile = "backup.lock"
	// MetaFile represents file name
	MetaFile = "backupmeta"
	// MetaJSONFile represents backup meta json file name
	MetaJSONFile = "backupmeta.json"
	// SavedMetaFile represents saved meta file name for recovering later
	SavedMetaFile = "backupmeta.bak"
)
View Source 
const (

	// DefaultBRGCSafePointTTL means PD keep safePoint limit at least 5min.
	DefaultBRGCSafePointTTL = 5 * 60
)

Variables

This section is empty.

Functions

func ArchiveSize 

func ArchiveSize(meta *backuppb.BackupMeta) uint64

ArchiveSize returns the total size of the backup archive.

func CheckGCSafePoint 

func CheckGCSafePoint(ctx context.Context, pdClient pd.Client, ts uint64) error

CheckGCSafePoint checks whether the ts is older than GC safepoint. Note: It ignores errors other than exceed GC safepoint.

func ClampInt 

func ClampInt(n, min, max int) int

ClampInt restrict a value to a certain interval.

func CompareEndKey 

func CompareEndKey(a, b []byte) int

CompareEndKey compared two keys that BOTH represent the EXCLUSIVE ending of some range. An empty end key is the very end, so an empty key is greater than any other keys. Please note that this function is not applicable if any one argument is not an EXCLUSIVE ending of a range.

func EncloseDBAndTable 

func EncloseDBAndTable(database, table string) string

EncloseDBAndTable formats the database and table name in sql.

func EncloseName 

func EncloseName(name string) string

EncloseName formats name in sql.

func IsSysDB 

func IsSysDB(dbLowerName string) bool

IsSysDB tests whether the database is system DB. Currently, the only system DB is mysql.

func LoadBackupTables 

func LoadBackupTables(meta *backuppb.BackupMeta) (map[string]*Database, error)

LoadBackupTables loads schemas from BackupMeta.

func LogEnvVariables 

func LogEnvVariables()

LogEnvVariables logs related environment variables.

func MakeSafePointID 

func MakeSafePointID() string

MakeSafePointID makes a unique safe point ID, for reduce name conflict.

func MarshalBackupMeta 

func MarshalBackupMeta(meta *backuppb.BackupMeta) ([]byte, error)

MarshalBackupMeta converts the backupmeta strcture to JSON. Unlike json.Marshal, this function also format some []byte fields for human reading.

func MaxInt 

func MaxInt(x int, xs ...int) int

MaxInt choice biggest integer from its arguments.

func MessageIsNotFoundStorageError 

func MessageIsNotFoundStorageError(msg string) bool

MessageIsNotFoundStorageError checks whether the message returning from TiKV is "NotFound" storage I/O error

func MessageIsPermissionDeniedStorageError 

func MessageIsPermissionDeniedStorageError(msg string) bool

MessageIsPermissionDeniedStorageError checks whether the message returning from TiKV is "PermissionDenied" storage I/O error

func MessageIsRetryableStorageError 

func MessageIsRetryableStorageError(msg string) bool

MessageIsRetryableStorageError checks whether the message returning from TiKV is retryable ExternalStorageError.

func MinInt 

func MinInt(x int, xs ...int) int

MinInt choice smallest integer from its arguments.

func MinInt64 

func MinInt64(x int64, xs ...int64) int64

MinInt64 choice smallest integer from its arguments.

func NeedAutoID 

func NeedAutoID(tblInfo *model.TableInfo) bool

NeedAutoID checks whether the table needs backing up with an autoid.

func NextPowerOfTwo 

func NextPowerOfTwo(i int64) int64

NextPowerOfTwo returns the smallest power of two greater than or equal to `i` Caller should guarantee that i > 0 and the return value is not overflow.

func ParseKey 

func ParseKey(format, key string) ([]byte, error)

ParseKey parse key by given format.

func StartDynamicPProfListener 

func StartDynamicPProfListener(tls *tidbutils.TLS)

StartDynamicPProfListener starts the listener that will enable pprof when received `startPProfSignal`.

func StartPProfListener 

func StartPProfListener(statusAddr string, wrapper *tidbutils.TLS) error

StartPProfListener forks a new goroutine listening on specified port and provide pprof info.

func StartServiceSafePointKeeper 

func StartServiceSafePointKeeper(
	ctx context.Context,
	pdClient pd.Client,
	sp BRServiceSafePoint,
) error

StartServiceSafePointKeeper will run UpdateServiceSafePoint periodicity hence keeping service safepoint won't lose.

func TemporaryDBName 

func TemporaryDBName(db string) model.CIStr

TemporaryDBName makes a 'private' database name.

func UnmarshalBackupMeta 

func UnmarshalBackupMeta(data []byte) (*backuppb.BackupMeta, error)

UnmarshalBackupMeta converts the prettied JSON format of backupmeta (made by MarshalBackupMeta) back to the go structure.

func WithRetry 

func WithRetry(
	ctx context.Context,
	retryableFunc RetryableFunc,
	backoffer Backoffer,
) error

WithRetry retries a given operation with a backoff policy.

Returns nil if `retryableFunc` succeeded at least once. Otherwise, returns a multierr containing all errors encountered.

Types

type BRServiceSafePoint 

type BRServiceSafePoint struct {
	ID       string
	TTL      int64
	BackupTS uint64
}

BRServiceSafePoint is metadata of service safe point from a BR 'instance'.

func (BRServiceSafePoint) MarshalLogObject 

func (sp BRServiceSafePoint) MarshalLogObject(encoder zapcore.ObjectEncoder) error

MarshalLogObject implements zapcore.ObjectMarshaler.

type Backoffer 

type Backoffer interface {
	// NextBackoff returns a duration to wait before retrying again
	NextBackoff(err error) time.Duration
	// Attempt returns the remain attempt times
	Attempt() int
}

Backoffer implements a backoff policy for retrying operations.

type BytesBuffer 

type BytesBuffer struct {
	// contains filtered or unexported fields
}

BytesBuffer represents the reuse buffer.

func NewBytesBuffer 

func NewBytesBuffer() *BytesBuffer

NewBytesBuffer creates the BytesBuffer.

func (*BytesBuffer) AddBuf 

func (b *BytesBuffer) AddBuf()

AddBuf adds buffer to BytesBuffer.

func (*BytesBuffer) AddBytes 

func (b *BytesBuffer) AddBytes(bytes []byte) []byte

AddBytes add the bytes into this BytesBuffer.

func (*BytesBuffer) Destroy 

func (b *BytesBuffer) Destroy()

Destroy free all buffer.

func (*BytesBuffer) Reset 

func (b *BytesBuffer) Reset()

Reset reset the buffer.

func (*BytesBuffer) TotalSize 

func (b *BytesBuffer) TotalSize() int64

TotalSize represents the total memory size of this BytesBuffer.

type Database 

type Database struct {
	Info   *model.DBInfo
	Tables []*Table
}

Database wraps the schema and tables of a database.

func (*Database) GetTable 

func (db *Database) GetTable(name string) *Table

GetTable returns a table of the database by name.

type ProgressPrinter 

type ProgressPrinter struct {
	// contains filtered or unexported fields
}

ProgressPrinter prints a progress bar.

func NewProgressPrinter 

func NewProgressPrinter(
	name string,
	total int64,
	redirectLog bool,
) *ProgressPrinter

NewProgressPrinter returns a new progress printer.

func StartProgress 

func StartProgress(
	ctx context.Context,
	name string,
	total int64,
	redirectLog bool,
	log logFunc,
) *ProgressPrinter

StartProgress starts progress bar.

func (*ProgressPrinter) Close 

func (pp *ProgressPrinter) Close()

Close closes the current progress bar.

func (*ProgressPrinter) Inc 

func (pp *ProgressPrinter) Inc()

Inc increases the current progress bar.

type RetryableFunc 

type RetryableFunc func() error

RetryableFunc presents a retryable operation.

type Table 

type Table struct {
	DB              *model.DBInfo
	Info            *model.TableInfo
	Crc64Xor        uint64
	TotalKvs        uint64
	TotalBytes      uint64
	Files           []*backuppb.File
	TiFlashReplicas int
	Stats           *handle.JSONTable
}

Table wraps the schema and files of a table.

func (*Table) NoChecksum 

func (tbl *Table) NoChecksum() bool

NoChecksum checks whether the table has a calculated checksum.

type Worker 

type Worker struct {
	ID uint64
}

Worker identified by ID.

type WorkerPool 

type WorkerPool struct {
	// contains filtered or unexported fields
}

WorkerPool contains a pool of workers.

func NewWorkerPool 

func NewWorkerPool(limit uint, name string) *WorkerPool

NewWorkerPool returns a WorkPool.

func (*WorkerPool) Apply 

func (pool *WorkerPool) Apply(fn taskFunc)

Apply executes a task.

func (*WorkerPool) ApplyOnErrorGroup 

func (pool *WorkerPool) ApplyOnErrorGroup(eg *errgroup.Group, fn func() error)

ApplyOnErrorGroup executes a task in an errorgroup.

func (*WorkerPool) ApplyWithID 

func (pool *WorkerPool) ApplyWithID(fn identifiedTaskFunc)

ApplyWithID execute a task and provides it with the worker ID.

func (*WorkerPool) ApplyWithIDInErrorGroup 

func (pool *WorkerPool) ApplyWithIDInErrorGroup(eg *errgroup.Group, fn func(id uint64) error)

ApplyWithIDInErrorGroup executes a task in an errorgroup and provides it with the worker ID.

func (*WorkerPool) ApplyWorker 

func (pool *WorkerPool) ApplyWorker() *Worker

ApplyWorker apply a worker.

func (*WorkerPool) HasWorker 

func (pool *WorkerPool) HasWorker() bool

HasWorker checks if the pool has unallocated workers.

func (*WorkerPool) RecycleWorker 

func (pool *WorkerPool) RecycleWorker(worker *Worker)

RecycleWorker recycle a worker.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.