utils

package
v0.0.0-...-a345a4b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 4, 2024 License: Apache-2.0, Apache-2.0 Imports: 34 Imported by: 0

Documentation

Index

Constants

View Source 
const (

	// DefaultBRGCSafePointTTL means PD keep safePoint limit at least 5min.
	DefaultBRGCSafePointTTL = time.Duration(5) * time.Minute
	DefaultBRSafeInterval   = time.Minute // safe interval is used to calc the backup-ts.
)

Variables

View Source 
var (
	APIV2KeyPrefix    = [...]byte{'r', 0, 0, 0}
	APIV2KeyPrefixEnd = [...]byte{'r', 0, 0, 1}
	APIV2KeyPrefixLen = len(APIV2KeyPrefix)
)

Functions

func CheckGCSafePoint 

func CheckGCSafePoint(ctx context.Context, pdClient pd.Client, ts uint64) error

CheckGCSafePoint checks whether the ts is older than GC safepoint. Note: It ignores errors other than exceed GC safepoint.

func ClampInt 

func ClampInt(n, min, max int) int

ClampInt restrict a value to a certain interval.

func CompareEndKey 

func CompareEndKey(a, b []byte) int

CompareEndKey compared two keys that BOTH represent the EXCLUSIVE ending of some range. An empty end key is the very end, so an empty key is greater than any other keys. Please note that this function is not applicable if any one argument is not an EXCLUSIVE ending of a range.

func DefineTLSFlags 

func DefineTLSFlags(flags *pflag.FlagSet)

DefineTLSFlags define flags for tls config

func EncloseDBAndTable 

func EncloseDBAndTable(database, table string) string

EncloseDBAndTable formats the database and table name in sql.

func EncloseName 

func EncloseName(name string) string

EncloseName formats name in sql.

func FormatAPIV2Key 

func FormatAPIV2Key(key []byte, isEnd bool) []byte

func IsRetryableError 

func IsRetryableError(err error) bool

IsRetryableError returns whether the error is transient (e.g. network connection dropped) or irrecoverable (e.g. user pressing Ctrl+C). This function returns `false` (irrecoverable) if `err == nil`.

If the error is a multierr, returns true only if all suberrors are retryable.

func LogEnvVariables 

func LogEnvVariables()

LogEnvVariables logs related environment variables.

func MakeSafePointID 

func MakeSafePointID() string

MakeSafePointID makes a unique safe point ID, for reduce name conflict.

func MarshalBackupMeta 

func MarshalBackupMeta(meta *backuppb.BackupMeta) ([]byte, error)

MarshalBackupMeta converts the backupmeta strcture to JSON. Unlike json.Marshal, this function also format some []byte fields for human reading.

func MaxInt 

func MaxInt(x int, xs ...int) int

MaxInt choice biggest integer from its arguments.

func MessageIsNotFoundStorageError 

func MessageIsNotFoundStorageError(msg string) bool

MessageIsNotFoundStorageError checks whether the message returning from TiKV is "NotFound" storage I/O error

func MessageIsPermissionDeniedStorageError 

func MessageIsPermissionDeniedStorageError(msg string) bool

MessageIsPermissionDeniedStorageError checks whether the message returning from TiKV is "PermissionDenied" storage I/O error

func MessageIsRetryableStorageError 

func MessageIsRetryableStorageError(msg string) bool

MessageIsRetryableStorageError checks whether the message returning from TiKV is retryable ExternalStorageError.

func MinInt 

func MinInt(x int, xs ...int) int

MinInt choice smallest integer from its arguments.

func MinInt64 

func MinInt64(x int64, xs ...int64) int64

MinInt64 choice smallest integer from its arguments.

func NextPowerOfTwo 

func NextPowerOfTwo(i int64) int64

NextPowerOfTwo returns the smallest power of two greater than or equal to `i` Caller should guarantee that i > 0 and the return value is not overflow.

func ParseKey 

func ParseKey(format, key string) ([]byte, error)

ParseKey parse key by given format.

func StartServiceSafePointKeeper 

func StartServiceSafePointKeeper(
	ctx context.Context,
	pdClient pd.Client,
	sp BRServiceSafePoint,
) error

StartServiceSafePointKeeper will run UpdateServiceSafePoint periodicity hence keeping service safepoint won't lose.

func UnmarshalBackupMeta 

func UnmarshalBackupMeta(data []byte) (*backuppb.BackupMeta, error)

UnmarshalBackupMeta converts the prettied JSON format of backupmeta (made by MarshalBackupMeta) back to the go structure.

func UpdateServiceSafePoint 

func UpdateServiceSafePoint(ctx context.Context, pdClient pd.Client, sp BRServiceSafePoint) error

updateServiceSafePoint register BackupTS to PD, to lock down BackupTS as safePoint with TTL seconds.

func WithRetry 

func WithRetry(
	ctx context.Context,
	retryableFunc RetryableFunc,
	backoffer Backoffer,
) error

WithRetry retries a given operation with a backoff policy.

Returns nil if `retryableFunc` succeeded at least once. Otherwise, returns a multierr containing all errors encountered.

Types

type BRServiceSafePoint 

type BRServiceSafePoint struct {
	ID       string
	TTL      int64
	BackupTS uint64
}

BRServiceSafePoint is metadata of service safe point from a BR 'instance'.

func (BRServiceSafePoint) MarshalLogObject 

func (sp BRServiceSafePoint) MarshalLogObject(encoder zapcore.ObjectEncoder) error

MarshalLogObject implements zapcore.ObjectMarshaler.

type Backoffer 

type Backoffer interface {
	// NextBackoff returns a duration to wait before retrying again
	NextBackoff(err error) time.Duration
	// Attempt returns the remain attempt times
	Attempt() int
}

Backoffer implements a backoff policy for retrying operations.

func NewBackoffer 

func NewBackoffer(attempt int, delayTime, maxDelayTime time.Duration) Backoffer

NewBackoffer creates a new controller regulating a truncated exponential backoff.

func NewChecksumBackoffer 

func NewChecksumBackoffer() Backoffer

func NewDownloadSSTBackoffer 

func NewDownloadSSTBackoffer() Backoffer

func NewImportSSTBackoffer 

func NewImportSSTBackoffer() Backoffer

func NewPDReqBackoffer 

func NewPDReqBackoffer() Backoffer

type KeyRange 

type KeyRange struct {
	Start []byte
	End   []byte
}

func ConvertBackupConfigKeyRange 

func ConvertBackupConfigKeyRange(startKey, endKey []byte, srcAPIVer, dstAPIVer kvrpcpb.APIVersion) *KeyRange

ConvertBackupConfigKeyRange do conversion between formated APIVersion key and backupmeta key for example, backup apiv1 -> apiv2, add `r` prefix and `s` for empty end key. apiv2 -> apiv1, remove first byte.

func EncodeKeyRange 

func EncodeKeyRange(start, end []byte) *KeyRange

func FormatAPIV2KeyRange 

func FormatAPIV2KeyRange(startKey, endKey []byte) *KeyRange

FormatAPIV2KeyRange convert user key to APIV2 format.

type ProgressPrinter 

type ProgressPrinter struct {
	// contains filtered or unexported fields
}

ProgressPrinter prints a progress bar.

func NewProgressPrinter 

func NewProgressPrinter(
	name string,
	total int64,
	redirectLog bool,
) *ProgressPrinter

NewProgressPrinter returns a new progress printer.

func StartProgress 

func StartProgress(
	ctx context.Context,
	name string,
	total int64,
	redirectLog bool,
	log logFunc,
) *ProgressPrinter

StartProgress starts progress bar.

func (*ProgressPrinter) Close 

func (pp *ProgressPrinter) Close()

Close closes the current progress bar.

func (*ProgressPrinter) Inc 

func (pp *ProgressPrinter) Inc()

Inc increases the current progress bar.

type RetryableFunc 

type RetryableFunc func() error

RetryableFunc presents a retryable operation.

type TLSConfig 

type TLSConfig struct {
	CA   string `json:"ca" toml:"ca"`
	Cert string `json:"cert" toml:"cert"`
	Key  string `json:"key" toml:"key"`
}

TLSConfig is the common configuration for TLS connection.

func (*TLSConfig) IsEnabled 

func (tls *TLSConfig) IsEnabled() bool

IsEnabled checks if TLS open or not.

func (*TLSConfig) ParseFromFlags 

func (tls *TLSConfig) ParseFromFlags(flags *pflag.FlagSet) (err error)

ParseFromFlags parses the TLS config from the flag set.

func (*TLSConfig) ToTLSConfig 

func (tls *TLSConfig) ToTLSConfig() (*tls.Config, error)

ToTLSConfig generate tls.Config.

type Worker 

type Worker struct {
	ID uint64
}

Worker identified by ID.

type WorkerPool 

type WorkerPool struct {
	// contains filtered or unexported fields
}

WorkerPool contains a pool of workers.

func NewWorkerPool 

func NewWorkerPool(limit uint, name string) *WorkerPool

NewWorkerPool returns a WorkPool.

func (*WorkerPool) Apply 

func (pool *WorkerPool) Apply(fn taskFunc)

Apply executes a task.

func (*WorkerPool) ApplyOnErrorGroup 

func (pool *WorkerPool) ApplyOnErrorGroup(eg *errgroup.Group, fn func() error)

ApplyOnErrorGroup executes a task in an errorgroup.

func (*WorkerPool) ApplyWithID 

func (pool *WorkerPool) ApplyWithID(fn identifiedTaskFunc)

ApplyWithID execute a task and provides it with the worker ID.

func (*WorkerPool) ApplyWithIDInErrorGroup 

func (pool *WorkerPool) ApplyWithIDInErrorGroup(eg *errgroup.Group, fn func(id uint64) error)

ApplyWithIDInErrorGroup executes a task in an errorgroup and provides it with the worker ID.

func (*WorkerPool) ApplyWorker 

func (pool *WorkerPool) ApplyWorker() *Worker

ApplyWorker apply a worker.

func (*WorkerPool) HasWorker 

func (pool *WorkerPool) HasWorker() bool

HasWorker checks if the pool has unallocated workers.

func (*WorkerPool) IdleCount 

func (pool *WorkerPool) IdleCount() int

IdleCount counts how many idle workers in the pool.

func (*WorkerPool) Limit 

func (pool *WorkerPool) Limit() int

Limit is the limit of the pool

func (*WorkerPool) RecycleWorker 

func (pool *WorkerPool) RecycleWorker(worker *Worker)

RecycleWorker recycle a worker.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.