utils

package
v0.0.0-...-bcc2220 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 26, 2024 License: Apache-2.0 Imports: 38 Imported by: 6

Documentation

Index

Constants

View Source 
const (
	StartTimeFormat  = "2006-01-02 15:04:05"
	StartTimeFormat2 = "2006-01-02T15:04:05"
)
View Source 
const (

	// PosRelaySubDirSuffixSeparator is used to differ binlog position from multiple
	// (switched) masters, we added a suffix which comes from relay log subdirectory
	// into binlogPos.Name. And we also need support position with RelaySubDirSuffix
	// should always > position without RelaySubDirSuffix, so we can continue from
	// latter to former automatically. convertedPos.BinlogName =
	//   originalPos.BinlogBaseName + PosRelaySubDirSuffixSeparator + RelaySubDirSuffix + binlogFilenameSep + originalPos.BinlogSeq
	// eg. mysql-bin.000003 under folder c6ae5afe-c7a3-11e8-a19d-0242ac130006.000002 => mysql-bin|000002.000003
	// when new relay log subdirectory is created, RelaySubDirSuffix should increase.
	PosRelaySubDirSuffixSeparator = "|"
)

Variables

View Source 
var (
	UUIDIndexFilename = "server-uuid.index"
	MetaFilename      = "relay.meta"
)

not support to config yet.

View Source 
var (
	// OsExit is function placeholder for os.Exit.
	OsExit func(int)
)
View Source 
var ZeroSessionCtx sessionctx.Context

ZeroSessionCtx is used when the session variables is not important.

Functions

func AddSuffixForUUID 

func AddSuffixForUUID(uuid string, id int) string

AddSuffixForUUID adds a suffix for UUID, returns the name for relay log subdirectory.

func AdjustBinaryProtocolForDatum 

func AdjustBinaryProtocolForDatum(ctx sessionctx.Context, data []interface{}, cols []*model.ColumnInfo) ([]types.Datum, error)

AdjustBinaryProtocolForDatum converts the data in binlog to TiDB datum.

func CollectDirFiles 

func CollectDirFiles(path string) (map[string]struct{}, error)

CollectDirFiles gets files in path.

func CompareShardingDDLs 

func CompareShardingDDLs(s, t []string) bool

CompareShardingDDLs compares s and t ddls only concern in content, ignore order of ddl.

func ConstructFilename 

func ConstructFilename(baseName, seq string) string

ConstructFilename constructs a binlog filename from the basename and seq.

func ConstructFilenameWithUUIDSuffix 

func ConstructFilenameWithUUIDSuffix(originalName Filename, uuidSuffix string) string

ConstructFilenameWithUUIDSuffix constructs a binlog filename with UUID suffix.

func DecodeBinlogPosition 

func DecodeBinlogPosition(pos string) (*mysql.Position, error)

DecodeBinlogPosition parses a mysql.Position from string format.

func Decrypt 

func Decrypt(ciphertextB64 string) (string, error)

Decrypt tries to decrypt base64 encoded ciphertext to plaintext.

func DecryptOrPlaintext 

func DecryptOrPlaintext(ciphertextB64 string) string

DecryptOrPlaintext tries to decrypt base64 encoded ciphertext to plaintext or return plaintext. when a customized key is provided, we support both plaintext and ciphertext as password, if not provided, only plaintext is supported.

func Encrypt 

func Encrypt(plaintext string) (string, error)

Encrypt tries to encrypt plaintext to base64 encoded ciphertext.

func EncryptOrPlaintext 

func EncryptOrPlaintext(plaintext string) string

EncryptOrPlaintext tries to encrypt plaintext to base64 encoded ciphertext or return plaintext. dm-master might not set customized key, so we should handle the error and return plaintext directly.

func ExtractDBAndTableFromLockID 

func ExtractDBAndTableFromLockID(lockID string) (string, string)

ExtractDBAndTableFromLockID extract schema and table from lockID.

func ExtractRealName 

func ExtractRealName(name string) string

ExtractRealName removes relay log uuid suffix if it exists and returns real binlog name.

func ExtractTaskFromLockID 

func ExtractTaskFromLockID(lockID string) string

ExtractTaskFromLockID extract task from lockID.

func GenDDLLockID 

func GenDDLLockID(task, schema, table string) string

GenDDLLockID returns lock ID used in shard-DDL.

func GenFakeRotateEvent 

func GenFakeRotateEvent(nextLogName string, logPos uint64, serverID uint32) (*replication.BinlogEvent, error)

GenFakeRotateEvent generates a fake ROTATE_EVENT without checksum ref: https://github.com/mysql/mysql-server/blob/4f1d7cf5fcb11a3f84cff27e37100d7295e7d5ca/sql/rpl_binlog_sender.cc#L855

func GenHashKey 

func GenHashKey(key string) uint32

GenHashKey generates key with crc32 algorithm.

func GenSchemaID 

func GenSchemaID(table *filter.Table) string

GenSchemaID generates schema ID.

func GenTableID 

func GenTableID(table *filter.Table) string

GenTableID generates table ID.

func GenTableIDAndCheckSchemaOnly 

func GenTableIDAndCheckSchemaOnly(table *filter.Table) (id string, isSchemaOnly bool)

GenTableIDAndCheckSchemaOnly generates table ID and check if schema only.

func GetDBFromDumpFilename 

func GetDBFromDumpFilename(filename string) (db string, ok bool)

GetDBFromDumpFilename extracts db name from dump filename.

func GetFileSize 

func GetFileSize(file string) (int64, error)

GetFileSize return the size of the file. NOTE: do not support to get the size of the directory now.

func GetFilenameIndex 

func GetFilenameIndex(filename string) (int64, error)

GetFilenameIndex returns a int64 index value (seq number) of the filename.

func GetTableFromDumpFilename 

func GetTableFromDumpFilename(filename string) (db, table string, ok bool)

GetTableFromDumpFilename extracts db and table name from dump filename.

func GetUUIDBySuffix 

func GetUUIDBySuffix(uuids []string, suffix string) string

GetUUIDBySuffix gets relay log subdirectory name by matching suffix.

func GoLogWrapper 

func GoLogWrapper(logger log.Logger, fn func())

GoLogWrapper go routine wrapper, log error on panic.

func IgnoreErrorCheckpoint 

func IgnoreErrorCheckpoint(err error) bool

IgnoreErrorCheckpoint is used in checkpoint update.

func IsBuildInSkipDDL 

func IsBuildInSkipDDL(sql string) bool

IsBuildInSkipDDL return true when checked sql that will be skipped for syncer.

func IsContextCanceledError 

func IsContextCanceledError(err error) bool

IsContextCanceledError checks whether err is context.Canceled.

func IsDirExists 

func IsDirExists(name string) bool

IsDirExists checks if dir exists.

func IsFakeRotateEvent 

func IsFakeRotateEvent(header *replication.EventHeader) bool

IsFakeRotateEvent return true if is this event is a fake rotate event If log pos equals zero then the received event is a fake rotate event and contains only a name of the next binlog file See https://github.com/mysql/mysql-server/blob/8e797a5d6eb3a87f16498edcb7261a75897babae/sql/rpl_binlog_sender.h#L235 and https://github.com/mysql/mysql-server/blob/8cc757da3d87bf4a1f07dcfb2d3c96fed3806870/sql/rpl_binlog_sender.cc#L899

func IsFileExists 

func IsFileExists(name string) bool

IsFileExists checks if file exists.

func LogHTTPProxies 

func LogHTTPProxies(useLogger bool)

LogHTTPProxies logs HTTP proxy relative environment variables.

func NewSessionCtx 

func NewSessionCtx(vars map[string]string) sessionctx.Context

NewSessionCtx return a session context with specified session variables.

func NewStoppedTimer 

func NewStoppedTimer() *time.Timer

func NonRepeatStringsEqual 

func NonRepeatStringsEqual(a, b []string) bool

NonRepeatStringsEqual is used to compare two un-ordered, non-repeat-element string slice is equal.

func ParseFileSize 

func ParseFileSize(fileSizeStr string, defaultSize uint64) (uint64, error)

ParseFileSize parses the size in MiB from input.

func ParseRelaySubDir 

func ParseRelaySubDir(uuid string) (string, int, error)

ParseRelaySubDir parses relay log subdirectory name to (server UUID, RelaySubDirSuffix) pair.

func ParseStartTime 

func ParseStartTime(timeStr string) (time.Time, error)

ParseStartTime parses start-time of task-start and validation-start in local location.

func ParseStartTimeInLoc 

func ParseStartTimeInLoc(timeStr string, loc *time.Location) (time.Time, error)

ParseStartTimeInLoc parses start-time of task-start and validation-start.

func ParseTimeZone 

func ParseTimeZone(s string) (*time.Location, error)

ParseTimeZone parse the time zone location by name or offset

NOTE: we don't support the "SYSTEM" or "Local" time_zone.

func ParseUUIDIndex 

func ParseUUIDIndex(indexPath string) ([]string, error)

ParseUUIDIndex parses UUIDIndexFilename, return a list of relay log subdirectory names and error.

func SetToSlice 

func SetToSlice(set map[string]struct{}) []string

SetToSlice converts a map of struct{} value to a slice to pretty print.

func SplitFilenameWithUUIDSuffix 

func SplitFilenameWithUUIDSuffix(filename string) (baseName, uuidSuffix, seq string, err error)

SplitFilenameWithUUIDSuffix analyzes a binlog filename with UUID suffix.

func SuffixIntToStr 

func SuffixIntToStr(id int) string

SuffixIntToStr convert int-represented suffix to string-represented. TODO: assign RelaySubDirSuffix a type and implement Stringer.

func TrimCtrlChars 

func TrimCtrlChars(s string) string

TrimCtrlChars returns a slice of the string s with all leading and trailing control characters removed.

func TrimQuoteMark 

func TrimQuoteMark(s string) string

TrimQuoteMark tries to trim leading and tailing quote(") mark if exists only trim if leading and tailing quote matched as a pair.

func TruncateInterface 

func TruncateInterface(v interface{}, n int) string

TruncateInterface converts the interface to a string and returns a string with only the leading (at most) n runes of the input string. If the converted string is truncated, a `...` tail will be appended. It is not effective for large structure now.

func TruncateString 

func TruncateString(s string, n int) string

TruncateString returns a string with only the leading (at most) n runes of the input string. If the string is truncated, a `...` tail will be appended.

func TruncateStringQuiet 

func TruncateStringQuiet(s string, n int) string

TruncateStringQuiet returns a string with only the leading (at most) n runes of the input string.

func UnpackTableID 

func UnpackTableID(id string) *filter.Table

UnpackTableID unpacks table ID to <schema, table> pair.

func UnwrapScheme 

func UnwrapScheme(s string) string

UnwrapScheme removes http or https scheme from input.

func VerifyFilename 

func VerifyFilename(filename string) bool

VerifyFilename verifies whether is a valid MySQL/MariaDB binlog filename. valid format is `base + '.' + seq`.

func WaitSomething 

func WaitSomething(backoff int, waitTime time.Duration, fn func() bool) bool

WaitSomething waits for something done with `true`.

func WrapSchemes 

func WrapSchemes(s string, https bool) string

WrapSchemes adds http or https scheme to input if missing. input could be a comma-separated list.

func WrapSchemesForInitialCluster 

func WrapSchemesForInitialCluster(s string, https bool) string

WrapSchemesForInitialCluster acts like WrapSchemes, except input is "name=URL,...".

func WriteFileAtomic 

func WriteFileAtomic(filename string, data []byte, perm os.FileMode) error

WriteFileAtomic writes file to temp and atomically move when everything else succeeds.

Types

type Filename 

type Filename struct {
	BaseName string
	Seq      string
	SeqInt64 int64
}

Filename represents a binlog filename.

func ParseFilename 

func ParseFilename(filename string) (Filename, error)

ParseFilename parses a string representation binlog filename into a `Filename`.

func (Filename) GreaterThan 

func (f Filename) GreaterThan(other Filename) bool

GreaterThan checks whether this filename > other filename.

func (Filename) GreaterThanOrEqualTo 

func (f Filename) GreaterThanOrEqualTo(other Filename) bool

GreaterThanOrEqualTo checks whether this filename >= other filename.

func (Filename) LessThan 

func (f Filename) LessThan(other Filename) bool

LessThan checks whether this filename < other filename.

type StorageSize 

type StorageSize struct {
	Capacity  uint64
	Available uint64
}

StorageSize represents the storage's capacity and available size Learn from tidb-binlog source code.

func GetStorageSize 

func GetStorageSize(dir string) (size StorageSize, err error)

GetStorageSize gets storage's capacity and available size.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.