util

package
v7.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 30, 2023 License: Apache-2.0 Imports: 57 Imported by: 0

Documentation

Index

Constants

View Source 
const (

	// NewSessionDefaultRetryCnt is the default retry times when create new session.
	NewSessionDefaultRetryCnt = 3
	// NewSessionRetryUnlimited is the unlimited retry times when create new session.
	NewSessionRetryUnlimited = math.MaxInt64
)
View Source 
const (
	// DefaultMaxRetries indicates the max retry count.
	DefaultMaxRetries = 30
	// RetryInterval indicates retry interval.
	RetryInterval uint64 = 500
)
View Source 
const (
	// Country is type name for country.
	Country = "C"
	// Organization is type name for organization.
	Organization = "O"
	// OrganizationalUnit is type name for organizational unit.
	OrganizationalUnit = "OU"
	// Locality is type name for locality.
	Locality = "L"
	// Email is type name for email.
	Email = "emailAddress"
	// CommonName is type name for common name.
	CommonName = "CN"
	// Province is type name for province or state.
	Province = "ST"
)
View Source 
const (
	// URI indicates uri info in SAN.
	URI = SANType("URI")
	// DNS indicates dns info in SAN.
	DNS = SANType("DNS")
	// IP indicates ip info in SAN.
	IP = SANType("IP")
)
View Source 
const (
	// MaxServerID is maximum serverID.
	MaxServerID = 1<<22 - 1
)

Variables

View Source 
var (
	// InformationSchemaName is the `INFORMATION_SCHEMA` database name.
	InformationSchemaName = model.NewCIStr("INFORMATION_SCHEMA")
	// PerformanceSchemaName is the `PERFORMANCE_SCHEMA` database name.
	PerformanceSchemaName = model.NewCIStr("PERFORMANCE_SCHEMA")
	// MetricSchemaName is the `METRICS_SCHEMA` database name.
	MetricSchemaName = model.NewCIStr("METRICS_SCHEMA")
	// ClusterTableInstanceColumnName is the `INSTANCE` column name of the cluster table.
	ClusterTableInstanceColumnName = "INSTANCE"
)
View Source 
var (
	Version   = "None"
	BuildTS   = "None"
	GitHash   = "None"
	GitBranch = "None"
)

Version information.

View Source 
var GetSequenceByName func(is interface{}, schema, sequence model.CIStr) (SequenceTable, error)

GetSequenceByName could be used in expression package without import cycle problem.

View Source 
var SupportCipher = make(map[string]struct{}, len(tlsCipherString))

SupportCipher maintains cipher supported by TiDB.

Functions

func ChanMap 

func ChanMap[T, R any](c <-chan T, f func(T) R) <-chan R

ChanMap creates a channel which applies the function over the input Channel. Hint of Resource Leakage: In golang, channel isn't an interface so we must create a goroutine for handling the inputs. Hence the input channel must be closed properly or this function may leak a goroutine.

func CheckSupportX509NameOneline 

func CheckSupportX509NameOneline(oneline string) (err error)

CheckSupportX509NameOneline parses and validate input str is X509_NAME_oneline format and precheck check-item is supported by TiDB https://www.openssl.org/docs/manmaster/man3/X509_NAME_oneline.html

func ClientWithTLS 

func ClientWithTLS(tlsCfg *tls.Config) *http.Client

ClientWithTLS creates a http client wit tls

func ColumnToProto 

func ColumnToProto(c *model.ColumnInfo, forIndex bool) *tipb.ColumnInfo

ColumnToProto converts model.ColumnInfo to tipb.ColumnInfo.

func ColumnsToProto 

func ColumnsToProto(columns []*model.ColumnInfo, pkIsHandle bool, forIndex bool) []*tipb.ColumnInfo

ColumnsToProto converts a slice of model.ColumnInfo to a slice of tipb.ColumnInfo.

func ComposeURL 

func ComposeURL(address, path string) string

ComposeURL adds HTTP schema if missing and concats address with path

func CreateCertificates 

func CreateCertificates(certpath string, keypath string, rsaKeySize int, pubKeyAlgo x509.PublicKeyAlgorithm,
	signAlgo x509.SignatureAlgorithm) error

CreateCertificates creates and writes a cert based on the params.

func DelKeyWithPrefix 

func DelKeyWithPrefix(rm kv.RetrieverMutator, prefix kv.Key) error

DelKeyWithPrefix deletes keys with prefix.

func FmtNonASCIIPrintableCharToHex 

func FmtNonASCIIPrintableCharToHex(str string) string

FmtNonASCIIPrintableCharToHex turns non-printable-ASCII characters into Hex

func GenLogFields 

func GenLogFields(costTime time.Duration, info *ProcessInfo, needTruncateSQL bool) []zap.Field

GenLogFields generate log fields.

func GenRLimit 

func GenRLimit() uint64

GenRLimit get RLIMIT_NOFILE limit

func GetAutoAnalyzeProcID 

func GetAutoAnalyzeProcID(serverIDGetter func() uint64) uint64

GetAutoAnalyzeProcID returns processID for auto analyze TODO support IDs for concurrent auto-analyze

func GetCPUPercentage 

func GetCPUPercentage() float64

GetCPUPercentage calculates CPU usage and returns percentage in float64(e.g. 2.5 means 2.5%). http://man7.org/linux/man-pages/man2/getrusage.2.html

func GetGOGC 

func GetGOGC() int

GetGOGC returns the current value of GOGC.

func GetJSON 

func GetJSON(client *http.Client, url string, v interface{}) error

GetJSON fetches a page and parses it as JSON. The parsed result will be stored into the `v`. The variable `v` must be a pointer to a type that can be unmarshalled from JSON.

Example: 

client := &http.Client{}
var resp struct { IP string }
if err := util.GetJSON(client, "http://api.ipify.org/?format=json", &resp); err != nil {
	return errors.Trace(err)
}
fmt.Println(resp.IP)

nolint:unused

func GetLocalIP 

func GetLocalIP() string

GetLocalIP will return a local IP(non-loopback, non 0.0.0.0), if there is one

func GetRawInfo 

func GetRawInfo(app string) string

GetRawInfo do what its name tells nolint:unused

func HasCancelled 

func HasCancelled(ctx context.Context) (cancel bool)

HasCancelled checks whether context has be cancelled.

func InternalHTTPClient 

func InternalHTTPClient() *http.Client

InternalHTTPClient is used by TiDB-Server to request other components.

func InternalHTTPSchema 

func InternalHTTPSchema() string

InternalHTTPSchema specifies use http or https to request other components.

func IsMemDB 

func IsMemDB(dbLowerName string) bool

IsMemDB checks whether dbLowerName is memory database.

func IsMemOrSysDB 

func IsMemOrSysDB(dbLowerName string) bool

IsMemOrSysDB uses to check whether dbLowerName is memory database or system database.

func IsSysDB 

func IsSysDB(dbLowerName string) bool

IsSysDB checks whether dbLowerName is system database.

func IsSystemView 

func IsSystemView(dbLowerName string) bool

IsSystemView is similar to IsMemOrSyDB, but does not include the mysql schema

func IsTLSExpiredError 

func IsTLSExpiredError(err error) bool

IsTLSExpiredError checks error is caused by TLS expired.

func LoadTLSCertificates 

func LoadTLSCertificates(ca, key, cert string, autoTLS bool, rsaKeySize int) (tlsConfig *tls.Config, autoReload bool, err error)

LoadTLSCertificates loads CA/KEY/CERT for special paths.

func MockPkixAttribute 

func MockPkixAttribute(name, value string) pkix.AttributeTypeAndValue

MockPkixAttribute generates mock AttributeTypeAndValue. only used for test.

func NewSession 

func NewSession(ctx context.Context, logPrefix string, etcdCli *clientv3.Client, retryCnt, ttl int) (*concurrency.Session, error)

NewSession creates a new etcd session.

func NewTCPConnWithIOCounter 

func NewTCPConnWithIOCounter(conn *net.TCPConn, c *atomic.Uint64) net.Conn

NewTCPConnWithIOCounter creates a new TCPConnWithIOCounter.

func NewTLSConfig 

func NewTLSConfig(opts ...TLSConfigOption) (*tls.Config, error)

NewTLSConfig creates a tls.Config from the given options. If no certificate is provided, it will return (nil, nil).

func OriginError 

func OriginError(err error) error

OriginError return original err

func ParseAndCheckSAN 

func ParseAndCheckSAN(san string) (map[SANType][]string, error)

ParseAndCheckSAN parses and check SAN str.

func ParseHostPortAddr 

func ParseHostPortAddr(s string) ([]string, error)

ParseHostPortAddr returns a scheme://host:port or host:port list

func PrintInfo 

func PrintInfo(app string)

PrintInfo prints the app's basic information in log nolint:unused

func PrintableASCII 

func PrintableASCII(b byte) bool

PrintableASCII detects if b is a printable ASCII character. Ref to:http://facweb.cs.depaul.edu/sjost/it212/documents/ascii-pr.htm

func QueryStrForLog 

func QueryStrForLog(query string) string

QueryStrForLog trim the query if the query length more than 4096

func Recover 

func Recover(metricsLabel, funcInfo string, recoverFn func(), quit bool)

Recover includes operations such as recovering, clearing，and printing information. It will dump current goroutine stack into log if catch any recover result. 

metricsLabel: The label of PanicCounter metrics.
funcInfo:     Some information for the panic function.
recoverFn:    Handler will be called after recover and before dump stack, passing `nil` means noop.
quit:         If this value is true, the current program exits after recovery.

func RowKeyPrefixFilter 

func RowKeyPrefixFilter(rowKeyPrefix kv.Key) kv.FnKeyCmp

RowKeyPrefixFilter returns a function which checks whether currentKey has decoded rowKeyPrefix as prefix.

func RunWithRetry 

func RunWithRetry(retryCnt int, backoff uint64, f func() (bool, error)) (err error)

RunWithRetry will run the f with backoff and retry. retryCnt: Max retry count backoff: When run f failed, it will sleep backoff * triedCount time.Millisecond. Function f should have two return value. The first one is an bool which indicate if the err if retryable. The second is if the f meet any error.

func ScanMetaWithPrefix 

func ScanMetaWithPrefix(retriever kv.Retriever, prefix kv.Key, filter func(kv.Key, []byte) bool) error

ScanMetaWithPrefix scans metadata with the prefix.

func SetGOGC 

func SetGOGC(val int) int

SetGOGC update GOGC and related metrics.

func SliceToMap 

func SliceToMap(slice []string) map[string]interface{}

SliceToMap converts slice to map nolint:unused

func Str2Int64Map 

func Str2Int64Map(str string) map[int64]struct{}

Str2Int64Map converts a string to a map[int64]struct{}.

func StringsToInterfaces 

func StringsToInterfaces(strs []string) []interface{}

StringsToInterfaces converts string slice to interface slice

func SyntaxError 

func SyntaxError(err error) error

SyntaxError converts parser error to TiDB's syntax error.

func SyntaxWarn 

func SyntaxWarn(err error) error

SyntaxWarn converts parser warn to TiDB's syntax warn.

func TLSCipher2String 

func TLSCipher2String(n uint16) string

TLSCipher2String convert tls num to string. Taken from https://testssl.sh/openssl-rfc.mapping.html .

func TSOToRoughTime 

func TSOToRoughTime(ts int64) time.Time

TSOToRoughTime translates tso to rough time that used to display

func ToTLSConfig 

func ToTLSConfig(caPath, certPath, keyPath string) (*tls.Config, error)

ToTLSConfig generates tls's config.

func ToTLSConfigWithVerify 

func ToTLSConfigWithVerify(caPath, certPath, keyPath string, verifyCN []string) (*tls.Config, error)

ToTLSConfigWithVerify constructs a `*tls.Config` from the CA, certification and key paths, and add verify for CN.

If the CA path is empty, returns nil.

func WithRecovery 

func WithRecovery(exec func(), recoverFn func(r interface{}))

WithRecovery wraps goroutine startup call with force recovery. it will dump current goroutine stack into log if catch any recover result. 

exec:      execute logic function.
recoverFn: handler will be called after recover and before dump stack, passing `nil` means noop.

func X509NameOnline 

func X509NameOnline(n pkix.Name) string

X509NameOnline prints pkix.Name into old X509_NAME_oneline format. https://www.openssl.org/docs/manmaster/man3/X509_NAME_oneline.html

Types

type GlobalConnID 

type GlobalConnID struct {
	ServerIDGetter func() uint64
	ServerID       uint64
	LocalConnID    uint64
	Is64bits       bool
}

GlobalConnID is the global connection ID, providing UNIQUE connection IDs across the whole TiDB cluster. 64 bits version: 

 63 62                 41 40                                   1   0
+--+---------------------+--------------------------------------+------+
|  |      serverId       |             local connId             |markup|
|=0|       (22b)         |                 (40b)                |  =1  |
+--+---------------------+--------------------------------------+------+
32 bits version(coming soon):
 31                          1   0
+-----------------------------+------+
|             ???             |markup|
|             ???             |  =0  |
+-----------------------------+------+

func NewGlobalConnID 

func NewGlobalConnID(serverID uint64, is64Bits bool) GlobalConnID

NewGlobalConnID creates GlobalConnID with serverID

func NewGlobalConnIDWithGetter 

func NewGlobalConnIDWithGetter(serverIDGetter func() uint64, is64Bits bool) GlobalConnID

NewGlobalConnIDWithGetter creates GlobalConnID with serverIDGetter

func ParseGlobalConnID 

func ParseGlobalConnID(id uint64) (g GlobalConnID, isTruncated bool, err error)

ParseGlobalConnID parses an uint64 to GlobalConnID. 

`isTruncated` indicates that older versions of the client truncated the 64-bit GlobalConnID to 32-bit.

func (*GlobalConnID) ID 

func (g *GlobalConnID) ID() uint64

ID returns the connection id

func (*GlobalConnID) NextID 

func (g *GlobalConnID) NextID() uint64

NextID returns next connection id

type OOMAlarmVariablesInfo 

type OOMAlarmVariablesInfo struct {
	SessionAnalyzeVersion         int
	SessionEnabledRateLimitAction bool
	SessionMemQuotaQuery          int64
}

OOMAlarmVariablesInfo is a struct for OOM alarm variables.

type ProcessInfo 

type ProcessInfo struct {
	ProtectedTSList
	Time                  time.Time
	ExpensiveLogTime      time.Time
	Plan                  interface{}
	StmtCtx               *stmtctx.StatementContext
	RefCountOfStmtCtx     *stmtctx.ReferenceCount
	MemTracker            *memory.Tracker
	DiskTracker           *disk.Tracker
	StatsInfo             func(interface{}) map[string]uint64
	RuntimeStatsColl      *execdetails.RuntimeStatsColl
	DB                    string
	Digest                string
	Host                  string
	User                  string
	Info                  string
	Port                  string
	PlanExplainRows       [][]string
	OOMAlarmVariablesInfo OOMAlarmVariablesInfo
	ID                    uint64
	CurTxnStartTS         uint64
	// MaxExecutionTime is the timeout for select statement, in milliseconds.
	// If the query takes too long, kill it.
	MaxExecutionTime uint64
	State            uint16
	Command          byte
	RedactSQL        bool
}

ProcessInfo is a struct used for show processlist statement.

func (*ProcessInfo) GetMinStartTS 

func (pi *ProcessInfo) GetMinStartTS(lowerBound uint64) (ts uint64)

GetMinStartTS returns the minimum start-ts (used to delay GC) that greater than `lowerBound` (0 if no such one).

func (*ProcessInfo) String 

func (pi *ProcessInfo) String() string

func (*ProcessInfo) ToRow 

func (pi *ProcessInfo) ToRow(tz *time.Location) []interface{}

ToRow returns []interface{} for the row data of "SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST".

func (*ProcessInfo) ToRowForShow 

func (pi *ProcessInfo) ToRowForShow(full bool) []interface{}

ToRowForShow returns []interface{} for the row data of "SHOW [FULL] PROCESSLIST".

type ProtectedTSList 

type ProtectedTSList interface {
	// HoldTS holds the timestamp to prevent its data from being GCed.
	HoldTS(ts uint64) (unhold func())
	// GetMinProtectedTS returns the minimum protected timestamp that greater than `lowerBound` (0 if no such one).
	GetMinProtectedTS(lowerBound uint64) (ts uint64)
}

ProtectedTSList holds a list of timestamps that should delay GC.

type SANType 

type SANType string

SANType is enum value for GlobalPrivValue.SANs keys.

type SequenceTable 

type SequenceTable interface {
	GetSequenceID() int64
	GetSequenceNextVal(ctx interface{}, dbName, seqName string) (int64, error)
	SetSequenceVal(ctx interface{}, newVal int64, dbName, seqName string) (int64, bool, error)
}

SequenceTable is implemented by tableCommon, and it is specialised in handling sequence operation. Otherwise calling table will cause import cycle problem.

type SessionManager 

type SessionManager interface {
	ShowProcessList() map[uint64]*ProcessInfo
	ShowTxnList() []*txninfo.TxnInfo
	GetProcessInfo(id uint64) (*ProcessInfo, bool)
	Kill(connectionID uint64, query bool)
	KillAllConnections()
	UpdateTLSConfig(cfg *tls.Config)
	ServerID() uint64
	// StoreInternalSession puts the internal session pointer to the map in the SessionManager.
	StoreInternalSession(se interface{})
	// DeleteInternalSession deletes the internal session pointer from the map in the SessionManager.
	DeleteInternalSession(se interface{})
	// GetInternalSessionStartTSList gets all startTS of every transactions running in the current internal sessions.
	GetInternalSessionStartTSList() []uint64
	// CheckOldRunningTxn checks if there is an old transaction running in the current sessions
	CheckOldRunningTxn(job2ver map[int64]int64, job2ids map[int64]string)
	// KillNonFlashbackClusterConn kill all non flashback cluster connections.
	KillNonFlashbackClusterConn()
	// GetMinStartTS returns the minimum start-ts (used to delay GC) that greater than `lowerBound` (0 if no such one).
	GetMinStartTS(lowerBound uint64) uint64
}

SessionManager is an interface for session manage. Show processlist and kill statement rely on this interface.

type TCPConnWithIOCounter 

type TCPConnWithIOCounter struct {
	*net.TCPConn
	// contains filtered or unexported fields
}

TCPConnWithIOCounter is a wrapper of net.TCPConn with counter that accumulates the bytes this connection reads/writes.

func (*TCPConnWithIOCounter) Read 

func (t *TCPConnWithIOCounter) Read(b []byte) (n int, err error)

func (*TCPConnWithIOCounter) Write 

func (t *TCPConnWithIOCounter) Write(b []byte) (n int, err error)

type TLS 

type TLS struct {
	// contains filtered or unexported fields
}

TLS saves some information about tls

func NewTLS 

func NewTLS(caPath, certPath, keyPath, host string, verifyCN []string) (*TLS, error)

NewTLS constructs a new HTTP client with TLS configured with the CA, certificate and key paths.

If the CA path is empty, returns an instance where TLS is disabled.

func (*TLS) WrapListener 

func (tc *TLS) WrapListener(l net.Listener) net.Listener

WrapListener places a TLS layer on top of the existing listener.

type TLSConfigOption 

type TLSConfigOption func(*tlsConfigBuilder)

TLSConfigOption is used to build a tls.Config in NewTLSConfig.

func WithCAContent 

func WithCAContent(caContent []byte) TLSConfigOption

WithCAContent sets the CA content to build a tls.Config, and the peer should use the certificate which can be verified by this CA. It has lower priority than WithCAPath. empty `caContent` is no-op.

func WithCAPath 

func WithCAPath(caPath string) TLSConfigOption

WithCAPath sets the CA path to build a tls.Config, and the peer should use the certificate which can be verified by this CA. It has higher priority than WithCAContent. empty `caPath` is no-op.

func WithCertAndKeyContent 

func WithCertAndKeyContent(certContent, keyContent []byte) TLSConfigOption

WithCertAndKeyContent sets the client certificate and primary key content to build a tls.Config. It has lower priority than WithCertAndKeyPath. empty `certContent`/`keyContent` is no-op.

func WithCertAndKeyPath 

func WithCertAndKeyPath(certPath, keyPath string) TLSConfigOption

WithCertAndKeyPath sets the client certificate and primary key path to build a tls.Config. It has higher priority than WithCertAndKeyContent. empty `certPath`/`keyPath` is no-op. WithCertAndKeyPath also support rotation, which means if the client certificate or primary key file is changed, the new content will be used.

func WithVerifyCommonName 

func WithVerifyCommonName(verifyCN []string) TLSConfigOption

WithVerifyCommonName sets the Common Name the peer must provide before starting a TLS connection. empty `verifyCN` is no-op.

type WaitGroupEnhancedWrapper 

type WaitGroupEnhancedWrapper struct {
	sync.WaitGroup
	// contains filtered or unexported fields
}

WaitGroupEnhancedWrapper wrapper wg, it provides the basic ability of WaitGroupWrapper with checking unexited process if the `exited` signal is true by print them on log.

func NewWaitGroupEnhancedWrapper 

func NewWaitGroupEnhancedWrapper(source string, exit chan struct{}, exitedCheck bool) *WaitGroupEnhancedWrapper

NewWaitGroupEnhancedWrapper returns WaitGroupEnhancedWrapper, the empty source indicates the unit test, then the `checkUnExitedProcess` won't be executed.

func (*WaitGroupEnhancedWrapper) Run 

func (w *WaitGroupEnhancedWrapper) Run(exec func(), label string)

Run runs a function in a goroutine, adds 1 to WaitGroup and calls done when function returns. Please DO NOT use panic in the cb function. Note that the registered label shouldn't be duplicated.

func (*WaitGroupEnhancedWrapper) RunWithRecover 

func (w *WaitGroupEnhancedWrapper) RunWithRecover(exec func(), recoverFn func(r interface{}), label string)

RunWithRecover wraps goroutine startup call with force recovery, add 1 to WaitGroup and call done when function return. exec is that execute logic function. recoverFn is that handler will be called after recover and before dump stack, passing `nil` means noop. Note that the registered label shouldn't be duplicated.

type WaitGroupWrapper 

type WaitGroupWrapper struct {
	sync.WaitGroup
}

WaitGroupWrapper is a wrapper for sync.WaitGroup

func (*WaitGroupWrapper) Run 

func (w *WaitGroupWrapper) Run(exec func())

Run runs a function in a goroutine, adds 1 to WaitGroup and calls done when function returns. Please DO NOT use panic in the cb function.

func (*WaitGroupWrapper) RunWithRecover 

func (w *WaitGroupWrapper) RunWithRecover(exec func(), recoverFn func(r interface{}))

RunWithRecover wraps goroutine startup call with force recovery, add 1 to WaitGroup and call done when function return. it will dump current goroutine stack into log if catch any recover result. exec is that execute logic function. recoverFn is that handler will be called after recover and before dump stack, passing `nil` means noop.

Source Files

View all Source files

Directories

Path Synopsis
testutil
spmc
consistency
Package mock is just for test only.
 Package mock is just for test only.
sys
linux
storage
collector
collector/mock
reporter
reporter/mock
state
stmtstats

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.