util

package
v2.0.0-...-439aceb Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 15, 2023 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// InternalTxnOthers is the type of requests that consume low resources.
	// This reduces the size of metrics.
	InternalTxnOthers = "others"
	// InternalTxnGC is the type of GC txn.
	InternalTxnGC = "gc"
	// InternalTxnMeta is the type of the miscellaneous meta usage.
	InternalTxnMeta = InternalTxnOthers
)
View Source 
const (
	// InternalRequest is the scope of internal queries
	InternalRequest = "internal_"
	// ExternalRequest is the scope of external queries
	ExternalRequest = "external_"
	// SourceUnknown keeps same with the default value(empty string)
	SourceUnknown = "unknown"
)
View Source 
const GCTimeFormat = "20060102-15:04:05.000 -0700"

GCTimeFormat is the format that gc_worker used to store times.

Variables

View Source 
var (
	// CommitDetailCtxKey presents CommitDetail info key in context.
	CommitDetailCtxKey = commitDetailCtxKeyType{}

	// LockKeysDetailCtxKey presents LockKeysDetail info key in context.
	LockKeysDetailCtxKey = lockKeysDetailCtxKeyType{}

	// ExecDetailsKey presents ExecDetail info key in context.
	ExecDetailsKey = execDetailsCtxKeyType{}
)
View Source 
var RequestSourceKey = RequestSourceKeyType{}

RequestSourceKey is used as the key of request source type in context.

View Source 
var RequestSourceTypeKey = RequestSourceTypeKeyType{}

RequestSourceTypeKey is used as the key of request source type in context.

View Source 
var SessionID = sessionIDCtxKey{}

SessionID is the context key type to mark a session.

Functions

func BytesToString 

func BytesToString(numBytes int64) string

BytesToString converts the memory consumption to a readable string.

func CompatibleParseGCTime 

func CompatibleParseGCTime(value string) (time.Time, error)

CompatibleParseGCTime parses a string with `GCTimeFormat` and returns a time.Time. If `value` can't be parsed as that format, truncate to last space and try again. This function is only useful when loading times that saved by gc_worker. We have changed the format that gc_worker saves time (removed the last field), but when loading times it should be compatible with the old format.

func ContextWithTraceExecDetails 

func ContextWithTraceExecDetails(ctx context.Context) context.Context

ContextWithTraceExecDetails returns a context with trace-exec-details enabled

func EnableFailpoints 

func EnableFailpoints()

EnableFailpoints enables use of failpoints. It should be called before using client to avoid data race.

func EncodeToString 

func EncodeToString(src []byte) []byte

EncodeToString overrides hex.EncodeToString implementation. Difference: returns []byte, not string

func EvalFailpoint 

func EvalFailpoint(name string) (interface{}, error)

EvalFailpoint injects code for testing. It is used to replace `failpoint.Inject` to make it possible to be used in a library.

func FormatBytes 

func FormatBytes(numBytes int64) string

FormatBytes uses to format bytes, this function will prune precision before format bytes.

func FormatDuration 

func FormatDuration(d time.Duration) string

FormatDuration uses to format duration, this function will prune precision before format duration. Pruning precision is for human readability. The prune rule is:

  1. if the duration was less than 1us, return the original string.
  2. readable value >=10, keep 1 decimal, otherwise, keep 2 decimal. such as: 9.412345ms -> 9.41ms 10.412345ms -> 10.4ms 5.999s -> 6s 100.45µs -> 100.5µs

func HexRegionKey 

func HexRegionKey(key []byte) []byte

HexRegionKey converts region key to hex format. Used for formating region in logs.

func HexRegionKeyStr 

func HexRegionKeyStr(key []byte) string

HexRegionKeyStr converts region key to hex format. Used for formating region in logs.

func RequestSourceFromCtx 

func RequestSourceFromCtx(ctx context.Context) string

RequestSourceFromCtx extract source from passed context.

func ResourceGroupNameFromCtx 

func ResourceGroupNameFromCtx(ctx context.Context) string

ResourceGroupNameFromCtx extract resource group name from passed context, empty string is returned is the key is not set.

func SetSessionID 

func SetSessionID(ctx context.Context, sessionID uint64) context.Context

SetSessionID sets session id into context

func String 

func String(b []byte) (s string)

String converts slice of bytes to string without copy.

func ToUpperASCIIInplace 

func ToUpperASCIIInplace(s []byte) []byte

ToUpperASCIIInplace bytes.ToUpper but zero-cost

func TraceExecDetailsEnabled 

func TraceExecDetailsEnabled(ctx context.Context) bool

TraceExecDetailsEnabled checks whether trace-exec-details enabled

func WithInternalSourceType 

func WithInternalSourceType(ctx context.Context, source string) context.Context

WithInternalSourceType create context with internal source.

func WithRecovery 

func WithRecovery(exec func(), recoverFn func(r interface{}))

WithRecovery wraps goroutine startup call with force recovery. it will dump current goroutine stack into log if catch any recover result. 

exec:      execute logic function.
recoverFn: handler will be called after recover and before dump stack, passing `nil` means noop.

func WithResouceGroupName 

func WithResouceGroupName(ctx context.Context, groupName string) context.Context

WithResouceGroupName return a copy of the given context with a associated reosurce group name.

Types

type CommitDetails 

type CommitDetails struct {
	GetCommitTsTime        time.Duration
	GetLatestTsTime        time.Duration
	PrewriteTime           time.Duration
	WaitPrewriteBinlogTime time.Duration
	CommitTime             time.Duration
	LocalLatchTime         time.Duration
	Mu                     struct {
		sync.Mutex
		// The total backoff time used in both the prewrite and commit phases.
		CommitBackoffTime    int64
		PrewriteBackoffTypes []string
		CommitBackoffTypes   []string
		// The prewrite requests are executed concurrently so the slowest request information would be recorded.
		SlowestPrewrite ReqDetailInfo
		// It's recorded only when the commit mode is 2pc.
		CommitPrimary ReqDetailInfo
	}
	WriteKeys         int
	WriteSize         int
	PrewriteRegionNum int32
	TxnRetry          int
	ResolveLock       ResolveLockDetail
	PrewriteReqNum    int
}

CommitDetails contains commit detail information.

func (*CommitDetails) Clone 

func (cd *CommitDetails) Clone() *CommitDetails

Clone returns a deep copy of itself.

func (*CommitDetails) Merge 

func (cd *CommitDetails) Merge(other *CommitDetails)

Merge merges commit details into itself.

func (*CommitDetails) MergeCommitReqDetails 

func (cd *CommitDetails) MergeCommitReqDetails(reqDuration time.Duration, regionID uint64, addr string, execDetails *kvrpcpb.ExecDetailsV2)

MergeCommitReqDetails merges commit related ExecDetailsV2 into the current CommitDetails.

func (*CommitDetails) MergePrewriteReqDetails 

func (cd *CommitDetails) MergePrewriteReqDetails(reqDuration time.Duration, regionID uint64, addr string, execDetails *kvrpcpb.ExecDetailsV2)

MergePrewriteReqDetails merges prewrite related ExecDetailsV2 into the current CommitDetails.

type ExecDetails 

type ExecDetails struct {
	BackoffCount       int64
	BackoffDuration    int64
	WaitKVRespDuration int64
	WaitPDRespDuration int64
}

ExecDetails contains execution detail info.

type InterceptedPDClient 

type InterceptedPDClient struct {
	pd.Client
}

InterceptedPDClient is a PD's wrapper client to record stmt detail.

func (InterceptedPDClient) GetPrevRegion 

func (m InterceptedPDClient) GetPrevRegion(ctx context.Context, key []byte, opts ...pd.GetRegionOption) (*pd.Region, error)

GetPrevRegion implements pd.Client#GetPrevRegion.

func (InterceptedPDClient) GetRegion 

func (m InterceptedPDClient) GetRegion(ctx context.Context, key []byte, opts ...pd.GetRegionOption) (*pd.Region, error)

GetRegion implements pd.Client#GetRegion.

func (InterceptedPDClient) GetRegionByID 

func (m InterceptedPDClient) GetRegionByID(ctx context.Context, regionID uint64, opts ...pd.GetRegionOption) (*pd.Region, error)

GetRegionByID implements pd.Client#GetRegionByID.

func (InterceptedPDClient) GetStore 

func (m InterceptedPDClient) GetStore(ctx context.Context, storeID uint64) (*metapb.Store, error)

GetStore implements pd.Client#GetStore.

func (InterceptedPDClient) GetTS 

func (m InterceptedPDClient) GetTS(ctx context.Context) (int64, int64, error)

GetTS implements pd.Client#GetTS.

func (InterceptedPDClient) GetTSAsync 

func (m InterceptedPDClient) GetTSAsync(ctx context.Context) pd.TSFuture

GetTSAsync implements pd.Client#GetTSAsync.

func (InterceptedPDClient) ScanRegions 

func (m InterceptedPDClient) ScanRegions(ctx context.Context, key, endKey []byte, limit int) ([]*pd.Region, error)

ScanRegions implements pd.Client#ScanRegions.

type LockKeysDetails 

type LockKeysDetails struct {
	TotalTime   time.Duration
	RegionNum   int32
	LockKeys    int32
	ResolveLock ResolveLockDetail
	BackoffTime int64
	Mu          struct {
		sync.Mutex
		BackoffTypes        []string
		SlowestReqTotalTime time.Duration
		SlowestRegion       uint64
		SlowestStoreAddr    string
		SlowestExecDetails  TiKVExecDetails
	}
	LockRPCTime  int64
	LockRPCCount int64
	RetryCount   int
}

LockKeysDetails contains pessimistic lock keys detail information.

func (*LockKeysDetails) Clone 

func (ld *LockKeysDetails) Clone() *LockKeysDetails

Clone returns a deep copy of itself.

func (*LockKeysDetails) Merge 

func (ld *LockKeysDetails) Merge(lockKey *LockKeysDetails)

Merge merges lock keys execution details into self.

func (*LockKeysDetails) MergeReqDetails 

func (ld *LockKeysDetails) MergeReqDetails(reqDuration time.Duration, regionID uint64, addr string, execDetails *kvrpcpb.ExecDetailsV2)

MergeReqDetails merges ExecDetailsV2 into the current LockKeysDetails.

type RateLimit 

type RateLimit struct {
	// contains filtered or unexported fields
}

RateLimit wraps a fix sized channel to control concurrency.

func NewRateLimit 

func NewRateLimit(n int) *RateLimit

NewRateLimit creates a limit controller with capacity n.

func (*RateLimit) GetCapacity 

func (r *RateLimit) GetCapacity() int

GetCapacity returns the token capacity.

func (*RateLimit) GetToken 

func (r *RateLimit) GetToken(done <-chan struct{}) (exit bool)

GetToken acquires a token.

func (*RateLimit) PutToken 

func (r *RateLimit) PutToken()

PutToken puts a token back.

type ReqDetailInfo 

type ReqDetailInfo struct {
	ReqTotalTime time.Duration
	Region       uint64
	StoreAddr    string
	ExecDetails  TiKVExecDetails
}

ReqDetailInfo contains diagnose information about `TiKVExecDetails`, region, store and backoff.

type RequestSource 

type RequestSource struct {
	RequestSourceInternal bool
	RequestSourceType     string
}

RequestSource contains the source label of the request, used for tracking resource consuming.

func (*RequestSource) GetRequestSource 

func (r *RequestSource) GetRequestSource() string

GetRequestSource gets the request_source field of the request.

func (*RequestSource) SetRequestSourceInternal 

func (r *RequestSource) SetRequestSourceInternal(internal bool)

SetRequestSourceInternal sets the scope of the request source.

func (*RequestSource) SetRequestSourceType 

func (r *RequestSource) SetRequestSourceType(tp string)

SetRequestSourceType sets the type of the request source.

type RequestSourceKeyType 

type RequestSourceKeyType struct{}

RequestSourceKeyType is a dummy type to avoid naming collision in context.

type RequestSourceTypeKeyType 

type RequestSourceTypeKeyType struct{}

RequestSourceTypeKeyType is a dummy type to avoid naming collision in context.

type ResolveLockDetail 

type ResolveLockDetail struct {
	// ResolveLockTime is the total duration of resolving lock.
	ResolveLockTime int64
}

ResolveLockDetail contains the resolve lock detail information.

func (*ResolveLockDetail) Merge 

func (rd *ResolveLockDetail) Merge(resolveLock *ResolveLockDetail)

Merge merges resolve lock detail details into self.

type ScanDetail 

type ScanDetail struct {
	// TotalKeys is the approximate number of MVCC keys meet during scanning. It includes
	// deleted versions, but does not include RocksDB tombstone keys.
	TotalKeys int64
	// ProcessedKeys is the number of user keys scanned from the storage.
	// It does not include deleted version or RocksDB tombstone keys.
	// For Coprocessor requests, it includes keys that has been filtered out by Selection.
	ProcessedKeys int64
	// Number of bytes of user key-value pairs scanned from the storage, i.e.
	// total size of data returned from MVCC layer.
	ProcessedKeysSize int64
	// RocksdbDeleteSkippedCount is the total number of deletes and single deletes skipped over during
	// iteration, i.e. how many RocksDB tombstones are skipped.
	RocksdbDeleteSkippedCount uint64
	// RocksdbKeySkippedCount it the total number of internal keys skipped over during iteration.
	RocksdbKeySkippedCount uint64
	// RocksdbBlockCacheHitCount is the total number of RocksDB block cache hits.
	RocksdbBlockCacheHitCount uint64
	// RocksdbBlockReadCount is the total number of block reads (with IO).
	RocksdbBlockReadCount uint64
	// RocksdbBlockReadByte is the total number of bytes from block reads.
	RocksdbBlockReadByte uint64
	// RocksdbBlockReadDuration is the total time used for block reads.
	RocksdbBlockReadDuration time.Duration
	// GetSnapshotDuration is the time spent getting an engine snapshot.
	GetSnapshotDuration time.Duration

	ResolveLock *ResolveLockDetail
}

ScanDetail contains coprocessor scan detail information.

func (*ScanDetail) Merge 

func (sd *ScanDetail) Merge(scanDetail *ScanDetail)

Merge merges scan detail execution details into self.

func (*ScanDetail) MergeFromScanDetailV2 

func (sd *ScanDetail) MergeFromScanDetailV2(scanDetail *kvrpcpb.ScanDetailV2)

MergeFromScanDetailV2 merges scan detail from pb into itself.

func (*ScanDetail) String 

func (sd *ScanDetail) String() string

String implements the fmt.Stringer interface.

type TSSet 

type TSSet struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

TSSet is a set of timestamps.

func (*TSSet) GetAll 

func (s *TSSet) GetAll() []uint64

GetAll returns all timestamps in the set.

func (*TSSet) Put 

func (s *TSSet) Put(tss ...uint64)

Put puts timestamps into the map.

type TiKVExecDetails 

type TiKVExecDetails struct {
	TimeDetail  *TimeDetail
	ScanDetail  *ScanDetail
	WriteDetail *WriteDetail
}

TiKVExecDetails is the detail execution information at TiKV side.

func NewTiKVExecDetails 

func NewTiKVExecDetails(pb *kvrpcpb.ExecDetailsV2) TiKVExecDetails

NewTiKVExecDetails creates a TiKVExecDetails from a kvproto ExecDetailsV2.

func (*TiKVExecDetails) String 

func (ed *TiKVExecDetails) String() string

type TimeDetail 

type TimeDetail struct {
	// Off-cpu and on-cpu wall time elapsed to actually process the request payload. It does not
	// include `wait_wall_time`.
	// This field is very close to the CPU time in most cases. Some wait time spend in RocksDB
	// cannot be excluded for now, like Mutex wait time, which is included in this field, so that
	// this field is called wall time instead of CPU time.
	ProcessTime time.Duration
	// Off-cpu wall time elapsed in TiKV side. Usually this includes queue waiting time and
	// other kind of waits in series.
	WaitTime time.Duration
	// KvReadWallTimeMs is the time used in KV Scan/Get.
	KvReadWallTimeMs time.Duration
	// TotalRPCWallTime is Total wall clock time spent on this RPC in TiKV.
	TotalRPCWallTime time.Duration
}

TimeDetail contains coprocessor time detail information.

func (*TimeDetail) MergeFromTimeDetail 

func (td *TimeDetail) MergeFromTimeDetail(timeDetail *kvrpcpb.TimeDetail)

MergeFromTimeDetail merges time detail from pb into itself.

func (*TimeDetail) String 

func (td *TimeDetail) String() string

String implements the fmt.Stringer interface.

type WriteDetail 

type WriteDetail struct {
	// StoreBatchWaitDuration is the wait duration in the store loop.
	StoreBatchWaitDuration time.Duration
	// ProposeSendWaitDuration is the duration before sending proposal to peers.
	ProposeSendWaitDuration time.Duration
	// PersistLogDuration is the total time spent on persisting the log.
	PersistLogDuration time.Duration
	// RaftDbWriteLeaderWaitDuration is the wait time until the Raft log write leader begins to write.
	RaftDbWriteLeaderWaitDuration time.Duration
	// RaftDbSyncLogDuration is the time spent on synchronizing the Raft log to the disk.
	RaftDbSyncLogDuration time.Duration
	// RaftDbWriteMemtableDuration is the time spent on writing the Raft log to the Raft memtable.
	RaftDbWriteMemtableDuration time.Duration
	// CommitLogDuration is the time waiting for peers to confirm the proposal (counting from the instant when the leader sends the proposal message).
	CommitLogDuration time.Duration
	// ApplyBatchWaitDuration is the wait duration in the apply loop.
	ApplyBatchWaitDuration time.Duration
	// ApplyLogDuration is the total time spend to applying the log.
	ApplyLogDuration time.Duration
	// ApplyMutexLockDuration is the wait time until the KV RocksDB lock is acquired.
	ApplyMutexLockDuration time.Duration
	// ApplyWriteLeaderWaitDuration is the wait time until becoming the KV RocksDB write leader.
	ApplyWriteLeaderWaitDuration time.Duration
	// ApplyWriteWalDuration is the time spent on writing the KV DB WAL to the disk.
	ApplyWriteWalDuration time.Duration
	// ApplyWriteMemtableNanos is the time spent on writing to the memtable of the KV RocksDB.
	ApplyWriteMemtableDuration time.Duration
}

WriteDetail contains the detailed time breakdown of a write operation.

func (*WriteDetail) Merge 

func (wd *WriteDetail) Merge(writeDetail *WriteDetail)

Merge merges another WriteDetail protobuf into self.

func (*WriteDetail) MergeFromWriteDetailPb 

func (wd *WriteDetail) MergeFromWriteDetailPb(pb *kvrpcpb.WriteDetail)

MergeFromWriteDetailPb merges WriteDetail protobuf into the current WriteDetail

func (*WriteDetail) String 

func (wd *WriteDetail) String() string

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.