copr

package
v1.1.0-beta.0...-8e1a47d Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 26, 2024 License: Apache-2.0 Imports: 69 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	CopNextMaxBackoff = 20000
	CopSmallTaskRow   = 32 // 32 is the initial batch size of TiKV

)

Maximum total sleep time(in ms) for kv/cop commands.

View Source 
const (
	// DetectPeriod detect period
	DetectPeriod = 3 * time.Second
	// DetectTimeoutLimit detect timeout
	DetectTimeoutLimit = 2 * time.Second
	// MaxRecoveryTimeLimit wait TiFlash recovery,more than MPPStoreFailTTL
	MaxRecoveryTimeLimit = 15 * time.Minute
	// MaxObsoletTimeLimit no request for a long time,that might be obsoleted
	MaxObsoletTimeLimit = time.Hour
)
View Source 
const MockResponseSizeForTest = 100 * 1024 * 1024

MockResponseSizeForTest mock the response size

View Source 
const TiFlashReadTimeoutUltraLong = 3600 * time.Second

TiFlashReadTimeoutUltraLong represents the max time that tiflash request may take, since it may scan many regions for tiflash.

View Source 
const UnspecifiedLimit = -1

UnspecifiedLimit means no limit.

Variables

This section is empty.

Functions

func BuildKeyRanges 

func BuildKeyRanges(keys ...string) []kv.KeyRange

BuildKeyRanges is used for test, quickly build key ranges from paired keys.

Types

type Backoffer 

type Backoffer = backoff.Backoffer

Backoffer wraps tikv.Backoffer and converts the error which returns by the functions of tikv.Backoffer to tidb error.

type CopClient 

type CopClient struct {
	kv.RequestTypeSupportedChecker
	// contains filtered or unexported fields
}

CopClient is coprocessor client.

func (*CopClient) BuildCopIterator 

func (c *CopClient) BuildCopIterator(ctx context.Context, req *kv.Request, vars *tikv.Variables, option *kv.ClientSendOption) (*copIterator, kv.Response)

BuildCopIterator builds the iterator without calling `open`.

func (*CopClient) Send 

func (c *CopClient) Send(ctx context.Context, req *kv.Request, variables any, option *kv.ClientSendOption) kv.Response

Send builds the request and gets the coprocessor iterator response.

type CopInfo 

type CopInfo interface {
	// GetConcurrency returns the concurrency and small task concurrency.
	GetConcurrency() (int, int)
	// GetStoreBatchInfo returns the batched and fallback num.
	GetStoreBatchInfo() (uint64, uint64)
	// GetBuildTaskElapsed returns the duration of building task.
	GetBuildTaskElapsed() time.Duration
}

CopInfo is used to expose functions of copIterator.

type CopRuntimeStats 

type CopRuntimeStats struct {
	execdetails.ExecDetails
	ReqStats *tikv.RegionRequestRuntimeStats

	CoprCacheHit bool
}

CopRuntimeStats contains execution detail information.

type HasUnconsumedCopRuntimeStats 

type HasUnconsumedCopRuntimeStats interface {
	// CollectUnconsumedCopRuntimeStats returns unconsumed CopRuntimeStats.
	CollectUnconsumedCopRuntimeStats() []*CopRuntimeStats
}

HasUnconsumedCopRuntimeStats indicate whether has unconsumed CopRuntimeStats.

type KeyRanges 

type KeyRanges struct {
	// contains filtered or unexported fields
}

KeyRanges is like []kv.KeyRange, but may has extra elements at head/tail. It's for avoiding alloc big slice during build copTask.

func NewKeyRanges 

func NewKeyRanges(ranges []kv.KeyRange) *KeyRanges

NewKeyRanges constructs a KeyRanges instance.

func (*KeyRanges) At 

func (r *KeyRanges) At(i int) kv.KeyRange

At returns the range at the ith position.

func (*KeyRanges) Do 

func (r *KeyRanges) Do(f func(ran *kv.KeyRange))

Do applies a functions to all ranges.

func (*KeyRanges) Len 

func (r *KeyRanges) Len() int

Len returns the count of ranges.

func (*KeyRanges) RefAt 

func (r *KeyRanges) RefAt(i int) *kv.KeyRange

RefAt returns the reference at the ith position without copy.

func (*KeyRanges) Slice 

func (r *KeyRanges) Slice(from, to int) *KeyRanges

Slice returns the sub ranges [from, to).

func (*KeyRanges) Split 

func (r *KeyRanges) Split(key []byte) (*KeyRanges, *KeyRanges)

Split ranges into (left, right) by key.

func (*KeyRanges) String 

func (r *KeyRanges) String() string

func (*KeyRanges) ToPBRanges 

func (r *KeyRanges) ToPBRanges() []*coprocessor.KeyRange

ToPBRanges converts ranges to wire type.

func (*KeyRanges) ToRanges 

func (r *KeyRanges) ToRanges() []kv.KeyRange

ToRanges converts ranges to []kv.KeyRange.

type LocationKeyRanges 

type LocationKeyRanges struct {
	// Location is the real location in PD.
	Location *tikv.KeyLocation
	// Ranges is the logic ranges the current Location contains.
	Ranges *KeyRanges
}

LocationKeyRanges wraps a real Location in PD and its logical ranges info.

type MPPClient 

type MPPClient struct {
	// contains filtered or unexported fields
}

MPPClient servers MPP requests.

func (*MPPClient) CancelMPPTasks 

func (c *MPPClient) CancelMPPTasks(param kv.CancelMPPTasksParam)

CancelMPPTasks cancels mpp tasks NOTE: We do not retry here, because retry is helpless when errors result from TiFlash or Network. If errors occur, the execution on TiFlash will finally stop after some minutes. This function is exclusively called, and only the first call succeeds sending tasks and setting all tasks as cancelled, while others will not work.

func (*MPPClient) CheckVisibility 

func (c *MPPClient) CheckVisibility(startTime uint64) error

CheckVisibility checks if it is safe to read using given ts.

func (*MPPClient) ConstructMPPTasks 

func (c *MPPClient) ConstructMPPTasks(ctx context.Context, req *kv.MPPBuildTasksRequest, ttl time.Duration, dispatchPolicy tiflashcompute.DispatchPolicy, tiflashReplicaReadPolicy tiflash.ReplicaRead, appendWarning func(error)) ([]kv.MPPTaskMeta, error)

ConstructMPPTasks receives ScheduleRequest, which are actually collects of kv ranges. We allocates MPPTaskMeta for them and returns.

func (*MPPClient) DispatchMPPTask 

func (c *MPPClient) DispatchMPPTask(param kv.DispatchMPPTaskParam) (resp *mpp.DispatchTaskResponse, retry bool, err error)

DispatchMPPTask dispatch mpp task, and returns valid response when retry = false and err is nil

func (*MPPClient) EstablishMPPConns 

func (c *MPPClient) EstablishMPPConns(param kv.EstablishMPPConnsParam) (*tikvrpc.MPPStreamResponse, error)

EstablishMPPConns build a mpp connection to receive data, return valid response when err is nil

func (*MPPClient) GetMPPStoreCount 

func (c *MPPClient) GetMPPStoreCount() (int, error)

GetMPPStoreCount returns number of TiFlash stores

type MPPFailedStoreProber 

type MPPFailedStoreProber struct {
	// contains filtered or unexported fields
}

MPPFailedStoreProber use for detecting of failed TiFlash instance 

var GlobalMPPFailedStoreProber *MPPFailedStoreProber

GlobalMPPFailedStoreProber mpp failed store probe

func (*MPPFailedStoreProber) Add 

func (t *MPPFailedStoreProber) Add(ctx context.Context, address string, tikvClient tikv.Client)

Add add a store when sync probe failed

func (*MPPFailedStoreProber) Delete 

func (t *MPPFailedStoreProber) Delete(address string)

Delete clean store from failed map

func (*MPPFailedStoreProber) IsRecovery 

func (t *MPPFailedStoreProber) IsRecovery(ctx context.Context, address string, recoveryTTL time.Duration) bool

IsRecovery check whether the store is recovery

func (*MPPFailedStoreProber) Run 

func (t *MPPFailedStoreProber) Run()

Run a loop of scan there can be only one background task

func (*MPPFailedStoreProber) Stop 

func (t *MPPFailedStoreProber) Stop()

Stop stop background goroutine

type MPPStoreState 

type MPPStoreState struct {
	// contains filtered or unexported fields
}

MPPStoreState the state for MPPStore.

type RegionBatchRequestSender 

type RegionBatchRequestSender struct {
	*tikv.RegionRequestSender
	// contains filtered or unexported fields
}

RegionBatchRequestSender sends BatchCop requests to TiFlash server by stream way.

func NewRegionBatchRequestSender 

func NewRegionBatchRequestSender(cache *RegionCache, client tikv.Client, oracle oracle.Oracle, enableCollectExecutionInfo bool) *RegionBatchRequestSender

NewRegionBatchRequestSender creates a RegionBatchRequestSender object.

func (*RegionBatchRequestSender) SendReqToAddr 

func (ss *RegionBatchRequestSender) SendReqToAddr(bo *Backoffer, rpcCtx *tikv.RPCContext, regionInfos []RegionInfo, req *tikvrpc.Request, timout time.Duration) (resp *tikvrpc.Response, retry bool, cancel func(), err error)

SendReqToAddr send batch cop request

type RegionCache 

type RegionCache struct {
	*tikv.RegionCache
}

RegionCache wraps tikv.RegionCache.

func NewRegionCache 

func NewRegionCache(rc *tikv.RegionCache) *RegionCache

NewRegionCache returns a new RegionCache.

func (*RegionCache) BuildBatchTask 

func (c *RegionCache) BuildBatchTask(bo *Backoffer, req *kv.Request, task *copTask, replicaRead kv.ReplicaReadType) (*batchedCopTask, error)

BuildBatchTask fetches store and peer info for cop task, wrap it as `batchedCopTask`.

func (*RegionCache) OnSendFailForBatchRegions 

func (c *RegionCache) OnSendFailForBatchRegions(bo *Backoffer, store *tikv.Store, regionInfos []RegionInfo, scheduleReload bool, err error)

OnSendFailForBatchRegions handles send request fail logic.

func (*RegionCache) SplitKeyRangesByBuckets 

func (c *RegionCache) SplitKeyRangesByBuckets(bo *Backoffer, ranges *KeyRanges) ([]*LocationKeyRanges, error)

SplitKeyRangesByBuckets splits the KeyRanges by buckets information in the cache. If regions don't have buckets, it's equal to SplitKeyRangesByLocations.

TODO(youjiali1995): Try to do it in one round and reduce allocations if bucket is not enabled.

func (*RegionCache) SplitKeyRangesByLocations 

func (c *RegionCache) SplitKeyRangesByLocations(bo *Backoffer, ranges *KeyRanges, limit int, needLeader, buckets bool) ([]*LocationKeyRanges, error)

SplitKeyRangesByLocations splits the KeyRanges by logical info in the cache. The buckets in the returned LocationKeyRanges are empty, regardless of whether the region is split by bucket.

func (*RegionCache) SplitRegionRanges 

func (c *RegionCache) SplitRegionRanges(bo *Backoffer, keyRanges []kv.KeyRange, limit int) ([]kv.KeyRange, error)

SplitRegionRanges gets the split ranges from pd region.

type RegionInfo 

type RegionInfo struct {
	Region         tikv.RegionVerID
	Meta           *metapb.Region
	Ranges         *KeyRanges
	AllStores      []uint64
	PartitionIndex int64 // used by PartitionTableScan, indicates the n-th partition of the partition table
}

RegionInfo contains region related information for batchCopTask

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store wraps tikv.KVStore and provides coprocessor utilities.

func NewStore 

func NewStore(s *tikv.KVStore, coprCacheConfig *config.CoprocessorCache) (*Store, error)

NewStore creates a new store instance.

func (Store) CheckVisibility 

func (s Store) CheckVisibility(startTime uint64) error

CheckVisibility checks if it is safe to read using given ts.

func (*Store) Close 

func (s *Store) Close()

Close releases resources allocated for coprocessor.

func (*Store) GetClient 

func (s *Store) GetClient() kv.Client

GetClient gets a client instance.

func (*Store) GetMPPClient 

func (s *Store) GetMPPClient() kv.MPPClient

GetMPPClient gets a mpp client instance.

func (Store) GetRegionCache 

func (s Store) GetRegionCache() *RegionCache

GetRegionCache returns the region cache instance.

func (Store) GetTiKVClient 

func (s Store) GetTiKVClient() tikv.Client

GetTiKVClient gets the client instance.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.