locate

package
v2.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 24, 2022 License: Apache-2.0 Imports: 39 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetStoreLivenessTimeout 

func GetStoreLivenessTimeout() time.Duration

GetStoreLivenessTimeout returns storeLivenessTimeout.

func IsFakeRegionError 

func IsFakeRegionError(err *errorpb.Error) bool

IsFakeRegionError returns true if err is fake region error.

func LoadShuttingDown 

func LoadShuttingDown() uint32

LoadShuttingDown atomically loads ShuttingDown.

func RecordRegionRequestRuntimeStats 

func RecordRegionRequestRuntimeStats(stats map[tikvrpc.CmdType]*RPCRuntimeStats, cmd tikvrpc.CmdType, d time.Duration)

RecordRegionRequestRuntimeStats records request runtime stats.

func SetRegionCacheTTLSec 

func SetRegionCacheTTLSec(t int64)

SetRegionCacheTTLSec sets regionCacheTTLSec to t.

func SetStoreLivenessTimeout 

func SetStoreLivenessTimeout(t time.Duration)

SetStoreLivenessTimeout sets storeLivenessTimeout to t.

func StoreShuttingDown 

func StoreShuttingDown(v uint32)

StoreShuttingDown atomically stores ShuttingDown into v.

Types

type AccessIndex 

type AccessIndex int

AccessIndex represent the index for accessIndex array

type CodecPDClient 

type CodecPDClient struct {
	pd.Client
}

CodecPDClient wraps a PD Client to decode the encoded keys in region meta.

func NewCodeCPDClient 

func NewCodeCPDClient(client pd.Client) *CodecPDClient

NewCodeCPDClient creates a CodecPDClient.

func (*CodecPDClient) GetPrevRegion 

func (c *CodecPDClient) GetPrevRegion(ctx context.Context, key []byte) (*pd.Region, error)

GetPrevRegion encodes the key before send requests to pd-server and decodes the returned StartKey && EndKey from pd-server.

func (*CodecPDClient) GetRegion 

func (c *CodecPDClient) GetRegion(ctx context.Context, key []byte) (*pd.Region, error)

GetRegion encodes the key before send requests to pd-server and decodes the returned StartKey && EndKey from pd-server.

func (*CodecPDClient) GetRegionByID 

func (c *CodecPDClient) GetRegionByID(ctx context.Context, regionID uint64) (*pd.Region, error)

GetRegionByID encodes the key before send requests to pd-server and decodes the returned StartKey && EndKey from pd-server.

func (*CodecPDClient) ScanRegions 

func (c *CodecPDClient) ScanRegions(ctx context.Context, startKey []byte, endKey []byte, limit int) ([]*pd.Region, error)

ScanRegions encodes the key before send requests to pd-server and decodes the returned StartKey && EndKey from pd-server.

type InvalidReason 

type InvalidReason int32

InvalidReason is the reason why a cached region is invalidated. The region cache may take different strategies to handle different reasons. For example, when a cached region is invalidated due to no leader, region cache will always access to a different peer. 

const (
	// Ok indicates the cached region is valid
	Ok InvalidReason = iota
	// NoLeader indicates it's invalidated due to no leader
	NoLeader
	// RegionNotFound indicates it's invalidated due to region not found in the store
	RegionNotFound
	// EpochNotMatch indicates it's invalidated due to epoch not match
	EpochNotMatch
	// StoreNotFound indicates it's invalidated due to store not found in PD
	StoreNotFound
	// Other indicates it's invalidated due to other reasons, e.g., the store
	// is removed from the cluster, fail to send requests to the store.
	Other
)

type KeyLocation 

type KeyLocation struct {
	Region   RegionVerID
	StartKey []byte
	EndKey   []byte
}

KeyLocation is the region and range that a key is located.

func (*KeyLocation) Contains 

func (l *KeyLocation) Contains(key []byte) bool

Contains checks if key is in [StartKey, EndKey).

func (*KeyLocation) String 

func (l *KeyLocation) String() string

String implements fmt.Stringer interface.

type RPCCanceller 

type RPCCanceller struct {
	sync.Mutex
	// contains filtered or unexported fields
}

RPCCanceller is rpc send cancelFunc collector.

func NewRPCanceller 

func NewRPCanceller() *RPCCanceller

NewRPCanceller creates RPCCanceller with init state.

func (*RPCCanceller) CancelAll 

func (h *RPCCanceller) CancelAll()

CancelAll cancels all inflight rpc context.

func (*RPCCanceller) WithCancel 

func (h *RPCCanceller) WithCancel(ctx context.Context) (context.Context, func())

WithCancel generates new context with cancel func.

type RPCCancellerCtxKey 

type RPCCancellerCtxKey struct{}

RPCCancellerCtxKey is context key attach rpc send cancelFunc collector to ctx.

type RPCContext 

type RPCContext struct {
	Region     RegionVerID
	Meta       *metapb.Region
	Peer       *metapb.Peer
	AccessIdx  AccessIndex
	Store      *Store
	Addr       string
	AccessMode accessMode
	ProxyStore *Store // nil means proxy is not used
	ProxyAddr  string // valid when ProxyStore is not nil
	TiKVNum    int    // Number of TiKV nodes among the region's peers. Assuming non-TiKV peers are all TiFlash peers.
}

RPCContext contains data that is needed to send RPC to a region.

func (*RPCContext) String 

func (c *RPCContext) String() string

type RPCRuntimeStats 

type RPCRuntimeStats struct {
	Count int64
	// Send region request consume time.
	Consume int64
}

RPCRuntimeStats indicates the RPC request count and consume time.

type Region 

type Region struct {
	// contains filtered or unexported fields
}

Region presents kv region

func (*Region) AnyStorePeer 

func (r *Region) AnyStorePeer(rs *regionStore, followerStoreSeed uint32, op *storeSelectorOp) (store *Store, peer *metapb.Peer, accessIdx AccessIndex, storeIdx int)

AnyStorePeer returns a leader or follower store with the associated peer.

func (*Region) Contains 

func (r *Region) Contains(key []byte) bool

Contains checks whether the key is in the region, for the maximum region endKey is empty. startKey <= key < endKey.

func (*Region) ContainsByEnd 

func (r *Region) ContainsByEnd(key []byte) bool

ContainsByEnd check the region contains the greatest key that is less than key. for the maximum region endKey is empty. startKey < key <= endKey.

func (*Region) EndKey 

func (r *Region) EndKey() []byte

EndKey returns EndKey.

func (*Region) FollowerStorePeer 

func (r *Region) FollowerStorePeer(rs *regionStore, followerStoreSeed uint32, op *storeSelectorOp) (store *Store, peer *metapb.Peer, accessIdx AccessIndex, storeIdx int)

FollowerStorePeer returns a follower store with follower peer.

func (*Region) GetID 

func (r *Region) GetID() uint64

GetID returns id.

func (*Region) GetLeaderPeerID 

func (r *Region) GetLeaderPeerID() uint64

GetLeaderPeerID returns leader peer ID.

func (*Region) GetLeaderStoreID 

func (r *Region) GetLeaderStoreID() uint64

GetLeaderStoreID returns the store ID of the leader region.

func (*Region) GetMeta 

func (r *Region) GetMeta() *metapb.Region

GetMeta returns region meta.

func (*Region) StartKey 

func (r *Region) StartKey() []byte

StartKey returns StartKey.

func (*Region) VerID 

func (r *Region) VerID() RegionVerID

VerID returns the Region's RegionVerID.

func (*Region) WorkStorePeer 

func (r *Region) WorkStorePeer(rs *regionStore) (store *Store, peer *metapb.Peer, accessIdx AccessIndex, storeIdx int)

WorkStorePeer returns current work store with work peer.

type RegionCache 

type RegionCache struct {
	// contains filtered or unexported fields
}

RegionCache caches Regions loaded from PD. All public methods of this struct should be thread-safe, unless explicitly pointed out or the method is for testing purposes only.

func NewRegionCache 

func NewRegionCache(pdClient pd.Client) *RegionCache

NewRegionCache creates a RegionCache.

func (*RegionCache) BatchLoadRegionsFromKey 

func (c *RegionCache) BatchLoadRegionsFromKey(bo *retry.Backoffer, startKey []byte, count int) ([]byte, error)

BatchLoadRegionsFromKey loads at most given numbers of regions to the RegionCache, from the given startKey. Returns the endKey of the last loaded region. If some of the regions has no leader, their entries in RegionCache will not be updated.

func (*RegionCache) BatchLoadRegionsWithKeyRange 

func (c *RegionCache) BatchLoadRegionsWithKeyRange(bo *retry.Backoffer, startKey []byte, endKey []byte, count int) (regions []*Region, err error)

BatchLoadRegionsWithKeyRange loads at most given numbers of regions to the RegionCache, within the given key range from the startKey to endKey. Returns the loaded regions.

func (*RegionCache) Close 

func (c *RegionCache) Close()

Close releases region cache's resource.

func (*RegionCache) GetAllValidTiFlashStores 

func (c *RegionCache) GetAllValidTiFlashStores(id RegionVerID, currentStore *Store) []uint64

GetAllValidTiFlashStores returns the store ids of all valid TiFlash stores, the store id of currentStore is always the first one

func (*RegionCache) GetCachedRegionWithRLock 

func (c *RegionCache) GetCachedRegionWithRLock(regionID RegionVerID) (r *Region)

GetCachedRegionWithRLock returns region with lock.

func (*RegionCache) GetStoresByType 

func (c *RegionCache) GetStoresByType(typ tikvrpc.EndpointType) []*Store

GetStoresByType gets stores by type `typ` TODO: revise it by get store by closure.

func (*RegionCache) GetTiFlashRPCContext 

func (c *RegionCache) GetTiFlashRPCContext(bo *retry.Backoffer, id RegionVerID, loadBalance bool) (*RPCContext, error)

GetTiFlashRPCContext returns RPCContext for a region must access flash store. If it returns nil, the region must be out of date and already dropped from cache or not flash store found. `loadBalance` is an option. For MPP and batch cop, it is pointless and might cause try the failed store repeatly.

func (*RegionCache) GetTiFlashStores 

func (c *RegionCache) GetTiFlashStores() []*Store

GetTiFlashStores returns the information of all tiflash nodes.

func (*RegionCache) GetTiKVRPCContext 

func (c *RegionCache) GetTiKVRPCContext(bo *retry.Backoffer, id RegionVerID, replicaRead kv.ReplicaReadType, followerStoreSeed uint32, opts ...StoreSelectorOption) (*RPCContext, error)

GetTiKVRPCContext returns RPCContext for a region. If it returns nil, the region must be out of date and already dropped from cache.

func (*RegionCache) GroupKeysByRegion 

func (c *RegionCache) GroupKeysByRegion(bo *retry.Backoffer, keys [][]byte, filter func(key, regionStartKey []byte) bool) (map[RegionVerID][][]byte, RegionVerID, error)

GroupKeysByRegion separates keys into groups by their belonging Regions. Specially it also returns the first key's region which may be used as the 'PrimaryLockKey' and should be committed ahead of others. filter is used to filter some unwanted keys.

func (*RegionCache) InvalidateCachedRegion 

func (c *RegionCache) InvalidateCachedRegion(id RegionVerID)

InvalidateCachedRegion removes a cached Region.

func (*RegionCache) InvalidateCachedRegionWithReason 

func (c *RegionCache) InvalidateCachedRegionWithReason(id RegionVerID, reason InvalidReason)

InvalidateCachedRegionWithReason removes a cached Region with the reason why it's invalidated.

func (*RegionCache) ListRegionIDsInKeyRange 

func (c *RegionCache) ListRegionIDsInKeyRange(bo *retry.Backoffer, startKey, endKey []byte) (regionIDs []uint64, err error)

ListRegionIDsInKeyRange lists ids of regions in [start_key,end_key].

func (*RegionCache) LoadRegionsInKeyRange 

func (c *RegionCache) LoadRegionsInKeyRange(bo *retry.Backoffer, startKey, endKey []byte) (regions []*Region, err error)

LoadRegionsInKeyRange lists regions in [start_key,end_key].

func (*RegionCache) LocateEndKey 

func (c *RegionCache) LocateEndKey(bo *retry.Backoffer, key []byte) (*KeyLocation, error)

LocateEndKey searches for the region and range that the key is located. Unlike LocateKey, start key of a region is exclusive and end key is inclusive.

func (*RegionCache) LocateKey 

func (c *RegionCache) LocateKey(bo *retry.Backoffer, key []byte) (*KeyLocation, error)

LocateKey searches for the region and range that the key is located.

func (*RegionCache) LocateRegionByID 

func (c *RegionCache) LocateRegionByID(bo *retry.Backoffer, regionID uint64) (*KeyLocation, error)

LocateRegionByID searches for the region with ID.

func (*RegionCache) OnRegionEpochNotMatch 

func (c *RegionCache) OnRegionEpochNotMatch(bo *retry.Backoffer, ctx *RPCContext, currentRegions []*metapb.Region) (bool, error)

OnRegionEpochNotMatch removes the old region and inserts new regions into the cache. It returns whether retries the request because it's possible the region epoch is ahead of TiKV's due to slow appling.

func (*RegionCache) OnSendFail 

func (c *RegionCache) OnSendFail(bo *retry.Backoffer, ctx *RPCContext, scheduleReload bool, err error)

OnSendFail handles send request fail logic.

func (*RegionCache) OnSendFailForTiFlash 

func (c *RegionCache) OnSendFailForTiFlash(bo *retry.Backoffer, store *Store, region RegionVerID, prev *metapb.Region, scheduleReload bool, err error, skipSwitchPeerLog bool)

OnSendFailForTiFlash handles send request fail logic for tiflash.

func (*RegionCache) PDClient 

func (c *RegionCache) PDClient() pd.Client

PDClient returns the pd.Client in RegionCache.

func (*RegionCache) SetPDClient 

func (c *RegionCache) SetPDClient(client pd.Client)

SetPDClient replaces pd client,for testing only

func (*RegionCache) SetRegionCacheStore 

func (c *RegionCache) SetRegionCacheStore(id uint64, storeType tikvrpc.EndpointType, state uint64, labels []*metapb.StoreLabel)

SetRegionCacheStore is used to set a store in region cache, for testing only

func (*RegionCache) UpdateLeader 

func (c *RegionCache) UpdateLeader(regionID RegionVerID, leader *metapb.Peer, currentPeerIdx AccessIndex)

UpdateLeader update some region cache with newer leader info.

type RegionRequestRuntimeStats 

type RegionRequestRuntimeStats struct {
	Stats map[tikvrpc.CmdType]*RPCRuntimeStats
}

RegionRequestRuntimeStats records the runtime stats of send region requests.

func NewRegionRequestRuntimeStats 

func NewRegionRequestRuntimeStats() RegionRequestRuntimeStats

NewRegionRequestRuntimeStats returns a new RegionRequestRuntimeStats.

func (*RegionRequestRuntimeStats) Clone 

func (r *RegionRequestRuntimeStats) Clone() RegionRequestRuntimeStats

Clone returns a copy of itself.

func (*RegionRequestRuntimeStats) Merge 

func (r *RegionRequestRuntimeStats) Merge(rs RegionRequestRuntimeStats)

Merge merges other RegionRequestRuntimeStats.

func (*RegionRequestRuntimeStats) String 

func (r *RegionRequestRuntimeStats) String() string

String implements fmt.Stringer interface.

type RegionRequestSender 

type RegionRequestSender struct {
	RegionRequestRuntimeStats
	// contains filtered or unexported fields
}

RegionRequestSender sends KV/Cop requests to tikv server. It handles network errors and some region errors internally.

Typically, a KV/Cop request is bind to a region, all keys that are involved in the request should be located in the region. The sending process begins with looking for the address of leader store's address of the target region from cache, and the request is then sent to the destination tikv server over TCP connection. If region is updated, can be caused by leader transfer, region split, region merge, or region balance, tikv server may not able to process request and send back a RegionError. RegionRequestSender takes care of errors that does not relevant to region range, such as 'I/O timeout', 'NotLeader', and 'ServerIsBusy'. If fails to send the request to all replicas, a fake rregion error may be returned. Caller which receives the error should retry the request.

For other region errors, since region range have changed, the request may need to split, so we simply return the error to caller.

func NewRegionRequestSender 

func NewRegionRequestSender(regionCache *RegionCache, client client.Client) *RegionRequestSender

NewRegionRequestSender creates a new sender.

func (*RegionRequestSender) GetClient 

func (s *RegionRequestSender) GetClient() client.Client

GetClient returns the RPC client.

func (*RegionRequestSender) GetRPCError 

func (s *RegionRequestSender) GetRPCError() error

GetRPCError returns the RPC error.

func (*RegionRequestSender) GetRegionCache 

func (s *RegionRequestSender) GetRegionCache() *RegionCache

GetRegionCache returns the region cache.

func (*RegionRequestSender) GetStoreAddr 

func (s *RegionRequestSender) GetStoreAddr() string

GetStoreAddr returns the dest store address.

func (*RegionRequestSender) NeedReloadRegion 

func (s *RegionRequestSender) NeedReloadRegion(ctx *RPCContext) (need bool)

NeedReloadRegion checks is all peers has sent failed, if so need reload.

func (*RegionRequestSender) SendReq 

func (s *RegionRequestSender) SendReq(bo *retry.Backoffer, req *tikvrpc.Request, regionID RegionVerID, timeout time.Duration) (*tikvrpc.Response, error)

SendReq sends a request to tikv server. If fails to send the request to all replicas, a fake region error may be returned. Caller which receives the error should retry the request.

func (*RegionRequestSender) SendReqCtx 

func (s *RegionRequestSender) SendReqCtx(
	bo *retry.Backoffer,
	req *tikvrpc.Request,
	regionID RegionVerID,
	timeout time.Duration,
	et tikvrpc.EndpointType,
	opts ...StoreSelectorOption,
) (
	resp *tikvrpc.Response,
	rpcCtx *RPCContext,
	err error,
)

SendReqCtx sends a request to tikv server and return response and RPCCtx of this RPC.

func (*RegionRequestSender) SetRPCError 

func (s *RegionRequestSender) SetRPCError(err error)

SetRPCError rewrite the rpc error.

func (*RegionRequestSender) SetStoreAddr 

func (s *RegionRequestSender) SetStoreAddr(addr string)

SetStoreAddr specifies the dest store address.

type RegionVerID 

type RegionVerID struct {
	// contains filtered or unexported fields
}

RegionVerID is a unique ID that can identify a Region at a specific version.

func NewRegionVerID 

func NewRegionVerID(id, confVer, ver uint64) RegionVerID

NewRegionVerID creates a region ver id, which used for invalidating regions.

func (*RegionVerID) Equals 

func (r *RegionVerID) Equals(another RegionVerID) bool

Equals checks whether the RegionVerID equals to another one

func (*RegionVerID) GetConfVer 

func (r *RegionVerID) GetConfVer() uint64

GetConfVer returns the conf ver of the region's epoch

func (*RegionVerID) GetID 

func (r *RegionVerID) GetID() uint64

GetID returns the id of the region

func (*RegionVerID) GetVer 

func (r *RegionVerID) GetVer() uint64

GetVer returns the version of the region's epoch

func (*RegionVerID) String 

func (r *RegionVerID) String() string

String formats the RegionVerID to string

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store contains a kv process's address.

func (*Store) GetAddr 

func (s *Store) GetAddr() string

GetAddr returns the address of the store

func (*Store) IsLabelsMatch 

func (s *Store) IsLabelsMatch(labels []*metapb.StoreLabel) bool

IsLabelsMatch return whether the store's labels match the target labels

func (*Store) IsSameLabels 

func (s *Store) IsSameLabels(labels []*metapb.StoreLabel) bool

IsSameLabels returns whether the store have the same labels with target labels

func (*Store) IsTiFlash 

func (s *Store) IsTiFlash() bool

IsTiFlash returns true if the storeType is TiFlash

func (*Store) StoreID 

func (s *Store) StoreID() uint64

StoreID returns storeID.

type StoreSelectorOption 

type StoreSelectorOption func(*storeSelectorOp)

StoreSelectorOption configures storeSelectorOp.

func WithLeaderOnly 

func WithLeaderOnly() StoreSelectorOption

WithLeaderOnly indicates selecting stores with leader only.

func WithMatchLabels 

func WithMatchLabels(labels []*metapb.StoreLabel) StoreSelectorOption

WithMatchLabels indicates selecting stores with matched labels.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.