vtgate

package
v0.0.0-...-d4b8c80 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 24, 2014 License: BSD-3-Clause Imports: 22 Imported by: 0

Documentation

Overview

Package vtgate provides query routing rpc services for vttablets.

Package vtgate provides query routing rpc services for vttablets.

Index

Constants

This section is empty.

Variables

View Source
var (
	RpcVTGate *VTGate

	QPSByOperation *stats.Rates
	QPSByKeyspace  *stats.Rates
	QPSByDbType    *stats.Rates

	ErrorsByOperation *stats.Rates
	ErrorsByKeyspace  *stats.Rates
	ErrorsByDbType    *stats.Rates
)
View Source
var RegisterVTGates []RegisterVTGate

Functions

func Init

func Init(serv SrvTopoServer, cell string, retryDelay time.Duration, retryCount int, timeout time.Duration)

func StrsEquals

func StrsEquals(a, b []string) bool

StrsEquals compares contents of two string slices.

Types

type Balancer

type Balancer struct {
	// contains filtered or unexported fields
}

Balancer is a simple round-robin load balancer. It allows you to temporarily mark down nodes that are non-functional.

func NewBalancer

func NewBalancer(getEndPoints GetEndPointsFunc, retryDelay time.Duration) *Balancer

NewBalancer creates a Balancer. getAddreses is the function it will use to refresh the list of addresses if one of the nodes has been marked down. The list of addresses is shuffled. retryDelay specifies the minimum time a node will be marked down before it will be cleared for a retry.

func (*Balancer) Get

func (blc *Balancer) Get() (endPoint topo.EndPoint, err error)

Get returns a single endpoint that was not recently marked down. If it finds an address that was down for longer than retryDelay, it refreshes the list of addresses and returns the next available node. If all addresses are marked down, it waits and retries. If a refresh fails, it returns an error.

func (*Balancer) MarkDown

func (blc *Balancer) MarkDown(uid uint32)

MarkDown marks the specified address down. Such addresses will not be used by Balancer for the duration of retryDelay.

type EndPointsCacheStatus

type EndPointsCacheStatus struct {
	Cell             string
	Keyspace         string
	Shard            string
	TabletType       topo.TabletType
	Value            *topo.EndPoints
	OriginalValue    *topo.EndPoints
	LastError        error
	LastErrorContext context.Context
}

EndPointsCacheStatus is the current value for an EndPoints object

func (*EndPointsCacheStatus) StatusAsHTML

func (st *EndPointsCacheStatus) StatusAsHTML() template.HTML

StatusAsHTML returns an HTML version of our status. It works best if there is data in the cache.

type EndPointsCacheStatusList

type EndPointsCacheStatusList []*EndPointsCacheStatus

EndPointsCacheStatusList is used for sorting

func (EndPointsCacheStatusList) Len

func (epcsl EndPointsCacheStatusList) Len() int

Len is part of sort.Interface

func (EndPointsCacheStatusList) Less

func (epcsl EndPointsCacheStatusList) Less(i, j int) bool

Less is part of sort.Interface

func (EndPointsCacheStatusList) Swap

func (epcsl EndPointsCacheStatusList) Swap(i, j int)

Swap is part of sort.Interface

type GetEndPointsFunc

type GetEndPointsFunc func() (*topo.EndPoints, error)

type RegisterVTGate

type RegisterVTGate func(*VTGate)

registration mechanism

type ResilientSrvTopoServer

type ResilientSrvTopoServer struct {
	// contains filtered or unexported fields
}

ResilientSrvTopoServer is an implementation of SrvTopoServer based on a topo.Server that uses a cache for two purposes: - limit the QPS to the underlying topo.Server - return the last known value of the data if there is an error

func NewResilientSrvTopoServer

func NewResilientSrvTopoServer(base topo.Server, counterName string) *ResilientSrvTopoServer

NewResilientSrvTopoServer creates a new ResilientSrvTopoServer based on the provided SrvTopoServer.

func (*ResilientSrvTopoServer) CacheStatus

CacheStatus returns a displayable version of the cache

func (*ResilientSrvTopoServer) DegradedEndpointCount

func (server *ResilientSrvTopoServer) DegradedEndpointCount() map[string]int64

DegradedEndpointCount returns how many degraded endpoints we have in the cache (entries that are not 100% healthy, because they are behind on replication for instance)

func (*ResilientSrvTopoServer) EndpointCount

func (server *ResilientSrvTopoServer) EndpointCount() map[string]int64

EndpointCount returns how many endpoints we have per keyspace/shard/dbtype.

func (*ResilientSrvTopoServer) GetEndPoints

func (server *ResilientSrvTopoServer) GetEndPoints(context context.Context, cell, keyspace, shard string, tabletType topo.TabletType) (*topo.EndPoints, error)

func (*ResilientSrvTopoServer) GetSrvKeyspace

func (server *ResilientSrvTopoServer) GetSrvKeyspace(context context.Context, cell, keyspace string) (*topo.SrvKeyspace, error)

func (*ResilientSrvTopoServer) GetSrvKeyspaceNames

func (server *ResilientSrvTopoServer) GetSrvKeyspaceNames(context context.Context, cell string) ([]string, error)

type ResilientSrvTopoServerCacheStatus

type ResilientSrvTopoServerCacheStatus struct {
	SrvKeyspaceNames SrvKeyspaceNamesCacheStatusList
	SrvKeyspaces     SrvKeyspaceCacheStatusList
	EndPoints        EndPointsCacheStatusList
}

ResilientSrvTopoServerCacheStatus has the full status of the cache

type Resolver

type Resolver struct {
	// contains filtered or unexported fields
}

Resolver is the layer to resolve KeyspaceIds and KeyRanges to shards. It will try to re-resolve shards if ScatterConn returns retryable error, which may imply horizontal or vertical resharding happened.

func NewResolver

func NewResolver(serv SrvTopoServer, statsName, cell string, retryDelay time.Duration, retryCount int, timeout time.Duration) *Resolver

NewResolver creates a new Resolver. All input parameters are passed through for creating ScatterConn.

func (*Resolver) Commit

func (res *Resolver) Commit(context context.Context, inSession *proto.Session) error

Commit commits a transaction.

func (*Resolver) Execute

func (res *Resolver) Execute(
	context context.Context,
	sql string,
	bindVars map[string]interface{},
	keyspace string,
	tabletType topo.TabletType,
	session *proto.Session,
	mapToShards func(string) (string, []string, error),
) (*mproto.QueryResult, error)

Execute executes a non-streaming query based on shards resolved by given func. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) ExecuteBatch

func (res *Resolver) ExecuteBatch(
	context context.Context,
	queries []tproto.BoundQuery,
	keyspace string,
	tabletType topo.TabletType,
	session *proto.Session,
	mapToShards func(string) (string, []string, error),
) (*tproto.QueryResultList, error)

ExecuteBatch executes a group of queries based on shards resolved by given func. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) ExecuteBatchKeyspaceIds

func (res *Resolver) ExecuteBatchKeyspaceIds(context context.Context, query *proto.KeyspaceIdBatchQuery) (*tproto.QueryResultList, error)

ExecuteBatchKeyspaceIds executes a group of queries based on KeyspaceIds. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) ExecuteEntityIds

func (res *Resolver) ExecuteEntityIds(
	context context.Context,
	query *proto.EntityIdsQuery,
) (*mproto.QueryResult, error)

ExecuteEntityIds executes a non-streaming query based on given KeyspaceId map. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) ExecuteKeyRanges

func (res *Resolver) ExecuteKeyRanges(context context.Context, query *proto.KeyRangeQuery) (*mproto.QueryResult, error)

ExecuteKeyRanges executes a non-streaming query based on KeyRanges. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) ExecuteKeyspaceIds

func (res *Resolver) ExecuteKeyspaceIds(context context.Context, query *proto.KeyspaceIdQuery) (*mproto.QueryResult, error)

ExecuteKeyspaceIds executes a non-streaming query based on KeyspaceIds. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) Rollback

func (res *Resolver) Rollback(context context.Context, inSession *proto.Session) error

Rollback rolls back a transaction.

func (*Resolver) StreamExecute

func (res *Resolver) StreamExecute(
	context context.Context,
	sql string,
	bindVars map[string]interface{},
	keyspace string,
	tabletType topo.TabletType,
	session *proto.Session,
	mapToShards func(string) (string, []string, error),
	sendReply func(*mproto.QueryResult) error,
) error

StreamExecuteShard executes a streaming query on shards resolved by given func. This function currently temporarily enforces the restriction of executing on one shard since it cannot merge-sort the results to guarantee ordering of response which is needed for checkpointing.

func (*Resolver) StreamExecuteKeyRanges

func (res *Resolver) StreamExecuteKeyRanges(context context.Context, query *proto.KeyRangeQuery, sendReply func(*mproto.QueryResult) error) error

StreamExecuteKeyRanges executes a streaming query on the specified KeyRanges. The KeyRanges are resolved to shards using the serving graph. This function currently temporarily enforces the restriction of executing on one shard since it cannot merge-sort the results to guarantee ordering of response which is needed for checkpointing. The api supports supplying multiple keyranges to make it future proof.

func (*Resolver) StreamExecuteKeyspaceIds

func (res *Resolver) StreamExecuteKeyspaceIds(context context.Context, query *proto.KeyspaceIdQuery, sendReply func(*mproto.QueryResult) error) error

StreamExecuteKeyspaceIds executes a streaming query on the specified KeyspaceIds. The KeyspaceIds are resolved to shards using the serving graph. This function currently temporarily enforces the restriction of executing on one shard since it cannot merge-sort the results to guarantee ordering of response which is needed for checkpointing. The api supports supplying multiple KeyspaceIds to make it future proof.

type SafeSession

type SafeSession struct {
	*proto.Session
	// contains filtered or unexported fields
}

func NewSafeSession

func NewSafeSession(sessn *proto.Session) *SafeSession

func (*SafeSession) Append

func (session *SafeSession) Append(shardSession *proto.ShardSession)

func (*SafeSession) Find

func (session *SafeSession) Find(keyspace, shard string, tabletType topo.TabletType) int64

func (*SafeSession) InTransaction

func (session *SafeSession) InTransaction() bool

func (*SafeSession) Reset

func (session *SafeSession) Reset()

type ScatterConn

type ScatterConn struct {
	// contains filtered or unexported fields
}

ScatterConn is used for executing queries across multiple ShardConn connections.

func NewScatterConn

func NewScatterConn(serv SrvTopoServer, statsName, cell string, retryDelay time.Duration, retryCount int, timeout time.Duration) *ScatterConn

NewScatterConn creates a new ScatterConn. All input parameters are passed through for creating the appropriate ShardConn.

func (*ScatterConn) Close

func (stc *ScatterConn) Close() error

Close closes the underlying ShardConn connections.

func (*ScatterConn) Commit

func (stc *ScatterConn) Commit(context context.Context, session *SafeSession) (err error)

Commit commits the current transaction. There are no retries on this operation.

func (*ScatterConn) Execute

func (stc *ScatterConn) Execute(
	context context.Context,
	query string,
	bindVars map[string]interface{},
	keyspace string,
	shards []string,
	tabletType topo.TabletType,
	session *SafeSession,
) (*mproto.QueryResult, error)

Execute executes a non-streaming query on the specified shards.

func (*ScatterConn) ExecuteBatch

func (stc *ScatterConn) ExecuteBatch(
	context context.Context,
	queries []tproto.BoundQuery,
	keyspace string,
	shards []string,
	tabletType topo.TabletType,
	session *SafeSession,
) (qrs *tproto.QueryResultList, err error)

ExecuteBatch executes a batch of non-streaming queries on the specified shards.

func (*ScatterConn) ExecuteEntityIds

func (stc *ScatterConn) ExecuteEntityIds(
	context context.Context,
	shards []string,
	sqls map[string]string,
	bindVars map[string]map[string]interface{},
	keyspace string,
	tabletType topo.TabletType,
	session *SafeSession,
) (*mproto.QueryResult, error)

func (*ScatterConn) Rollback

func (stc *ScatterConn) Rollback(context context.Context, session *SafeSession) (err error)

Rollback rolls back the current transaction. There are no retries on this operation.

func (*ScatterConn) StreamExecute

func (stc *ScatterConn) StreamExecute(
	context context.Context,
	query string,
	bindVars map[string]interface{},
	keyspace string,
	shards []string,
	tabletType topo.TabletType,
	session *SafeSession,
	sendReply func(reply *mproto.QueryResult) error,
) error

StreamExecute executes a streaming query on vttablet. The retry rules are the same.

type ShardConn

type ShardConn struct {
	// contains filtered or unexported fields
}

ShardConn represents a load balanced connection to a group of vttablets that belong to the same shard. ShardConn can be concurrently used across goroutines. Such requests are interleaved on the same underlying connection.

func NewShardConn

func NewShardConn(ctx context.Context, serv SrvTopoServer, cell, keyspace, shard string, tabletType topo.TabletType, retryDelay time.Duration, retryCount int, timeout time.Duration) *ShardConn

NewShardConn creates a new ShardConn. It creates a Balancer using serv, cell, keyspace, tabletType and retryDelay. retryCount is the max number of retries before a ShardConn returns an error on an operation.

func (*ShardConn) Begin

func (sdc *ShardConn) Begin(ctx context.Context) (transactionID int64, err error)

Begin begins a transaction. The retry rules are the same as Execute.

func (*ShardConn) Close

func (sdc *ShardConn) Close()

Close closes the underlying TabletConn. ShardConn can be reused after this because it opens connections on demand.

func (*ShardConn) Commit

func (sdc *ShardConn) Commit(ctx context.Context, transactionID int64) (err error)

Commit commits the current transaction. The retry rules are the same as Execute.

func (*ShardConn) Execute

func (sdc *ShardConn) Execute(ctx context.Context, query string, bindVars map[string]interface{}, transactionID int64) (qr *mproto.QueryResult, err error)

Execute executes a non-streaming query on vttablet. If there are connection errors, it retries retryCount times before failing. It does not retry if the connection is in the middle of a transaction.

func (*ShardConn) ExecuteBatch

func (sdc *ShardConn) ExecuteBatch(ctx context.Context, queries []tproto.BoundQuery, transactionID int64) (qrs *tproto.QueryResultList, err error)

ExecuteBatch executes a group of queries. The retry rules are the same as Execute.

func (*ShardConn) Rollback

func (sdc *ShardConn) Rollback(ctx context.Context, transactionID int64) (err error)

Rollback rolls back the current transaction. The retry rules are the same as Execute.

func (*ShardConn) StreamExecute

func (sdc *ShardConn) StreamExecute(ctx context.Context, query string, bindVars map[string]interface{}, transactionID int64) (<-chan *mproto.QueryResult, tabletconn.ErrFunc)

StreamExecute executes a streaming query on vttablet. The retry rules are the same as Execute.

func (*ShardConn) WrapError

func (sdc *ShardConn) WrapError(in error, conn tabletconn.TabletConn, inTransaction bool) (wrapped error)

WrapError returns ShardConnError which preserves the original error code if possible, adds the connection context and adds a bit to determine whether the keyspace/shard needs to be re-resolved for a potential sharding event.

type ShardConnError

type ShardConnError struct {
	Code            int
	ShardIdentifier string
	InTransaction   bool
	Err             string
}

func (*ShardConnError) Error

func (e *ShardConnError) Error() string

type SrvKeyspaceCacheStatus

type SrvKeyspaceCacheStatus struct {
	Cell             string
	Keyspace         string
	Value            *topo.SrvKeyspace
	LastError        error
	LastErrorContext context.Context
}

SrvKeyspaceCacheStatus is the current value for a SrvKeyspace object

func (*SrvKeyspaceCacheStatus) StatusAsHTML

func (st *SrvKeyspaceCacheStatus) StatusAsHTML() template.HTML

StatusAsHTML returns an HTML version of our status. It works best if there is data in the cache.

type SrvKeyspaceCacheStatusList

type SrvKeyspaceCacheStatusList []*SrvKeyspaceCacheStatus

SrvKeyspaceCacheStatusList is used for sorting

func (SrvKeyspaceCacheStatusList) Len

func (skcsl SrvKeyspaceCacheStatusList) Len() int

Len is part of sort.Interface

func (SrvKeyspaceCacheStatusList) Less

func (skcsl SrvKeyspaceCacheStatusList) Less(i, j int) bool

Less is part of sort.Interface

func (SrvKeyspaceCacheStatusList) Swap

func (skcsl SrvKeyspaceCacheStatusList) Swap(i, j int)

Swap is part of sort.Interface

type SrvKeyspaceNamesCacheStatus

type SrvKeyspaceNamesCacheStatus struct {
	Cell             string
	Value            []string
	LastError        error
	LastErrorContext context.Context
}

SrvKeyspaceNamesCacheStatus is the current value for SrvKeyspaceNames

type SrvKeyspaceNamesCacheStatusList

type SrvKeyspaceNamesCacheStatusList []*SrvKeyspaceNamesCacheStatus

SrvKeyspaceNamesCacheStatusList is used for sorting

func (SrvKeyspaceNamesCacheStatusList) Len

func (skncsl SrvKeyspaceNamesCacheStatusList) Len() int

Len is part of sort.Interface

func (SrvKeyspaceNamesCacheStatusList) Less

func (skncsl SrvKeyspaceNamesCacheStatusList) Less(i, j int) bool

Less is part of sort.Interface

func (SrvKeyspaceNamesCacheStatusList) Swap

func (skncsl SrvKeyspaceNamesCacheStatusList) Swap(i, j int)

Swap is part of sort.Interface

type SrvTopoServer

type SrvTopoServer interface {
	GetSrvKeyspaceNames(context context.Context, cell string) ([]string, error)

	GetSrvKeyspace(context context.Context, cell, keyspace string) (*topo.SrvKeyspace, error)

	GetEndPoints(context context.Context, cell, keyspace, shard string, tabletType topo.TabletType) (*topo.EndPoints, error)
}

SrvTopoServer is a subset of topo.Server that only contains the serving graph read-only calls used by clients to resolve serving addresses.

type VTGate

type VTGate struct {
	// contains filtered or unexported fields
}

VTGate is the rpc interface to vtgate. Only one instance can be created.

func (*VTGate) Begin

func (vtg *VTGate) Begin(context context.Context, outSession *proto.Session) error

Begin begins a transaction. It has to be concluded by a Commit or Rollback.

func (*VTGate) Commit

func (vtg *VTGate) Commit(context context.Context, inSession *proto.Session) error

Commit commits a transaction.

func (*VTGate) ExecuteBatchKeyspaceIds

func (vtg *VTGate) ExecuteBatchKeyspaceIds(context context.Context, query *proto.KeyspaceIdBatchQuery, reply *proto.QueryResultList) error

ExecuteBatchKeyspaceIds executes a group of queries based on the specified keyspace ids.

func (*VTGate) ExecuteBatchShard

func (vtg *VTGate) ExecuteBatchShard(context context.Context, batchQuery *proto.BatchQueryShard, reply *proto.QueryResultList) error

ExecuteBatchShard executes a group of queries on the specified shards.

func (*VTGate) ExecuteEntityIds

func (vtg *VTGate) ExecuteEntityIds(context context.Context, query *proto.EntityIdsQuery, reply *proto.QueryResult) error

ExecuteEntityIds excutes a non-streaming query based on given KeyspaceId map.

func (*VTGate) ExecuteKeyRanges

func (vtg *VTGate) ExecuteKeyRanges(context context.Context, query *proto.KeyRangeQuery, reply *proto.QueryResult) error

ExecuteKeyRanges executes a non-streaming query based on the specified keyranges.

func (*VTGate) ExecuteKeyspaceIds

func (vtg *VTGate) ExecuteKeyspaceIds(context context.Context, query *proto.KeyspaceIdQuery, reply *proto.QueryResult) error

ExecuteKeyspaceIds executes a non-streaming query based on the specified keyspace ids.

func (*VTGate) ExecuteShard

func (vtg *VTGate) ExecuteShard(context context.Context, query *proto.QueryShard, reply *proto.QueryResult) error

ExecuteShard executes a non-streaming query on the specified shards.

func (*VTGate) Rollback

func (vtg *VTGate) Rollback(context context.Context, inSession *proto.Session) error

Rollback rolls back a transaction.

func (*VTGate) StreamExecuteKeyRanges

func (vtg *VTGate) StreamExecuteKeyRanges(context context.Context, query *proto.KeyRangeQuery, sendReply func(*proto.QueryResult) error) error

StreamExecuteKeyRanges executes a streaming query on the specified KeyRanges. The KeyRanges are resolved to shards using the serving graph. This function currently temporarily enforces the restriction of executing on one shard since it cannot merge-sort the results to guarantee ordering of response which is needed for checkpointing. The api supports supplying multiple keyranges to make it future proof.

func (*VTGate) StreamExecuteKeyspaceIds

func (vtg *VTGate) StreamExecuteKeyspaceIds(context context.Context, query *proto.KeyspaceIdQuery, sendReply func(*proto.QueryResult) error) error

StreamExecuteKeyspaceIds executes a streaming query on the specified KeyspaceIds. The KeyspaceIds are resolved to shards using the serving graph. This function currently temporarily enforces the restriction of executing on one shard since it cannot merge-sort the results to guarantee ordering of response which is needed for checkpointing. The api supports supplying multiple KeyspaceIds to make it future proof.

func (*VTGate) StreamExecuteShard

func (vtg *VTGate) StreamExecuteShard(context context.Context, query *proto.QueryShard, sendReply func(*proto.QueryResult) error) error

StreamExecuteShard executes a streaming query on the specified shards.

Directories

Path Synopsis
Package gorpcvtgateservice provides to go rpc glue for vtgate
Package gorpcvtgateservice provides to go rpc glue for vtgate

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL