proxy

package
v1.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 20, 2024 License: Apache-2.0 Imports: 51 Imported by: 0

Documentation

Overview

Package proxy is used to route client connection to specified CN server and keep balance between CN servers.

1. Login AS MO use MySQL protocol, the handshake process is needed during the login phase. The general process is as follows: (1) The client sends a TCP connection to the proxy. (2) The proxy sends the init handshake package to the client. (3) The client sends the handshake response to the proxy, including user name, auth information, connection attributes, etc., and the proxy will save this part of the information and use it when switching CN servers in the future. (4) According to the login information of the client, the proxy selects a CN server to establish a connection, and uses the handshake response just saved for verification. (5) If the verification is successful, return the client OK package, otherwise return an error package.

2. Connection management When a client connects to the proxy, its connection will be saved and managed, and the number of connections established on each CN server will be recorded in real time. This information is used for CN server selection. When a new connection arrives and the proxy tries to select a CN server, it will choose the CN server with the fewest connections to establish the connection.

3. Data interaction Proxy uses MySQL packet as the basic unit for forwarding, and internally maintains a buffer. When receiving data from the client, at least one complete MySQL packet is received before forwarding to the CN server. Conversely, the same is true for data received from the server. Therefore, it can be observed internally whether the current connection is in the process of executing a transaction.

4. Connection balance Proxy will regularly load balance the connections on the CN servers. When there are too many connections on a certain CN server, it will select these connections and try to migrate them to other CN servers. The migration process is transparent to the client.

A security judgment will be made before attempting to migrate: (1) Ignore if migration is in progress. (2) The message returned by the server must be later than the client's request, otherwise the migration cannot be done. (3) If you are in a transaction, you cannot do migration. Tracking of transaction state is recorded as data is interacted.

5. Scaling: The proxy module regularly checks the work state of the cn service. If the state is draining, then migrate the tunnels on the cn to appropriate cns. If no suitable cn is found, an error will be reported and the original connection will become unavailable.

6. Usage Proxy is mainly used on the cloud platform. If you want to use proxy locally, you need to add configuration -with-proxy to start the proxy module in launch configuration mode, and add configuration in CN config file: [cn.frontend] proxy-enabled = true

The default port is of proxy service 6009.

After startup, connect to port 6001 to log in normally, and then use the internal command to configure the label of cn:

select mo_ctl('cn', 'label', 'cn_uuid:key:value');

Examples:

1. configure tenant information:

mysql -h127.0.0.1 -udump -P6001 -p111

create account acc1 admin_name 'root' identified by '111'; select mo_ctl('cn', 'label', 'dd1dccb4-4d3c-41f8-b482-5251dc7a41bf:account:acc1');

Then you can connect to the proxy to log in to the cluster:

mysql -h127.0.0.1 -uacc1:root -P6009 -p111

2. Common labels

Common labels need to pass to MO server with JDBC connection:

jdbc:mysql://localhost:6009/db123?serverTimezone=UTC&connectionAttributes=key1:value1

The labels on CN server need to be set through mo_ctl in local env or yaml file in cloud env.

Common labels also is supported by mysql client:

mysql -h127.0.0.1 -udump?k1:v1,k2:v2 -P6009 -p111

Index

Constants

View Source 
const (
	BootstrapInterval = time.Millisecond * 200
	BootstrapTimeout  = time.Minute * 5
)
View Source 
const (
	// TypeMin is the minimal event type.
	TypeMin eventType = 0
	// TypeKill indicates the kill query statement.
	TypeKill eventType = 1
	// TypeSetVar indicates the set variable statement.
	TypeSetVar eventType = 2
	// TypeUpgrade indicates the "upgrade account all" statement.
	TypeUpgrade eventType = 4
)
View Source 
const (
	// ProxyProtocolV2Signature is the signature of the Proxy Protocol version 2 header.
	ProxyProtocolV2Signature = "\x0D\x0A\x0D\x0A\x00\x0D\x0A\x51\x55\x49\x54\x0A"

	ProxyHeaderLength = 16
)
View Source 
const SQLUsername = "proxy_user"

Variables

View Source 
var ErrNoAvailableCNServers = moerr.NewInternalErrorNoCtx("no available CN servers")
View Source 
var RebalancePolicyMapping = map[string]RebalancePolicy{
	"active":  RebalancePolicyActive,
	"passive": RebalancePolicyPassive,
}

Functions

func WithProxyProtocolCodec 

func WithProxyProtocolCodec(c codec.Codec) codec.Codec

Types

type CNServer 

type CNServer struct {
	// contains filtered or unexported fields
}

CNServer represents the backend CN server, including salt, tenant, uuid and address. When there is a new client connection, a new CNServer will be created.

func (*CNServer) Connect 

func (s *CNServer) Connect(logger *zap.Logger, timeout time.Duration) (goetty.IOSession, error)

Connect connects to backend server and returns IOSession.

type ClientConn 

type ClientConn interface {
	// ConnID returns the connection ID.
	ConnID() uint32
	// GetSalt returns the salt value of this connection.
	GetSalt() []byte
	// GetHandshakePack returns the handshake response packet
	// which is received from client.
	GetHandshakePack() *frontend.Packet
	// RawConn return the raw connection.
	RawConn() net.Conn
	// GetTenant returns the tenant which this connection belongs to.
	GetTenant() Tenant
	// SendErrToClient sends access error to MySQL client.
	SendErrToClient(err error)
	// BuildConnWithServer selects a CN server and connects to it, then
	// returns the connection. If sendToClient is true, means that the
	// packet received from CN server should be sent to client because
	// it is the first time to build connection and login. And if it is
	// false, means that the packet received from CN server should NOT
	// be sent to client because we are transferring CN server connection,
	// and it is not the first time to build connection and login has been
	// finished already.
	// prevAddr is empty if it is the first time to build connection with
	// a cn server; otherwise, it is the address of the previous cn node
	// when it is transferring connection and the handshake phase is ignored.
	BuildConnWithServer(prevAddr string) (ServerConn, error)
	// HandleEvent handles event that comes from tunnel data flow.
	HandleEvent(ctx context.Context, e IEvent, resp chan<- []byte) error
	// Close closes the client connection.
	Close() error
}

ClientConn is the connection to the client.

type Config 

type Config struct {
	UUID          string `toml:"uuid"`
	ListenAddress string `toml:"listen-address" user_setting:"basic"`
	// RebalanceInterval is the interval between two rebalance operations.
	RebalanceInterval toml.Duration `toml:"rebalance-interval" user_setting:"advanced"`
	// RebalanceDisabled indicates that the rebalancer is disabled.
	RebalanceDisabled bool `toml:"rebalance-disabled" user_setting:"advanced"`
	// RebalanceTolerance indicates the rebalancer's tolerance.
	// Connections above the avg*(1+tolerance) will be migrated to
	// other CN servers. This value should be less than 1.
	RebalanceTolerance float64 `toml:"rebalance-tolerance" user_setting:"advanced"`
	// RebalancePolicy indicates that the rebalance policy, which could be active or
	// passive currently. Active means that the connection transfer will be more proactive
	// to make sure the sessions are balanced in all CN servers. Default value is "active".
	RebalancePolicy string `toml:"rebalance-proactive" user_setting:"advanced"`
	// ConnectTimeout is the timeout duration when proxy connects to backend
	// CN servers. If proxy connects to cn timeout, it will return a retryable
	// error and try to connect to other cn servers.
	ConnectTimeout toml.Duration `toml:"connect-timeout" user_setting:"advanced"`
	// AuthTimeout is the timeout duration when proxy handshakes with backend
	// CN servers. If proxy handshakes with cn timeout, it will return a retryable
	// error and try to connect to other cn servers.
	AuthTimeout toml.Duration `toml:"auth-timeout" user_setting:"advanced"`
	// TLSConnectTimeout is the timeout duration when TLS connect to server.
	TLSConnectTimeout toml.Duration `toml:"tls-connect-timeout" user_setting:"advanced"`

	// Default is false. With true. Server will support tls.
	// This value should be ths same with all CN servers, and the name
	// of this parameter is enableTls.
	TLSEnabled bool `toml:"tls-enabled" user_setting:"advanced"`
	// TSLCAFile is the file path of file that contains list of trusted
	// SSL CAs for client.
	TLSCAFile string `toml:"tls-ca-file" user_setting:"advanced"`
	// TLSCertFile is the file path of file that contains X509 certificate
	// in PEM format for client.
	TLSCertFile string `toml:"tls-cert-file" user_setting:"advanced"`
	// TLSKeyFile is the file path of file that contains X509 key in PEM
	// format for client.
	TLSKeyFile string `toml:"tls-key-file" user_setting:"advanced"`
	// InternalCIDRs is the config which indicates that the CIDR list of
	// internal network. The addresses outside the range are external
	// addresses.
	InternalCIDRs []string `toml:"internal-cidrs"`

	// HAKeeper is the configuration of HAKeeper.
	HAKeeper struct {
		// ClientConfig is HAKeeper client configuration.
		ClientConfig logservice.HAKeeperClientConfig
		// HeartbeatInterval heartbeat interval to send message to HAKeeper. Default is 1s.
		HeartbeatInterval toml.Duration `toml:"hakeeper-heartbeat-interval"`
		// HeartbeatTimeout heartbeat request timeout. Default is 3s.
		HeartbeatTimeout toml.Duration `toml:"hakeeper-heartbeat-timeout"`
	}
	// Cluster is the configuration of MO Cluster.
	Cluster struct {
		// RefreshInterval refresh cluster info from hakeeper interval
		RefreshInterval toml.Duration `toml:"refresh-interval"`
	}
	// Plugin specifies an optional proxy plugin backend
	//
	// NB: the connection between proxy and plugin is assumed to be stable, external orchestrators
	// are responsible for ensuring the stability of rpc tunnels, for example, by deploying proxy and
	// plugin in a same machine and communicate through local loopback address
	Plugin *PluginConfig `toml:"plugin"`
}

Config is the configuration of proxy server.

func (*Config) FillDefault 

func (c *Config) FillDefault()

FillDefault fill the default config values of proxy server.

func (*Config) Validate 

func (c *Config) Validate() error

Validate validates the configuration of proxy server.

type IEvent 

type IEvent interface {
	// contains filtered or unexported methods
}

IEvent is the event interface.

type LabelHash 

type LabelHash string

LabelHash defines hash value, which is hashed from labelInfo.

type MySQLCmd 

type MySQLCmd byte

MySQLCmd is the type indicate the cmd of statement.

type MySQLConn 

type MySQLConn struct {
	net.Conn
	// contains filtered or unexported fields
}

MySQLConn contains a buffer to save data which may be only part of a packet.

type Option 

type Option func(*Server)

Option is used to set up configuration.

func WithConfigData added in v1.1.0 

func WithConfigData(data map[string]*logservicepb.ConfigItem) Option

WithConfigData saves the data from the config file

func WithRuntime 

func WithRuntime(runtime runtime.Runtime) Option

WithRuntime sets the runtime of proxy server.

func WithTLSCAFile 

func WithTLSCAFile(f string) Option

WithTLSCAFile sets the CA file.

func WithTLSCertFile 

func WithTLSCertFile(f string) Option

WithTLSCertFile sets the cert file.

func WithTLSEnabled 

func WithTLSEnabled() Option

WithTLSEnabled enable the TLS.

func WithTLSKeyFile 

func WithTLSKeyFile(f string) Option

WithTLSKeyFile sets the key file.

func WithTest added in v1.2.3 

func WithTest() Option

WithTest set the test field to true.

type Plugin 

type Plugin interface {
	// RecommendCN returns the recommended CN server.
	RecommendCN(ctx context.Context, client clientInfo) (*plugin.Recommendation, error)
}

Plugin is the interface of proxy plugin.

type PluginConfig 

type PluginConfig struct {
	// Backend is the plugin backend URL
	Backend string `toml:"backend"`
	// Timeout is the rpc timeout when communicate with the plugin backend
	Timeout time.Duration `toml:"timeout"`
}

type ProxyAddr 

type ProxyAddr struct {
	SourceAddress net.IP
	SourcePort    uint16
	TargetAddress net.IP
	TargetPort    uint16
}

ProxyAddr contains the source and target address.

type ProxyHeaderV2 

type ProxyHeaderV2 struct {
	Signature          [12]byte
	ProtocolVersionCmd uint8
	ProtocolFamily     uint8
	Length             uint16
}

ProxyHeaderV2 is the structure of the Proxy Protocol version 2 header.

type RebalancePolicy added in v1.1.2 

type RebalancePolicy int
const (
	RebalancePolicyActive RebalancePolicy = iota
	RebalancePolicyPassive
)

type RefreshableRouter added in v1.1.0 

type RefreshableRouter interface {
	Router

	Refresh(sync bool)
}

RefreshableRouter is a router that can be refreshed to get latest route strategy

type Router 

type Router interface {
	// Route selects the best CN server according to the clientInfo.
	// This is the only method that allocate *CNServer, and other
	// SelectXXX method in this interface select CNServer from the
	// ones it allocated.
	// filter is a function which is used to do more checks whether the
	// CN server is a proper one. If it returns true, means that CN
	// server is not a valid one.
	Route(ctx context.Context, client clientInfo, filter func(string) bool) (*CNServer, error)

	// SelectByConnID selects the CN server which has the connection ID.
	SelectByConnID(connID uint32) (*CNServer, error)

	// AllServers returns all CN servers. Note that the request user have to be
	// sys tenant.
	AllServers() ([]*CNServer, error)

	// Connect connects to the CN server and returns the connection.
	// It should take a handshakeResp as a parameter, which is the auth
	// request from client including tenant, username, database and others.
	Connect(c *CNServer, handshakeResp *frontend.Packet, t *tunnel) (ServerConn, []byte, error)
}

Router is an interface to select CN server and connects to it.

type SQLRouter added in v1.2.3 

type SQLRouter interface {
	// GetCNServerByConnID gets CN server by connection ID.
	// It connects to CN backend server to query data.
	GetCNServerByConnID(uint32) (*CNServer, error)
}

SQLRouter routes connections with SQL query. It executes some queries to get information about tenant, connection ID or others and returns the property CN servers.

type SQLWorker added in v1.2.3 

type SQLWorker interface {
	// SetSQLUser sets up the auth information to connect to backend servers.
	// The information comes from TaskTableUser which is in HAKeeper.
	SetSQLUser(username, password string)
	// SetAddressFn sets up the function which returns the backend server address.
	// The information comes from TaskTableUser which is in HAKeeper.
	SetAddressFn(f sqlAddressFn)
}

SQLWorker is the interface which contains methods to init the SQL worker.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

func NewServer 

func NewServer(ctx context.Context, config Config, opts ...Option) (*Server, error)

NewServer creates the proxy server.

NB: runtime must be included in opts.

func (*Server) Close 

func (s *Server) Close() error

Close closes the proxy server.

func (*Server) Start 

func (s *Server) Start() error

Start starts the proxy server.

type ServerConn 

type ServerConn interface {
	// ConnID returns connection ID of the backend CN server.
	ConnID() uint32
	// RawConn return the raw connection.
	RawConn() net.Conn
	// HandleHandshake handles the handshake communication with CN server.
	// handshakeResp is a auth packet received from client.
	HandleHandshake(handshakeResp *frontend.Packet, timeout time.Duration) (*frontend.Packet, error)
	// ExecStmt executes a simple statement, it sends a query to backend server.
	// After it finished, server connection should be closed immediately because
	// it is a temp connection.
	// The first return value indicates that if the execution result is OK.
	// NB: the stmt can only be simple stmt, which returns OK or Err only.
	ExecStmt(stmt internalStmt, resp chan<- []byte) (bool, error)
	// Close closes the connection to CN server.
	Close() error
}

ServerConn is the connection to the backend server.

type Tenant 

type Tenant string

Tenant defines alias tenant name type of string. 

var EmptyTenant Tenant = ""

EmptyTenant is an empty tenant.

type TunnelDeliver added in v1.2.1 

type TunnelDeliver interface {
	// Deliver enqueues the tunnel to the transfer queue.
	// It makes sure that if the tunnel is in the queue or being transferred,
	// the tunnel will not be enqueue into the tunnel queue.
	Deliver(*tunnel, transferType)

	// Count returns the number of tunnels in the queue.
	Count() int
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.