dump

Documentation

Overview

Package dump 用于dump协议交互的， 从github.com/go-sql-driver/mysql的基础上修改而来，主要功能如下： 1.通过MysqlConn可以执行简单的sql命令，如set命令， 2.通过MysqlConn来和mysql库进行binlog dump

github.com/go-sql-driver/mysql已经支持了所有的协议包的读写，但是 由于以下原因需要修改：1.该包不支持dump协议的交互。2.在解析mysql网络 数据重复使用了同一个内存块，由于Package replication中每一个binlogEvent 需要独立缓存一个内存块，导致binlogEvent会读取脏数据。

目前dump已经支持单条连接完成dump协议的交互，取消了重复使用内存机制， 使其在mysql网络数据时使用独立缓存，从而保障了与Package replication 不再读到脏数据。

DSN参数说明：

[username[:password]@][protocol[(address)]]/dbname[?param1=value1&...&paramN=valueN] username是mysql的用户名，password是mysql的密码，protocol协议一般是tcp,address是mysql的地址， dbname是mysql的数据库名

常用参数说明：

charset

Type: string Valid Values: <name> Default: none Sets the charset used for client-server interaction ("SET NAMES <value>"). If multiple charsets are set (separated by a comma), the following charset is used if setting the charset failes. This enables for example support for utf8mb4 (introduced in MySQL 5.5.3) with fallback to utf8 for older servers (charset=utf8mb4,utf8).

Usage of the charset parameter is discouraged because it issues additional queries to the server. Unless you need the fallback behavior, please use collation instead.

collation

Type: string Valid Values: <name> Default: utf8mb4_general_ci Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail.

A list of valid charsets for a server is retrievable with SHOW COLLATION.

The default collation (utf8mb4_general_ci) is supported from MySQL 5.5. You should use an older collation (e.g. utf8_general_ci) for older MySQL.

Collations for charset "ucs2", "utf16", "utf16le", and "utf32" can not be used (ref).

loc

Type: string Valid Values: <escaped name> Default: UTC Sets the location for time.Time values (when using parseTime=true). "Local" sets the system's location. See time.LoadLocation for details.

Note that this sets the location for time.Time values but does not change MySQL's time_zone setting. For that see the time_zone system variable, which can also be set as a DSN parameter.

Please keep in mind, that param values must be url.QueryEscape'ed. Alternatively you can manually replace the / with %2F. For example US/Pacific would be loc=US%2FPacific.

maxAllowedPacket

Type: decimal number Default: 4194304 Max packet size allowed in bytes. The default value is 4 MiB and should be adjusted to match the server settings. maxAllowedPacket=0 can be used to automatically fetch the max_allowed_packet variable from server on every connection.

parseTime

Type: bool Valid Values: true, false Default: false parseTime=true changes the output type of DATE and DATETIME values to time.Time instead of []byte / string The date or datetime like 0000-00-00 00:00:00 is converted into zero value of time.Time.

readTimeout

Type: duration Default: 0 I/O read timeout. The value must be a decimal number with a unit suffix ("ms", "s", "m", "h"), such as "30s", "0.5m" or "1m30s".

timeout

Type: duration Default: OS default Timeout for establishing connections, aka dial timeout. The value must be a decimal number with a unit suffix ("ms", "s", "m", "h"), such as "30s", "0.5m" or "1m30s".

writeTimeout

Type: duration Default: 0 I/O write timeout. The value must be a decimal number with a unit suffix ("ms", "s", "m", "h"), such as "30s", "0.5m" or "1m30s".

Index

• Constants
• Variables
• func DeregisterTLSConfig(key string)
• func RegisterTLSConfig(key string, config *tls.Config) error
• func SetLogger(logger Logger) error
• type Config
  • func ParseDSN(dsn string) (cfg *Config, err error)
  • func (cfg *Config) FormatDSN() string
• type Logger
• type MyRows
• type MySQLError
  • func (me *MySQLError) Error() string
• type MySQLWarning
• type MySQLWarnings
  • func (mws MySQLWarnings) Error() string
• type MysqlConn
  • func NewMysqlConn(dsn string) (*MysqlConn, error)
  • func (mc *MysqlConn) Close() (err error)
  • func (mc *MysqlConn) Exec(query string) error
  • func (mc *MysqlConn) HandleErrorPacket(data []byte) error
  • func (mc *MysqlConn) NoticeDump(serverID, offset uint32, filename string, flags uint16) error
  • func (mc *MysqlConn) ReadPacket() ([]byte, error)
• type NullTime
  • func (nt *NullTime) Scan(value interface{}) (err error)
  • func (nt NullTime) Value() (driver.Value, error)

Constants

const (
	PacketOK  = iOK  //用于标识mysql反馈包是正确信息
	PacketEOF = iEOF //用于标识mysql反馈包是结束信息
	PacketERR = iERR //用于标识mysql反馈包是错误信息
)

mysql反馈包类型

Variables

var (
	ErrInvalidConn       = errors.New("invalid connection")
	ErrMalformPkt        = errors.New("malformed packet")
	ErrNoTLS             = errors.New("TLS requested but server does not support TLS")
	ErrCleartextPassword = errors.New("this user requires clear text authentication. If you still want to use it, please add 'allowCleartextPasswords=1' to your DSN")
	ErrNativePassword    = errors.New("this user requires mysql native password authentication")
	ErrOldPassword       = errors.New("this user requires old password authentication. If you still want to use it, please add 'allowOldPasswords=1' to your DSN. See also https://github.com/go-sql-driver/mysql/wiki/old_passwords")
	ErrUnknownPlugin     = errors.New("this authentication plugin is not supported")
	ErrOldProtocol       = errors.New("MySQL server does not support required protocol 41+")
	ErrPktSync           = errors.New("commands out of sync. You can't run this command now")
	ErrPktSyncMul        = errors.New("commands out of sync. Did you run multiple statements at once")
	ErrPktTooLarge       = errors.New("packet for query is too large. Try adjusting the 'max_allowed_packet' variable on the server")
	ErrBusyBuffer        = errors.New("busy buffer")
	ErrBadConn           = errors.New("bad connection")
)

Various errors the driver might return. Can change between driver versions.

Functions

func DeregisterTLSConfig

func DeregisterTLSConfig(key string)

DeregisterTLSConfig removes the tls.Config associated with key.

func RegisterTLSConfig

func RegisterTLSConfig(key string, config *tls.Config) error

RegisterTLSConfig registers a custom tls.Config to be used with sql.Open. Use the key as a value in the DSN where tls=value.

rootCertPool := x509.NewCertPool()
pem, err := ioutil.ReadFile("/path/ca-cert.pem")
if err != nil {
    log.Fatal(err)
}
if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
    log.Fatal("Failed to append PEM.")
}
clientCert := make([]tls.Certificate, 0, 1)
certs, err := tls.LoadX509KeyPair("/path/client-cert.pem", "/path/client-key.pem")
if err != nil {
    log.Fatal(err)
}
clientCert = append(clientCert, certs)
mysql.RegisterTLSConfig("custom", &tls.Config{
    RootCAs: rootCertPool,
    Certificates: clientCert,
})
db, err := sql.Open("mysql", "user@tcp(localhost:3306)/test?tls=custom")

func SetLogger

func SetLogger(logger Logger) error

SetLogger is used to set the logger for critical errors. The initial logger is os.Stderr.

Types

type Config

type Config struct {
	User             string            // Username
	Passwd           string            // Password (requires User)
	Net              string            // Network type
	Addr             string            // Network address (requires Net)
	DBName           string            // Database name
	Params           map[string]string // Connection parameters
	Collation        string            // Connection collation
	Loc              *time.Location    // Location for time.Time values
	MaxAllowedPacket int               // Max packet size allowed
	TLSConfig        string            // TLS configuration name

	Timeout      time.Duration // Dial timeout
	ReadTimeout  time.Duration // I/O read timeout
	WriteTimeout time.Duration // I/O write timeout

	AllowAllFiles           bool // Allow all files to be used with LOAD DATA LOCAL INFILE
	AllowCleartextPasswords bool // Allows the cleartext client side plugin
	AllowNativePasswords    bool // Allows the native password authentication method
	AllowOldPasswords       bool // Allows the old insecure password method
	ClientFoundRows         bool // Return number of matching rows instead of rows changed
	ColumnsWithAlias        bool // Prepend table alias to column names
	InterpolateParams       bool // Interpolate placeholders into query string
	MultiStatements         bool // Allow multiple statements in one query
	ParseTime               bool // Parse time values to time.Time
	Strict                  bool // Return warnings as errors
	// contains filtered or unexported fields
}

Config is a configuration parsed from a DSN string

func ParseDSN

func ParseDSN(dsn string) (cfg *Config, err error)

ParseDSN parses the DSN string to a Config

func (*Config) FormatDSN

func (cfg *Config) FormatDSN() string

FormatDSN formats the given Config into a DSN string which can be passed to the driver.

type Logger

type Logger interface {
	Print(v ...interface{})
}

Logger is used to log critical error messages.

type MyRows

type MyRows interface {
	Columns() []string
	Close() error
	Next([]interface{}) error
}

MyRows 用于实现Rows

type MySQLError

type MySQLError struct {
	Number  uint16
	Message string
}

MySQLError is an error type which represents a single MySQL error

func (*MySQLError) Error

func (me *MySQLError) Error() string

type MySQLWarning

type MySQLWarning struct {
	Level   string
	Code    string
	Message string
}

MySQLWarning is an error type which represents a single MySQL warning. Warnings are returned in groups only. See MySQLWarnings

type MySQLWarnings

type MySQLWarnings []MySQLWarning

MySQLWarnings is an error type which represents a group of one or more MySQL warnings

func (MySQLWarnings) Error

func (mws MySQLWarnings) Error() string

type MysqlConn

type MysqlConn struct {
	// contains filtered or unexported fields
}

MysqlConn mysql连接，用于执行dump和其他命令

func NewMysqlConn

func NewMysqlConn(dsn string) (*MysqlConn, error)

NewMysqlConn dsn是数据库连接信息

func (*MysqlConn) Close

func (mc *MysqlConn) Close() (err error)

Close 用于关闭连接和清理连接信息

func (*MysqlConn) Exec

func (mc *MysqlConn) Exec(query string) error

Exec Internal function to execute commands

func (*MysqlConn) HandleErrorPacket

func (mc *MysqlConn) HandleErrorPacket(data []byte) error

HandleErrorPacket 处理mysql返回的错误

func (*MysqlConn) NoticeDump

func (mc *MysqlConn) NoticeDump(serverID, offset uint32, filename string, flags uint16) error

NoticeDump 通知开始从哪个binlog位置开始以serverID为编号开始同步数据

func (*MysqlConn) ReadPacket

func (mc *MysqlConn) ReadPacket() ([]byte, error)

ReadPacket 读取mysql协议包

type NullTime

type NullTime struct {
	Time  time.Time
	Valid bool // Valid is true if Time is not NULL
}

NullTime represents a time.Time that may be NULL. NullTime implements the Scanner interface so it can be used as a scan destination:

var nt NullTime
err := db.QueryRow("SELECT time FROM foo WHERE id=?", id).Scan(&nt)
...
if nt.Valid {
   // use nt.Time
} else {
   // NULL value
}

This NullTime implementation is not driver-specific

func (*NullTime) Scan

func (nt *NullTime) Scan(value interface{}) (err error)

Scan implements the Scanner interface. The value type must be time.Time or string / []byte (formatted time-string), otherwise Scan fails.

func (NullTime) Value

func (nt NullTime) Value() (driver.Value, error)

Value implements the driver Valuer interface.