mysql

package module
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 30, 2018 License: MPL-2.0 Imports: 25 Imported by: 0

README

Go-MySQL-Driver

A MySQL-Driver for Go's database/sql package

Go-MySQL-Driver logo

Features

  • Lightweight and fast
  • Native Go implementation. No C-bindings, just pure Go
  • Connections over TCP/IPv4, TCP/IPv6, Unix domain sockets or custom protocols
  • Automatic handling of broken connections
  • Automatic Connection Pooling (by database/sql package)
  • Supports queries larger than 16MB
  • Full sql.RawBytes support.
  • Intelligent LONG DATA handling in prepared statements
  • Secure LOAD DATA LOCAL INFILE support with file Whitelisting and io.Reader support
  • Optional time.Time parsing
  • Optional placeholder interpolation

Requirements

  • Go 1.7 or higher. We aim to support the 3 latest versions of Go.
  • MySQL (4.1+), MariaDB, Percona Server, Google CloudSQL or Sphinx (2.2.3+)

Installation

Simple install the package to your $GOPATH with the go tool from shell:

$ go get -u github.com/go-sql-driver/mysql

Make sure Git is installed on your machine and in your system's PATH.

Usage

Go MySQL Driver is an implementation of Go's database/sql/driver interface. You only need to import the driver and can use the full database/sql API then.

Use mysql as driverName and a valid DSN as dataSourceName:

import "database/sql"
import _ "github.com/go-sql-driver/mysql"

db, err := sql.Open("mysql", "user:password@/dbname")

Examples are available in our Wiki.

DSN (Data Source Name)

The Data Source Name has a common format, like e.g. PEAR DB uses it, but without type-prefix (optional parts marked by squared brackets):

[username[:password]@][protocol[(address)]]/dbname[?param1=value1&...&paramN=valueN]

A DSN in its fullest form:

username:password@protocol(address)/dbname?param=value

Except for the databasename, all values are optional. So the minimal DSN is:

/dbname

If you do not want to preselect a database, leave dbname empty:

/

This has the same effect as an empty DSN string:

Alternatively, Config.FormatDSN can be used to create a DSN string by filling a struct.

Password

Passwords can consist of any character. Escaping is not necessary.

Protocol

See net.Dial for more information which networks are available. In general you should use an Unix domain socket if available and TCP otherwise for best performance.

Address

For TCP and UDP networks, addresses have the form host[:port]. If port is omitted, the default port will be used. If host is a literal IPv6 address, it must be enclosed in square brackets. The functions net.JoinHostPort and net.SplitHostPort manipulate addresses in this form.

For Unix domain sockets the address is the absolute path to the MySQL-Server-socket, e.g. /var/run/mysqld/mysqld.sock or /tmp/mysql.sock.

Parameters

Parameters are case-sensitive!

Notice that any of true, TRUE, True or 1 is accepted to stand for a true boolean value. Not surprisingly, false can be specified as any of: false, FALSE, False or 0.

allowAllFiles
Type:           bool
Valid Values:   true, false
Default:        false

allowAllFiles=true disables the file Whitelist for LOAD DATA LOCAL INFILE and allows all files. Might be insecure!

allowCleartextPasswords
Type:           bool
Valid Values:   true, false
Default:        false

allowCleartextPasswords=true allows using the cleartext client side plugin if required by an account, such as one defined with the PAM authentication plugin. Sending passwords in clear text may be a security problem in some configurations. To avoid problems if there is any possibility that the password would be intercepted, clients should connect to MySQL Server using a method that protects the password. Possibilities include TLS / SSL, IPsec, or a private network.

allowNativePasswords
Type:           bool
Valid Values:   true, false
Default:        true

allowNativePasswords=false disallows the usage of MySQL native password method.

allowOldPasswords
Type:           bool
Valid Values:   true, false
Default:        false

allowOldPasswords=true allows the usage of the insecure old password method. This should be avoided, but is necessary in some cases. See also the old_passwords wiki page.

charset
Type:           string
Valid Values:   <name>
Default:        none

Sets the charset used for client-server interaction ("SET NAMES <value>"). If multiple charsets are set (separated by a comma), the following charset is used if setting the charset failes. This enables for example support for utf8mb4 (introduced in MySQL 5.5.3) with fallback to utf8 for older servers (charset=utf8mb4,utf8).

Usage of the charset parameter is discouraged because it issues additional queries to the server. Unless you need the fallback behavior, please use collation instead.

collation
Type:           string
Valid Values:   <name>
Default:        utf8_general_ci

Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail.

A list of valid charsets for a server is retrievable with SHOW COLLATION.

clientFoundRows
Type:           bool
Valid Values:   true, false
Default:        false

clientFoundRows=true causes an UPDATE to return the number of matching rows instead of the number of rows changed.

columnsWithAlias
Type:           bool
Valid Values:   true, false
Default:        false

When columnsWithAlias is true, calls to sql.Rows.Columns() will return the table alias and the column name separated by a dot. For example:

SELECT u.id FROM users as u

will return u.id instead of just id if columnsWithAlias=true.

interpolateParams
Type:           bool
Valid Values:   true, false
Default:        false

If interpolateParams is true, placeholders (?) in calls to db.Query() and db.Exec() are interpolated into a single query string with given parameters. This reduces the number of roundtrips, since the driver has to prepare a statement, execute it with given parameters and close the statement again with interpolateParams=false.

This can not be used together with the multibyte encodings BIG5, CP932, GB2312, GBK or SJIS. These are blacklisted as they may introduce a SQL injection vulnerability!

loc
Type:           string
Valid Values:   <escaped name>
Default:        UTC

Sets the location for time.Time values (when using parseTime=true). "Local" sets the system's location. See time.LoadLocation for details.

Note that this sets the location for time.Time values but does not change MySQL's time_zone setting. For that see the time_zone system variable, which can also be set as a DSN parameter.

Please keep in mind, that param values must be url.QueryEscape'ed. Alternatively you can manually replace the / with %2F. For example US/Pacific would be loc=US%2FPacific.

maxAllowedPacket
Type:          decimal number
Default:       4194304

Max packet size allowed in bytes. The default value is 4 MiB and should be adjusted to match the server settings. maxAllowedPacket=0 can be used to automatically fetch the max_allowed_packet variable from server on every connection.

multiStatements
Type:           bool
Valid Values:   true, false
Default:        false

Allow multiple statements in one query. While this allows batch queries, it also greatly increases the risk of SQL injections. Only the result of the first query is returned, all other results are silently discarded.

When multiStatements is used, ? parameters must only be used in the first statement.

parseTime
Type:           bool
Valid Values:   true, false
Default:        false

parseTime=true changes the output type of DATE and DATETIME values to time.Time instead of []byte / string

readTimeout
Type:           duration
Default:        0

I/O read timeout. The value must be a decimal number with a unit suffix ("ms", "s", "m", "h"), such as "30s", "0.5m" or "1m30s".

rejectReadOnly
Type:           bool
Valid Values:   true, false
Default:        false

rejectReadOnly=true causes the driver to reject read-only connections. This is for a possible race condition during an automatic failover, where the mysql client gets connected to a read-only replica after the failover.

Note that this should be a fairly rare case, as an automatic failover normally happens when the primary is down, and the race condition shouldn't happen unless it comes back up online as soon as the failover is kicked off. On the other hand, when this happens, a MySQL application can get stuck on a read-only connection until restarted. It is however fairly easy to reproduce, for example, using a manual failover on AWS Aurora's MySQL-compatible cluster.

If you are not relying on read-only transactions to reject writes that aren't supposed to happen, setting this on some MySQL providers (such as AWS Aurora) is safer for failovers.

Note that ERROR 1290 can be returned for a read-only server and this option will cause a retry for that error. However the same error number is used for some other cases. You should ensure your application will never cause an ERROR 1290 except for read-only mode when enabling this option.

timeout
Type:           duration
Default:        OS default

Timeout for establishing connections, aka dial timeout. The value must be a decimal number with a unit suffix ("ms", "s", "m", "h"), such as "30s", "0.5m" or "1m30s".

tls
Type:           bool / string
Valid Values:   true, false, skip-verify, <name>
Default:        false

tls=true enables TLS / SSL encrypted connection to the server. Use skip-verify if you want to use a self-signed or invalid certificate (server side). Use a custom value registered with mysql.RegisterTLSConfig.

writeTimeout
Type:           duration
Default:        0

I/O write timeout. The value must be a decimal number with a unit suffix ("ms", "s", "m", "h"), such as "30s", "0.5m" or "1m30s".

maxRetry
Type:           int
Default:        0

Maximum number a request should be retried when an error happens. It is the implementation of retry-strategy.

enableCircuitBreaker
Type:           bool
Valid Values:   true, false
Default:        false

enableCircuitBreaker=true enables circuit breaker. Any success or failure will be recorded. When the error percentage threshold is reached, the circuit will be in open state. At that moment, any request will be ignored so if something happens with the database, the client will not fail. It is hopefully be able to increase system resiliency. The error percentage threshold is 10% with volume threshold about 1.000 requests. Circuit breaker also has timeout. Its timeout is the same dial timeout and uses 3s as timeout if not configured.

connectionName
Type:           string
Valid Values:   <escaped name>

connectionName is used to identify connection. Developers are strongly recommended to specify their connection. It will be very useful to identify connection metric.

System Variables

Any other parameters are interpreted as system variables:

  • <boolean_var>=<value>: SET <boolean_var>=<value>
  • <enum_var>=<value>: SET <enum_var>=<value>
  • <string_var>=%27<value>%27: SET <string_var>='<value>'

Rules:

  • The values for string variables must be quoted with '.
  • The values must also be url.QueryEscape'ed! (which implies values of string variables must be wrapped with %27).

Examples:

Examples
user@unix(/path/to/socket)/dbname

root:pw@unix(/tmp/mysql.sock)/myDatabase?loc=Local

user:password@tcp(localhost:5555)/dbname?tls=skip-verify&autocommit=true

user:password@tcp(localhost:5555)/dbname?maxRetry=3

user:password@tcp(localhost:5555)/dbname?maxRetry=3&connectionName=espeon&enableCircuitBreaker=true

Treat warnings as errors by setting the system variable sql_mode:

user:password@/dbname?sql_mode=TRADITIONAL

TCP via IPv6:

user:password@tcp([de:ad:be:ef::ca:fe]:80)/dbname?timeout=90s&collation=utf8mb4_unicode_ci

TCP on a remote host, e.g. Amazon RDS:

id:password@tcp(your-amazonaws-uri.com:3306)/dbname

Google Cloud SQL on App Engine (First Generation MySQL Server):

user@cloudsql(project-id:instance-name)/dbname

Google Cloud SQL on App Engine (Second Generation MySQL Server):

user@cloudsql(project-id:regionname:instance-name)/dbname

TCP using default port (3306) on localhost:

user:password@tcp/dbname?charset=utf8mb4,utf8&sys_var=esc%40ped

Use the default protocol (tcp) and host (localhost:3306):

user:password@/dbname

No Database preselected:

user:password@/

There are many parameters that can be used in DSN. From all of them, for resiliency purpose, we recommend to specify the following parameters:

  • timeout=<your-timeout>
  • enableCircuitBreaker=true
  • connectionName=<your-connection-name>
  • maxRetry=<your-max-retry>

Developers are strongly recommended to enable circuit breaker and set max retry value, though, maxRetry=0 should be enough.

Connection pool and timeouts

The connection pool is managed by Go's database/sql package. For details on how to configure the size of the pool and how long connections stay in the pool see *DB.SetMaxOpenConns, *DB.SetMaxIdleConns, and *DB.SetConnMaxLifetime in the database/sql documentation. The read, write, and dial timeouts for each individual connection are configured with the DSN parameters readTimeout, writeTimeout, and timeout, respectively.

ColumnType Support

This driver supports the ColumnType interface introduced in Go 1.8, with the exception of ColumnType.Length(), which is currently not supported.

context.Context Support

Go 1.8 added database/sql support for context.Context. This driver supports query timeouts and cancellation via contexts. See context support in the database/sql package for more details.

LOAD DATA LOCAL INFILE support

For this feature you need direct access to the package. Therefore you must change the import path (no _):

import "github.com/go-sql-driver/mysql"

Files must be whitelisted by registering them with mysql.RegisterLocalFile(filepath) (recommended) or the Whitelist check must be deactivated by using the DSN parameter allowAllFiles=true (Might be insecure!).

To use a io.Reader a handler function must be registered with mysql.RegisterReaderHandler(name, handler) which returns a io.Reader or io.ReadCloser. The Reader is available with the filepath Reader::<name> then. Choose different names for different handlers and DeregisterReaderHandler when you don't need it anymore.

See the godoc of Go-MySQL-Driver for details.

time.Time support

The default internal output type of MySQL DATE and DATETIME values is []byte which allows you to scan the value into a []byte, string or sql.RawBytes variable in your program.

However, many want to scan MySQL DATE and DATETIME values into time.Time variables, which is the logical opposite in Go to DATE and DATETIME in MySQL. You can do that by changing the internal output type from []byte to time.Time with the DSN parameter parseTime=true. You can set the default time.Time location with the loc DSN parameter.

Caution: As of Go 1.1, this makes time.Time the only variable type you can scan DATE and DATETIME values into. This breaks for example sql.RawBytes support.

Alternatively you can use the NullTime type as the scan destination, which works with both time.Time and string / []byte.

Unicode support

Since version 1.1 Go-MySQL-Driver automatically uses the collation utf8_general_ci by default.

Other collations / charsets can be set using the collation DSN parameter.

Version 1.0 of the driver recommended adding &charset=utf8 (alias for SET NAMES utf8) to the DSN to enable proper UTF-8 support. This is not necessary anymore. The collation parameter should be preferred to set another collation / charset than the default.

See http://dev.mysql.com/doc/refman/5.7/en/charset-unicode.html for more details on MySQL's Unicode support.

Testing / Development

To run the driver tests you may need to adjust the configuration. See the Testing Wiki-Page for details.

Go-MySQL-Driver is not feature-complete yet. Your help is very appreciated. If you want to contribute, you can work on an open issue or review a pull request.

See the Contribution Guidelines for details.

License

Go-MySQL-Driver is licensed under the Mozilla Public License Version 2.0

Mozilla summarizes the license scope as follows:

MPL: The copyleft applies to any files containing MPLed code.

That means:

  • You can use the unchanged source code both in private and commercially.
  • When distributing, you must publish the source code of any changed files licensed under the MPL 2.0 under a) the MPL 2.0 itself or b) a compatible license (e.g. GPL 3.0 or Apache License 2.0).
  • You needn't publish the source code of your library as long as the files licensed under the MPL 2.0 are unchanged.

Please read the MPL 2.0 FAQ if you have further questions regarding the license.

You can read the full terms here: LICENSE.

Go Gopher and MySQL Dolphin

Documentation

Overview

Package mysql provides a MySQL driver for Go's database/sql package.

The driver should be used via the database/sql package: 

import "database/sql"
import _ "github.com/go-sql-driver/mysql"

db, err := sql.Open("mysql", "user:password@/dbname")

See https://github.com/go-sql-driver/mysql#usage for details

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrInvalidConn       = errors.New("invalid connection")
	ErrMalformPkt        = errors.New("malformed packet")
	ErrNoTLS             = errors.New("TLS requested but server does not support TLS")
	ErrCleartextPassword = errors.New("this user requires clear text authentication. If you still want to use it, please add 'allowCleartextPasswords=1' to your DSN")
	ErrNativePassword    = errors.New("this user requires mysql native password authentication.")
	ErrOldPassword       = errors.New("this user requires old password authentication. If you still want to use it, please add 'allowOldPasswords=1' to your DSN. See also https://github.com/go-sql-driver/mysql/wiki/old_passwords")
	ErrUnknownPlugin     = errors.New("this authentication plugin is not supported")
	ErrOldProtocol       = errors.New("MySQL server does not support required protocol 41+")
	ErrPktSync           = errors.New("commands out of sync. You can't run this command now")
	ErrPktSyncMul        = errors.New("commands out of sync. Did you run multiple statements at once?")
	ErrPktTooLarge       = errors.New("packet for query is too large. Try adjusting the 'max_allowed_packet' variable on the server")
	ErrBusyBuffer        = errors.New("busy buffer")
)

Various errors the driver might return. Can change between driver versions.

Functions

func CounterVec added in v1.4.0 

func CounterVec() *prometheus.CounterVec

CounterVec returns prometheus.CounterVec. It is used by mysql driver to count how many requests fall for earch circuit breaker state. Developers can export this counter to their project so they can monitor circuit breaker usage.

func DeregisterLocalFile 

func DeregisterLocalFile(filePath string)

DeregisterLocalFile removes the given filepath from the whitelist.

func DeregisterReaderHandler 

func DeregisterReaderHandler(name string)

DeregisterReaderHandler removes the ReaderHandler function with the given name from the registry.

func DeregisterTLSConfig added in v1.4.0 

func DeregisterTLSConfig(key string)

DeregisterTLSConfig removes the tls.Config associated with key.

func RegisterDial added in v1.4.0 

func RegisterDial(net string, dial DialFunc)

RegisterDial registers a custom dial function. It can then be used by the network address mynet(addr), where mynet is the registered new network. addr is passed as a parameter to the dial function.

func RegisterLocalFile 

func RegisterLocalFile(filePath string)

RegisterLocalFile adds the given file to the file whitelist, so that it can be used by "LOAD DATA LOCAL INFILE <filepath>". Alternatively you can allow the use of all local files with the DSN parameter 'allowAllFiles=true' 

filePath := "/home/gopher/data.csv"
mysql.RegisterLocalFile(filePath)
err := db.Exec("LOAD DATA LOCAL INFILE '" + filePath + "' INTO TABLE foo")
if err != nil {
...

func RegisterReaderHandler 

func RegisterReaderHandler(name string, handler func() io.Reader)

RegisterReaderHandler registers a handler function which is used to receive a io.Reader. The Reader can be used by "LOAD DATA LOCAL INFILE Reader::<name>". If the handler returns a io.ReadCloser Close() is called when the request is finished. 

mysql.RegisterReaderHandler("data", func() io.Reader {
	var csvReader io.Reader // Some Reader that returns CSV data
	... // Open Reader here
	return csvReader
})
err := db.Exec("LOAD DATA LOCAL INFILE 'Reader::data' INTO TABLE foo")
if err != nil {
...

func RegisterTLSConfig added in v1.4.0 

func RegisterTLSConfig(key string, config *tls.Config) error

RegisterTLSConfig registers a custom tls.Config to be used with sql.Open. Use the key as a value in the DSN where tls=value.

Note: The tls.Config provided to needs to be exclusively owned by the driver after registering. 

rootCertPool := x509.NewCertPool()
pem, err := ioutil.ReadFile("/path/ca-cert.pem")
if err != nil {
    log.Fatal(err)
}
if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
    log.Fatal("Failed to append PEM.")
}
clientCert := make([]tls.Certificate, 0, 1)
certs, err := tls.LoadX509KeyPair("/path/client-cert.pem", "/path/client-key.pem")
if err != nil {
    log.Fatal(err)
}
clientCert = append(clientCert, certs)
mysql.RegisterTLSConfig("custom", &tls.Config{
    RootCAs: rootCertPool,
    Certificates: clientCert,
})
db, err := sql.Open("mysql", "user@tcp(localhost:3306)/test?tls=custom")

func SetLogger added in v1.4.0 

func SetLogger(logger Logger) error

SetLogger is used to set the logger for critical errors. The initial logger is os.Stderr.

Types

type Config added in v1.4.0 

type Config struct {
	User             string            // Username
	Passwd           string            // Password (requires User)
	Net              string            // Network type
	Addr             string            // Network address (requires Net)
	DBName           string            // Database name
	Params           map[string]string // Connection parameters
	Collation        string            // Connection collation
	Loc              *time.Location    // Location for time.Time values
	MaxAllowedPacket int               // Max packet size allowed
	TLSConfig        string            // TLS configuration name

	Timeout      time.Duration // Dial timeout
	ReadTimeout  time.Duration // I/O read timeout
	WriteTimeout time.Duration // I/O write timeout

	AllowAllFiles           bool // Allow all files to be used with LOAD DATA LOCAL INFILE
	AllowCleartextPasswords bool // Allows the cleartext client side plugin
	AllowNativePasswords    bool // Allows the native password authentication method
	AllowOldPasswords       bool // Allows the old insecure password method
	ClientFoundRows         bool // Return number of matching rows instead of rows changed
	ColumnsWithAlias        bool // Prepend table alias to column names
	InterpolateParams       bool // Interpolate placeholders into query string
	MultiStatements         bool // Allow multiple statements in one query
	ParseTime               bool // Parse time values to time.Time
	RejectReadOnly          bool // Reject read-only connections

	// Additional Usage
	MaxRetry             int        // Max number of retry
	Intervaler           intervaler // Backoff strategy to be used
	EnableCircuitBreaker bool       // Enable circuit breaker strategy
	ConnectionName       string     // Connection's name to identify specific connection
	// contains filtered or unexported fields
}

Config is a configuration parsed from a DSN string. If a new Config is created instead of being parsed from a DSN string, the NewConfig function should be used, which sets default values.

func NewConfig added in v1.4.0 

func NewConfig() *Config

NewConfig creates a new Config and sets default values.

func ParseDSN added in v1.4.0 

func ParseDSN(dsn string) (cfg *Config, err error)

ParseDSN parses the DSN string to a Config

func (*Config) FormatDSN added in v1.4.0 

func (cfg *Config) FormatDSN() string

FormatDSN formats the given Config into a DSN string which can be passed to the driver.

type DialFunc added in v1.4.0 

type DialFunc func(addr string) (net.Conn, error)

DialFunc is a function which can be used to establish the network connection. Custom dial functions must be registered with RegisterDial

type Logger added in v1.4.0 

type Logger interface {
	Print(v ...interface{})
}

Logger is used to log critical error messages.

type MySQLDriver added in v1.4.0 

type MySQLDriver struct{}

MySQLDriver is exported to make the driver directly accessible. In general the driver is used via the database/sql package.

func (MySQLDriver) Open added in v1.4.0 

func (d MySQLDriver) Open(dsn string) (driver.Conn, error)

Open new Connection. See https://github.com/go-sql-driver/mysql#dsn-data-source-name for how the DSN string is formated

type MySQLError 

type MySQLError struct {
	Number  uint16
	Message string
}

MySQLError is an error type which represents a single MySQL error

func (*MySQLError) Error 

func (me *MySQLError) Error() string

type NullTime 

type NullTime struct {
	Time  time.Time
	Valid bool // Valid is true if Time is not NULL
}

NullTime represents a time.Time that may be NULL. NullTime implements the Scanner interface so it can be used as a scan destination: 

var nt NullTime
err := db.QueryRow("SELECT time FROM foo WHERE id=?", id).Scan(&nt)
...
if nt.Valid {
   // use nt.Time
} else {
   // NULL value
}

This NullTime implementation is not driver-specific

func (*NullTime) Scan 

func (nt *NullTime) Scan(value interface{}) (err error)

Scan implements the Scanner interface. The value type must be time.Time or string / []byte (formatted time-string), otherwise Scan fails.

func (NullTime) Value 

func (nt NullTime) Value() (driver.Value, error)

Value implements the driver Valuer interface.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.