mysqlctl

package
v2.1.1+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 22, 2017 License: BSD-3-Clause Imports: 48 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// SQLStartSlave is the SQl command issued to start MySQL replication
	SQLStartSlave = "START SLAVE"

	// SQLStopSlave is the SQl command issued to stop MySQL replication
	SQLStopSlave = "STOP SLAVE"
)

Variables

View Source 
var (
	// ErrNoBackup is returned when there is no backup.
	ErrNoBackup = errors.New("no available backup")

	// ErrExistingDB is returned when there's already an active DB.
	ErrExistingDB = errors.New("skipping restore due to existing database")
)
View Source 
var (
	// ErrNotSlave means there is no slave status
	ErrNotSlave = errors.New("no slave status")
	// ErrNotMaster means there is no master status
	ErrNotMaster = errors.New("no master status")
)
View Source 
var (
	// ErrBinlogUnavailable is returned by this library when we
	// cannot find a suitable binlog to satisfy the request.
	ErrBinlogUnavailable = fmt.Errorf("cannot find relevant binlogs on this server")
)

Functions

func Backup 

func Backup(ctx context.Context, mysqld MysqlDaemon, logger logutil.Logger, dir, name string, backupConcurrency int, hookExtraEnv map[string]string) error

Backup is the main entry point for a backup: - uses the BackupStorage service to store a new backup - shuts down Mysqld during the backup - remember if we were replicating, restore the exact same state

func ConcurrentMap 

func ConcurrentMap(concurrency, n int, fun MapFunc) error

ConcurrentMap applies fun in a concurrent manner on integers from 0 to n-1 (they are assumed to be indexes of some slice containing items to be processed). The first error returned by a fun application will returned (subsequent errors will only be logged). It will use concurrency goroutines.

func CreateReparentJournal 

func CreateReparentJournal() []string

CreateReparentJournal returns the commands to execute to create the _vt.reparent_journal table. It is safe to run these commands even if the table already exists.

If the table was created by Vitess version 2.0, the following command may need to be run: ALTER TABLE _vt.reparent_journal MODIFY COLUMN replication_position VARBINARY(64000);

func FindSlaves 

func FindSlaves(mysqld MysqlDaemon) ([]string, error)

FindSlaves gets IP addresses for all currently connected slaves.

func GetPermissions 

func GetPermissions(mysqld MysqlDaemon) (*tabletmanagerdatapb.Permissions, error)

GetPermissions lists the permissions on the mysqld

func MycnfFile 

func MycnfFile(uid uint32) string

MycnfFile returns the default location of the my.cnf file.

func PopulateReparentJournal 

func PopulateReparentJournal(timeCreatedNS int64, actionName, masterAlias string, pos replication.Position) string

PopulateReparentJournal returns the SQL command to use to populate the _vt.reparent_journal table, as well as the time_created_ns value used.

func RegisterFlags 

func RegisterFlags()

RegisterFlags registers the command line flags for specifying the values of a mycnf config file. See NewMycnfFromFlags to get the supported modes.

func ResolveTables 

func ResolveTables(mysqld MysqlDaemon, dbName string, tables []string) ([]string, error)

ResolveTables returns a list of actual tables+views matching a list of regexps

func Restore 

func Restore(
	ctx context.Context,
	mysqld MysqlDaemon,
	dir string,
	restoreConcurrency int,
	hookExtraEnv map[string]string,
	localMetadata map[string]string,
	logger logutil.Logger,
	deleteBeforeRestore bool,
	dbName string) (replication.Position, error)

Restore is the main entry point for backup restore. If there is no appropriate backup on the BackupStorage, Restore logs an error and returns ErrNoBackup. Any other error is returned.

func StartSlave 

func StartSlave(md MysqlDaemon, hookExtraEnv map[string]string) error

StartSlave starts a slave on the provided MysqldDaemon

func StatusToProto 

func StatusToProto(r Status) *replicationdatapb.Status

StatusToProto translates a Status to proto3

func StopSlave 

func StopSlave(md MysqlDaemon, hookExtraEnv map[string]string) error

StopSlave stops a slave on the provided MysqldDaemon

func TabletDir 

func TabletDir(uid uint32) string

TabletDir returns the default directory for a tablet

func TopLevelDirs 

func TopLevelDirs() []string

TopLevelDirs returns the list of directories in the toplevel tablet directory that might be located in a different place.

func WaitBlpPosition 

func WaitBlpPosition(ctx context.Context, mysqld MysqlDaemon, sql string, replicationPosition string) error

WaitBlpPosition will wait for the filtered replication to reach at least the provided position.

func WaitForSlaveStart 

func WaitForSlaveStart(mysqld MysqlDaemon, slaveStartDeadline int) error

WaitForSlaveStart waits until the deadline for replication to start. This validates the current master is correct and can be connected to.

Types

type BackupManifest 

type BackupManifest struct {
	// FileEntries contains all the files in the backup
	FileEntries []FileEntry

	// Position is the position at which the backup was taken
	Position replication.Position

	// TransformHook that was used on the files, if any.
	TransformHook string

	// SkipCompress can be set if the backup files were not run
	// through gzip. It is the negative of the flag, so old
	// backups that don't have this flag are assumed to be
	// compressed.
	SkipCompress bool
}

BackupManifest represents the backup. It lists all the files, the Position that the backup was taken at, and the transform hook used, if any.

type FakeMysqlDaemon 

type FakeMysqlDaemon struct {

	// Mycnf will be returned by Cnf()
	Mycnf *Mycnf

	// Running is used by Start / Shutdown
	Running bool

	// MysqlPort will be returned by GetMysqlPort(). Set to -1 to
	// return an error.
	MysqlPort int32

	// Replicating is updated when calling StartSlave / StopSlave
	// (it is not used at all when calling SlaveStatus, it is the
	// test owner responsability to have these two match)
	Replicating bool

	// ResetReplicationResult is returned by ResetReplication
	ResetReplicationResult []string

	// ResetReplicationError is returned by ResetReplication
	ResetReplicationError error

	// CurrentMasterPosition is returned by MasterPosition
	// and SlaveStatus
	CurrentMasterPosition replication.Position

	// SlaveStatusError is used by SlaveStatus
	SlaveStatusError error

	// CurrentMasterHost is returned by SlaveStatus
	CurrentMasterHost string

	// CurrentMasterport is returned by SlaveStatus
	CurrentMasterPort int

	// SecondsBehindMaster is returned by SlaveStatus
	SecondsBehindMaster uint

	// ReadOnly is the current value of the flag
	ReadOnly bool

	// SetSlavePositionCommandsPos is matched against the input
	// of SetSlavePositionCommands. If it doesn't match,
	// SetSlavePositionCommands will return an error.
	SetSlavePositionCommandsPos replication.Position

	// SetSlavePositionCommandsResult is what
	// SetSlavePositionCommands will return
	SetSlavePositionCommandsResult []string

	// SetMasterCommandsInput is matched against the input
	// of SetMasterCommands (as "%v:%v"). If it doesn't match,
	// SetMasterCommands will return an error.
	SetMasterCommandsInput string

	// SetMasterCommandsResult is what
	// SetMasterCommands will return
	SetMasterCommandsResult []string

	// DemoteMasterPosition is returned by DemoteMaster
	DemoteMasterPosition replication.Position

	// WaitMasterPosition is checked by WaitMasterPos, if the
	// same it returns nil, if different it returns an error
	WaitMasterPosition replication.Position

	// PromoteSlaveResult is returned by PromoteSlave
	PromoteSlaveResult replication.Position

	// SchemaFunc provides the return value for GetSchema.
	// If not defined, the "Schema" field will be used instead, see below.
	SchemaFunc func() (*tabletmanagerdatapb.SchemaDefinition, error)

	// Schema will be returned by GetSchema. If nil we'll
	// return an error.
	Schema *tabletmanagerdatapb.SchemaDefinition

	// PreflightSchemaChangeResult will be returned by PreflightSchemaChange.
	// If nil we'll return an error.
	PreflightSchemaChangeResult []*tabletmanagerdatapb.SchemaChangeResult

	// ApplySchemaChangeResult will be returned by ApplySchemaChange.
	// If nil we'll return an error.
	ApplySchemaChangeResult *tabletmanagerdatapb.SchemaChangeResult

	// DbAppConnectionFactory is the factory for making fake db app connections
	DbAppConnectionFactory func() (dbconnpool.PoolConnection, error)

	// ExpectedExecuteSuperQueryList is what we expect
	// ExecuteSuperQueryList to be called with. If it doesn't
	// match, ExecuteSuperQueryList will return an error.
	// Note each string is just a substring if it begins with SUB,
	// so we support partial queries (usefull when queries contain
	// data fields like timestamps)
	ExpectedExecuteSuperQueryList []string

	// ExpectedExecuteSuperQueryCurrent is the current index of the queries
	// we expect
	ExpectedExecuteSuperQueryCurrent int

	// FetchSuperQueryResults is used by FetchSuperQuery
	FetchSuperQueryMap map[string]*sqltypes.Result

	// BinlogPlayerEnabled is used by {Enable,Disable}BinlogPlayer
	BinlogPlayerEnabled bool

	// SemiSyncMasterEnabled represents the state of rpl_semi_sync_master_enabled.
	SemiSyncMasterEnabled bool
	// SemiSyncSlaveEnabled represents the state of rpl_semi_sync_slave_enabled.
	SemiSyncSlaveEnabled bool
	// contains filtered or unexported fields
}

FakeMysqlDaemon implements MysqlDaemon and allows the user to fake everything.

func NewFakeMysqlDaemon 

func NewFakeMysqlDaemon(db *fakesqldb.DB) *FakeMysqlDaemon

NewFakeMysqlDaemon returns a FakeMysqlDaemon where mysqld appears to be running, based on a fakesqldb.DB. 'db' can be nil if the test doesn't use a database at all.

func (*FakeMysqlDaemon) ApplySchemaChange 

func (fmd *FakeMysqlDaemon) ApplySchemaChange(dbName string, change *tmutils.SchemaChange) (*tabletmanagerdatapb.SchemaChangeResult, error)

ApplySchemaChange is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) CheckSuperQueryList 

func (fmd *FakeMysqlDaemon) CheckSuperQueryList() error

CheckSuperQueryList returns an error if all the queries we expected haven't been seen.

func (*FakeMysqlDaemon) Close 

func (fmd *FakeMysqlDaemon) Close()

Close is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) Cnf 

func (fmd *FakeMysqlDaemon) Cnf() *Mycnf

Cnf is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) DemoteMaster 

func (fmd *FakeMysqlDaemon) DemoteMaster() (replication.Position, error)

DemoteMaster is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) DisableBinlogPlayback 

func (fmd *FakeMysqlDaemon) DisableBinlogPlayback() error

DisableBinlogPlayback disable playback of binlog events

func (*FakeMysqlDaemon) EnableBinlogPlayback 

func (fmd *FakeMysqlDaemon) EnableBinlogPlayback() error

EnableBinlogPlayback is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) ExecuteSuperQueryList 

func (fmd *FakeMysqlDaemon) ExecuteSuperQueryList(ctx context.Context, queryList []string) error

ExecuteSuperQueryList is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) FetchSuperQuery 

func (fmd *FakeMysqlDaemon) FetchSuperQuery(ctx context.Context, query string) (*sqltypes.Result, error)

FetchSuperQuery returns the results from the map, if any

func (*FakeMysqlDaemon) GetAllPrivsConnection 

func (fmd *FakeMysqlDaemon) GetAllPrivsConnection() (*dbconnpool.DBConnection, error)

GetAllPrivsConnection is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) GetAppConnection 

func (fmd *FakeMysqlDaemon) GetAppConnection(ctx context.Context) (dbconnpool.PoolConnection, error)

GetAppConnection is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) GetDbaConnection 

func (fmd *FakeMysqlDaemon) GetDbaConnection() (*dbconnpool.DBConnection, error)

GetDbaConnection is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) GetMysqlPort 

func (fmd *FakeMysqlDaemon) GetMysqlPort() (int32, error)

GetMysqlPort is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) GetSchema 

func (fmd *FakeMysqlDaemon) GetSchema(dbName string, tables, excludeTables []string, includeViews bool) (*tabletmanagerdatapb.SchemaDefinition, error)

GetSchema is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) IsReadOnly 

func (fmd *FakeMysqlDaemon) IsReadOnly() (bool, error)

IsReadOnly is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) MasterPosition 

func (fmd *FakeMysqlDaemon) MasterPosition() (replication.Position, error)

MasterPosition is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) NewSlaveConnection 

func (fmd *FakeMysqlDaemon) NewSlaveConnection() (*SlaveConnection, error)

NewSlaveConnection is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) PreflightSchemaChange 

func (fmd *FakeMysqlDaemon) PreflightSchemaChange(dbName string, changes []string) ([]*tabletmanagerdatapb.SchemaChangeResult, error)

PreflightSchemaChange is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) PromoteSlave 

func (fmd *FakeMysqlDaemon) PromoteSlave(hookExtraEnv map[string]string) (replication.Position, error)

PromoteSlave is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) ReinitConfig 

func (fmd *FakeMysqlDaemon) ReinitConfig(ctx context.Context) error

ReinitConfig is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) ResetReplicationCommands 

func (fmd *FakeMysqlDaemon) ResetReplicationCommands() ([]string, error)

ResetReplicationCommands is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) RunMysqlUpgrade 

func (fmd *FakeMysqlDaemon) RunMysqlUpgrade() error

RunMysqlUpgrade is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) SemiSyncEnabled 

func (fmd *FakeMysqlDaemon) SemiSyncEnabled() (master, slave bool)

SemiSyncEnabled is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) SemiSyncSlaveStatus 

func (fmd *FakeMysqlDaemon) SemiSyncSlaveStatus() (bool, error)

SemiSyncSlaveStatus is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) SetMasterCommands 

func (fmd *FakeMysqlDaemon) SetMasterCommands(masterHost string, masterPort int) ([]string, error)

SetMasterCommands is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) SetReadOnly 

func (fmd *FakeMysqlDaemon) SetReadOnly(on bool) error

SetReadOnly is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) SetSemiSyncEnabled 

func (fmd *FakeMysqlDaemon) SetSemiSyncEnabled(master, slave bool) error

SetSemiSyncEnabled is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) SetSlavePositionCommands 

func (fmd *FakeMysqlDaemon) SetSlavePositionCommands(pos replication.Position) ([]string, error)

SetSlavePositionCommands is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) Shutdown 

func (fmd *FakeMysqlDaemon) Shutdown(ctx context.Context, waitForMysqld bool) error

Shutdown is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) SlaveStatus 

func (fmd *FakeMysqlDaemon) SlaveStatus() (Status, error)

SlaveStatus is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) Start 

func (fmd *FakeMysqlDaemon) Start(ctx context.Context, mysqldArgs ...string) error

Start is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) TabletDir 

func (fmd *FakeMysqlDaemon) TabletDir() string

TabletDir is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) Wait 

func (fmd *FakeMysqlDaemon) Wait(ctx context.Context) error

Wait is part of the MysqlDaemon interface.

func (*FakeMysqlDaemon) WaitForReparentJournal 

func (fmd *FakeMysqlDaemon) WaitForReparentJournal(ctx context.Context, timeCreatedNS int64) error

WaitForReparentJournal is part of the MysqlDaemon interface

func (*FakeMysqlDaemon) WaitMasterPos 

func (fmd *FakeMysqlDaemon) WaitMasterPos(_ context.Context, pos replication.Position) error

WaitMasterPos is part of the MysqlDaemon interface

type FileEntry 

type FileEntry struct {
	// Base is one of:
	// - backupInnodbDataHomeDir for files that go into Mycnf.InnodbDataHomeDir
	// - backupInnodbLogGroupHomeDir for files that go into Mycnf.InnodbLogGroupHomeDir
	// - backupData for files that go into Mycnf.DataDir
	Base string

	// Name is the file name, relative to Base
	Name string

	// Hash is the hash of the final data (transformed and
	// compressed if specified) stored in the BackupStorage.
	Hash string
}

FileEntry is one file to backup

type MapFunc 

type MapFunc func(index int) error

type Mycnf 

type Mycnf struct {
	// ServerID is the unique id for this server.
	// Used to create a bunch of named directories.
	ServerID uint32

	// MysqlPort is the port for the MySQL server running on this machine.
	// It is mainly used to communicate with topology server.
	MysqlPort int32

	// DataDir is where the table files are
	// (used by vt software for Clone)
	DataDir string

	// InnodbDataHomeDir is the data directory for innodb.
	// (used by vt software for Clone)
	InnodbDataHomeDir string

	// InnodbLogGroupHomeDir is the logs directory for innodb.
	// (used by vt software for Clone)
	InnodbLogGroupHomeDir string

	// SocketFile is the path to the local mysql.sock file.
	// (used by vt software to check server is running)
	SocketFile string

	// GeneralLogPath is the path to store general logs at,
	// if general-log is enabled.
	// (unused by vt software for now)
	GeneralLogPath string

	// ErrorLogPath is the path to store error logs at.
	// (unused by vt software for now)
	ErrorLogPath string

	// SlowLogPath is the slow query log path
	// (unused by vt software for now)
	SlowLogPath string

	// RelayLogPath is the path of the relay logs
	// (unused by vt software for now)
	RelayLogPath string

	// RelayLogIndexPath is the file name for the relay log index
	// (unused by vt software for now)
	RelayLogIndexPath string

	// RelayLogInfoPath is the file name for the relay log info file
	// (unused by vt software for now)
	RelayLogInfoPath string

	// BinLogPath is the base path for binlogs
	// (used by vt software for binlog streaming)
	BinLogPath string

	// MasterInfoFile is the master.info file location.
	// (unused by vt software for now)
	MasterInfoFile string

	// PidFile is the mysql.pid file location
	// (used by vt software to check server is running)
	PidFile string

	// TmpDir is where to create temporary tables
	// (unused by vt software for now)
	TmpDir string

	// SlaveLoadTmpDir is where to create tmp files for replication
	// (unused by vt software for now)
	SlaveLoadTmpDir string
	// contains filtered or unexported fields
}

Mycnf is a memory structure that contains a bunch of interesting parameters to start mysqld. It can be used to generate standard my.cnf files from a server id and mysql port. It can also be populated from an existing my.cnf, or by command line parameters.

func NewMycnf 

func NewMycnf(tabletUID uint32, mysqlPort int32) *Mycnf

NewMycnf fills the Mycnf structure with vt root paths and derived values. This is used to fill out the cnfTemplate values and generate my.cnf. uid is a unique id for a particular tablet - it must be unique within the tabletservers deployed within a keyspace, lest there be collisions on disk. mysqldPort needs to be unique per instance per machine.

func NewMycnfFromFlags 

func NewMycnfFromFlags(uid uint32) (mycnf *Mycnf, err error)

NewMycnfFromFlags creates a Mycnf object from the command line flags.

Multiple modes are supported:

  • at least mycnf_server_id is set on the command line --> then we read all parameters from the command line, and not from any my.cnf file.
  • mycnf_server_id is not passed in, but mycnf-file is passed in --> then we read that mycnf file
  • mycnf_server_id and mycnf-file are not passed in: --> then we use the default location of the my.cnf file for the provided uid and read that my.cnf file.

RegisterCommandLineFlags should have been called before calling this, otherwise we'll panic.

func ReadMycnf 

func ReadMycnf(cnfFile string) (mycnf *Mycnf, err error)

ReadMycnf will read an existing my.cnf from disk, and create a Mycnf object.

func (*Mycnf) RandomizeMysqlServerID 

func (cnf *Mycnf) RandomizeMysqlServerID() error

RandomizeMysqlServerID generates a random MySQL server_id.

The value assigned to ServerID will be in the range [100, 2^31): - It avoids 0 because that's reserved for mysqlbinlog dumps. - It also avoids 1-99 because low numbers are used for fake slave connections. See NewSlaveConnection() in slave_connection.go for more on that. - It avoids the 2^31 - 2^32-1 range, as there seems to be some confusion there. The main MySQL documentation at: http://dev.mysql.com/doc/refman/5.7/en/replication-options.html implies serverID is a full 32 bits number. The semi-sync log line at startup '[Note] Start semi-sync binlog_dump to slave ...' interprets the server_id as signed 32-bit (shows negative numbers for that range). Such an ID may also be responsible for a mysqld crash in semi-sync code, although we haven't been able to verify that yet. The issue for that is: https://github.com/youtube/vitess/issues/2280

type MysqlDaemon 

type MysqlDaemon interface {
	// Cnf returns the underlying mycnf
	Cnf() *Mycnf
	// TabletDir returns the tablet directory.
	TabletDir() string

	// methods related to mysql running or not
	Start(ctx context.Context, mysqldArgs ...string) error
	Shutdown(ctx context.Context, waitForMysqld bool) error
	RunMysqlUpgrade() error
	ReinitConfig(ctx context.Context) error
	Wait(ctx context.Context) error

	// GetMysqlPort returns the current port mysql is listening on.
	GetMysqlPort() (int32, error)

	// replication related methods
	SlaveStatus() (Status, error)
	SetSemiSyncEnabled(master, slave bool) error
	SemiSyncEnabled() (master, slave bool)
	SemiSyncSlaveStatus() (bool, error)

	// reparenting related methods
	ResetReplicationCommands() ([]string, error)
	MasterPosition() (replication.Position, error)
	IsReadOnly() (bool, error)
	SetReadOnly(on bool) error
	SetSlavePositionCommands(pos replication.Position) ([]string, error)
	SetMasterCommands(masterHost string, masterPort int) ([]string, error)
	WaitForReparentJournal(ctx context.Context, timeCreatedNS int64) error

	// DemoteMaster waits for all current transactions to finish,
	// and returns the current replication position. It will not
	// change the read_only state of the server.
	DemoteMaster() (replication.Position, error)

	WaitMasterPos(context.Context, replication.Position) error

	// PromoteSlave makes the slave the new master. It will not change
	// the read_only state of the server.
	PromoteSlave(map[string]string) (replication.Position, error)

	// Schema related methods
	GetSchema(dbName string, tables, excludeTables []string, includeViews bool) (*tabletmanagerdatapb.SchemaDefinition, error)
	PreflightSchemaChange(dbName string, changes []string) ([]*tabletmanagerdatapb.SchemaChangeResult, error)
	ApplySchemaChange(dbName string, change *tmutils.SchemaChange) (*tabletmanagerdatapb.SchemaChangeResult, error)

	// GetAppConnection returns a app connection to be able to talk to the database.
	GetAppConnection(ctx context.Context) (dbconnpool.PoolConnection, error)
	// GetDbaConnection returns a dba connection.
	GetDbaConnection() (*dbconnpool.DBConnection, error)
	// GetAllPrivsConnection returns an allprivs connection (for user with all privileges except SUPER).
	GetAllPrivsConnection() (*dbconnpool.DBConnection, error)

	// ExecuteSuperQueryList executes a list of queries, no result
	ExecuteSuperQueryList(ctx context.Context, queryList []string) error

	// FetchSuperQuery executes one query, returns the result
	FetchSuperQuery(ctx context.Context, query string) (*sqltypes.Result, error)

	// NewSlaveConnection returns a SlaveConnection to the database.
	NewSlaveConnection() (*SlaveConnection, error)

	// EnableBinlogPlayback enables playback of binlog events
	EnableBinlogPlayback() error

	// DisableBinlogPlayback disable playback of binlog events
	DisableBinlogPlayback() error

	// Close will close this instance of Mysqld. It will wait for all dba
	// queries to be finished.
	Close()
}

MysqlDaemon is the interface we use for abstracting Mysqld.

type MysqlFlavor 

type MysqlFlavor interface {
	// VersionMatch returns true if the version string (from
	// SELECT VERSION()) represents a server that this flavor
	// knows how to talk to.
	VersionMatch(version string) bool

	// MasterPosition returns the ReplicationPosition of a master.
	MasterPosition(mysqld *Mysqld) (replication.Position, error)

	// SlaveStatus returns the ReplicationStatus of a slave.
	SlaveStatus(mysqld *Mysqld) (Status, error)

	// ResetReplicationCommands returns the commands to completely reset
	// replication on the host.
	ResetReplicationCommands() []string

	// PromoteSlaveCommands returns the commands to run to change
	// a slave into a master.
	PromoteSlaveCommands() []string

	// SetSlavePositionCommands returns the commands to set the
	// replication position at which the slave will resume
	// when it is later reparented with SetMasterCommands.
	SetSlavePositionCommands(pos replication.Position) ([]string, error)

	// SetMasterCommands returns the commands to use the provided master
	// as the new master (without changing any GTID position).
	// It is guaranteed to be called with replication stopped.
	// It should not start or stop replication.
	SetMasterCommands(params *sqldb.ConnParams, masterHost string, masterPort int, masterConnectRetry int) ([]string, error)

	// ParseGTID parses a GTID in the canonical format of this
	// MySQL flavor into a replication.GTID interface value.
	ParseGTID(string) (replication.GTID, error)

	// ParseReplicationPosition parses a replication position in
	// the canonical format of this MySQL flavor into a
	// replication.Position struct.
	ParseReplicationPosition(string) (replication.Position, error)

	// SendBinlogDumpCommand sends the flavor-specific version of
	// the COM_BINLOG_DUMP command to start dumping raw binlog
	// events over a slave connection, starting at a given GTID.
	SendBinlogDumpCommand(conn *SlaveConnection, startPos replication.Position) error

	// MakeBinlogEvent takes a raw packet from the MySQL binlog
	// stream connection and returns a BinlogEvent through which
	// the packet can be examined.
	MakeBinlogEvent(buf []byte) replication.BinlogEvent

	// WaitMasterPos waits until slave replication reaches at
	// least targetPos.
	WaitMasterPos(ctx context.Context, mysqld *Mysqld, targetPos replication.Position) error

	// EnableBinlogPlayback prepares the server to play back
	// events from a binlog stream.  Whatever it does for a given
	// flavor, it must be idempotent.
	EnableBinlogPlayback(mysqld *Mysqld) error

	// DisableBinlogPlayback returns the server to the normal
	// state after playback is done.  Whatever it does for a given
	// flavor, it must be idempotent.
	DisableBinlogPlayback(mysqld *Mysqld) error
}

MysqlFlavor is the abstract interface for a flavor.

type Mysqld 

type Mysqld struct {
	// contains filtered or unexported fields
}

Mysqld is the object that represents a mysqld daemon running on this server.

func CreateMysqld 

func CreateMysqld(tabletUID uint32, mysqlSocket string, mysqlPort int32, dbconfigFlags dbconfigs.DBConfigFlag) (*Mysqld, error)

CreateMysqld returns a Mysqld object to use for working with a MySQL installation that hasn't been set up yet. This will generate a new my.cnf from scratch and use that to call NewMysqld().

func NewMysqld 

func NewMysqld(config *Mycnf, dbcfgs *dbconfigs.DBConfigs, dbconfigsFlags dbconfigs.DBConfigFlag) *Mysqld

NewMysqld creates a Mysqld object based on the provided configuration and connection parameters.

func OpenMysqld 

func OpenMysqld(tabletUID uint32, dbconfigFlags dbconfigs.DBConfigFlag) (*Mysqld, error)

OpenMysqld returns a Mysqld object to use for working with a MySQL installation that already exists. This will look for an existing my.cnf file and use that to call NewMysqld().

func (*Mysqld) ApplySchemaChange 

func (mysqld *Mysqld) ApplySchemaChange(dbName string, change *tmutils.SchemaChange) (*tabletmanagerdatapb.SchemaChangeResult, error)

ApplySchemaChange will apply the schema change to the given database.

func (*Mysqld) Close 

func (mysqld *Mysqld) Close()

Close will close this instance of Mysqld. It will wait for all dba queries to be finished.

func (*Mysqld) Cnf 

func (mysqld *Mysqld) Cnf() *Mycnf

Cnf returns the mysql config for the daemon

func (*Mysqld) DemoteMaster 

func (mysqld *Mysqld) DemoteMaster() (rp replication.Position, err error)

DemoteMaster will gracefully demote a master mysql instance to read only. If the master is still alive, then we need to demote it gracefully make it read-only, flush the writes and get the position

func (*Mysqld) DisableBinlogPlayback 

func (mysqld *Mysqld) DisableBinlogPlayback() error

DisableBinlogPlayback returns the server to the normal state after streaming. Whatever it does for a given flavor, it must be idempotent.

func (*Mysqld) EnableBinlogPlayback 

func (mysqld *Mysqld) EnableBinlogPlayback() error

EnableBinlogPlayback prepares the server to play back events from a binlog stream. Whatever it does for a given flavor, it must be idempotent.

func (*Mysqld) ExecuteSuperQuery 

func (mysqld *Mysqld) ExecuteSuperQuery(ctx context.Context, query string) error

ExecuteSuperQuery allows the user to execute a query as a super user.

func (*Mysqld) ExecuteSuperQueryList 

func (mysqld *Mysqld) ExecuteSuperQueryList(ctx context.Context, queryList []string) error

ExecuteSuperQueryList alows the user to execute queries as a super user.

func (*Mysqld) FetchSuperQuery 

func (mysqld *Mysqld) FetchSuperQuery(ctx context.Context, query string) (*sqltypes.Result, error)

FetchSuperQuery returns the results of executing a query as a super user.

func (*Mysqld) GetAllPrivsConnection 

func (mysqld *Mysqld) GetAllPrivsConnection() (*dbconnpool.DBConnection, error)

GetAllPrivsConnection creates a new DBConnection.

func (*Mysqld) GetAppConnection 

func (mysqld *Mysqld) GetAppConnection(ctx context.Context) (dbconnpool.PoolConnection, error)

GetAppConnection returns a connection from the app pool. Recycle needs to be called on the result.

func (*Mysqld) GetColumns 

func (mysqld *Mysqld) GetColumns(dbName, table string) ([]string, error)

GetColumns returns the columns of table.

func (*Mysqld) GetDbaConnection 

func (mysqld *Mysqld) GetDbaConnection() (*dbconnpool.DBConnection, error)

GetDbaConnection creates a new DBConnection.

func (*Mysqld) GetMysqlPort 

func (mysqld *Mysqld) GetMysqlPort() (int32, error)

GetMysqlPort returns mysql port

func (*Mysqld) GetPrimaryKeyColumns 

func (mysqld *Mysqld) GetPrimaryKeyColumns(dbName, table string) ([]string, error)

GetPrimaryKeyColumns returns the primary key columns of table.

func (*Mysqld) GetSchema 

func (mysqld *Mysqld) GetSchema(dbName string, tables, excludeTables []string, includeViews bool) (*tabletmanagerdatapb.SchemaDefinition, error)

GetSchema returns the schema for database for tables listed in tables. If tables is empty, return the schema for all tables.

func (*Mysqld) Init 

func (mysqld *Mysqld) Init(ctx context.Context, initDBSQLFile string) error

Init will create the default directory structure for the mysqld process, generate / configure a my.cnf file, install a skeleton database, and apply the provided initial SQL file.

func (*Mysqld) IsReadOnly 

func (mysqld *Mysqld) IsReadOnly() (bool, error)

IsReadOnly return true if the instance is read only

func (*Mysqld) MasterPosition 

func (mysqld *Mysqld) MasterPosition() (rp replication.Position, err error)

MasterPosition returns master replication position

func (*Mysqld) NewSlaveConnection 

func (mysqld *Mysqld) NewSlaveConnection() (*SlaveConnection, error)

NewSlaveConnection creates a new slave connection to the mysqld instance. It uses a pools.IDPool to ensure that the server IDs used to connect are unique within this process. This is done with the assumptions that:

  1. No other processes are making fake slave connections to our mysqld.
  2. No real slave servers will have IDs in the range 1-N where N is the peak number of concurrent fake slave connections we will ever make.

func (*Mysqld) OnTerm 

func (mysqld *Mysqld) OnTerm(f func())

OnTerm registers a function to be called if mysqld terminates for any reason other than a call to Mysqld.Shutdown(). This only works if mysqld was actually started by calling Start() on this Mysqld instance.

func (*Mysqld) PreflightSchemaChange 

func (mysqld *Mysqld) PreflightSchemaChange(dbName string, changes []string) ([]*tabletmanagerdatapb.SchemaChangeResult, error)

PreflightSchemaChange checks the schema changes in "changes" by applying them to an intermediate database that has the same schema as the target database.

func (*Mysqld) PromoteSlave 

func (mysqld *Mysqld) PromoteSlave(hookExtraEnv map[string]string) (replication.Position, error)

PromoteSlave will promote a slave to be the new master.

func (*Mysqld) RefreshConfig 

func (mysqld *Mysqld) RefreshConfig() error

RefreshConfig attempts to recreate the my.cnf from templates, and log and swap in to place if it's updated. It keeps a copy of the last version in case fallback is required. Should be called from a stable replica, server_id is not regenerated.

func (*Mysqld) ReinitConfig 

func (mysqld *Mysqld) ReinitConfig(ctx context.Context) error

ReinitConfig updates the config file as if Mysqld is initializing. At the moment it only randomizes ServerID because it's not safe to restore a replica from a backup and then give it the same ServerID as before, MySQL can then skip transactions in the replication stream with the same server_id.

func (*Mysqld) ResetReplicationCommands 

func (mysqld *Mysqld) ResetReplicationCommands() ([]string, error)

ResetReplicationCommands returns the commands to run to reset all replication for this host.

func (*Mysqld) RunMysqlUpgrade 

func (mysqld *Mysqld) RunMysqlUpgrade() error

RunMysqlUpgrade will run the mysql_upgrade program on the current install. Will be called only when mysqld is running with no network and no grant tables.

func (*Mysqld) SemiSyncEnabled 

func (mysqld *Mysqld) SemiSyncEnabled() (master, slave bool)

SemiSyncEnabled returns whether semi-sync is enabled for master or slave. If the semi-sync plugin is not loaded, we assume semi-sync is disabled.

func (*Mysqld) SemiSyncSlaveStatus 

func (mysqld *Mysqld) SemiSyncSlaveStatus() (bool, error)

SemiSyncSlaveStatus returns whether semi-sync is currently used by replication.

func (*Mysqld) SetMasterCommands 

func (mysqld *Mysqld) SetMasterCommands(masterHost string, masterPort int) ([]string, error)

SetMasterCommands returns the commands to run to make the provided host / port the master.

func (*Mysqld) SetReadOnly 

func (mysqld *Mysqld) SetReadOnly(on bool) error

SetReadOnly set/unset the read_only flag

func (*Mysqld) SetSemiSyncEnabled 

func (mysqld *Mysqld) SetSemiSyncEnabled(master, slave bool) error

SetSemiSyncEnabled enables or disables semi-sync replication for master and/or slave mode.

func (*Mysqld) SetSlavePositionCommands 

func (mysqld *Mysqld) SetSlavePositionCommands(pos replication.Position) ([]string, error)

SetSlavePositionCommands returns the commands to set the replication position at which the slave will resume when it is later reparented with SetMasterCommands.

func (*Mysqld) Shutdown 

func (mysqld *Mysqld) Shutdown(ctx context.Context, waitForMysqld bool) error

Shutdown will stop the mysqld daemon that is running in the background.

waitForMysqld: should the function block until mysqld has stopped? This can actually take a *long* time if the buffer cache needs to be fully flushed - on the order of 20-30 minutes.

If a mysqlctld address is provided in a flag, Shutdown will run remotely.

func (*Mysqld) SlaveStatus 

func (mysqld *Mysqld) SlaveStatus() (Status, error)

SlaveStatus returns the slave replication statuses

func (*Mysqld) Start 

func (mysqld *Mysqld) Start(ctx context.Context, mysqldArgs ...string) error

Start will start the mysql daemon, either by running the 'mysqld_start' hook, or by running mysqld_safe in the background. If a mysqlctld address is provided in a flag, Start will run remotely. When waiting for mysqld to start, we will use the dba user.

func (*Mysqld) TabletDir 

func (mysqld *Mysqld) TabletDir() string

TabletDir returns the main tablet directory. It's a method so it can be accessed through the MysqlDaemon interface.

func (*Mysqld) Teardown 

func (mysqld *Mysqld) Teardown(ctx context.Context, force bool) error

Teardown will shutdown the running daemon, and delete the root directory.

func (*Mysqld) Wait 

func (mysqld *Mysqld) Wait(ctx context.Context) error

Wait returns nil when mysqld is up and accepting connections. It will use the dba credentials to try to connect. Use wait() with different credentials if needed.

func (*Mysqld) WaitForReparentJournal 

func (mysqld *Mysqld) WaitForReparentJournal(ctx context.Context, timeCreatedNS int64) error

WaitForReparentJournal will wait until the context is done for the row in the reparent_journal table.

func (*Mysqld) WaitMasterPos 

func (mysqld *Mysqld) WaitMasterPos(ctx context.Context, targetPos replication.Position) error

WaitMasterPos lets slaves wait to given replication position

type SlaveConnection 

type SlaveConnection struct {
	*mysqlconn.Conn
	// contains filtered or unexported fields
}

SlaveConnection represents a connection to mysqld that pretends to be a slave connecting for replication. Each such connection must identify itself to mysqld with a server ID that is unique both among other SlaveConnections and among actual slaves in the topology.

func (*SlaveConnection) Close 

func (sc *SlaveConnection) Close()

Close closes the slave connection, which also signals an ongoing dump started with StartBinlogDump() to stop and close its BinlogEvent channel. The ID for the slave connection is recycled back into the pool.

func (*SlaveConnection) StartBinlogDumpFromBinlogBeforeTimestamp 

func (sc *SlaveConnection) StartBinlogDumpFromBinlogBeforeTimestamp(ctx context.Context, timestamp int64) (<-chan replication.BinlogEvent, error)

StartBinlogDumpFromBinlogBeforeTimestamp requests a replication binlog dump from the master mysqld starting with a file that has timestamps smaller than the provided timestamp, and then sends binlog events to the provided channel.

The startup phase will list all the binary logs, and find the one that has events starting strictly before the provided timestamp. It will then start from there, and stream all events. It is the responsability of the calling site to filter the events more.

MySQL 5.6+ note: we need to do it that way because of the way the GTIDSet works. In the previous two streaming functions, we pass in the full GTIDSet (that has the list of all transactions seen in the replication stream). In this case, we don't know it, all we have is the binlog file names. We depend on parsing the first PREVIOUS_GTIDS_EVENT event in the logs to get it. So we need the caller to parse that event, and it can't be skipped because its timestamp is lower. Then, for each subsequent event, the caller also needs to add the event GTID to its GTIDSet. Otherwise it won't be correct ever. So the caller really needs to build up its GTIDSet along the entire file, not just for events whose timestamp is in a given range.

The stream will continue in the background, waiting for new events if necessary, until the connection is closed, either by the master or by canceling the context.

Note the context is valid and used until eventChan is closed.

func (*SlaveConnection) StartBinlogDumpFromCurrent 

func (sc *SlaveConnection) StartBinlogDumpFromCurrent(ctx context.Context) (replication.Position, <-chan replication.BinlogEvent, error)

StartBinlogDumpFromCurrent requests a replication binlog dump from the current position.

func (*SlaveConnection) StartBinlogDumpFromPosition 

func (sc *SlaveConnection) StartBinlogDumpFromPosition(ctx context.Context, startPos replication.Position) (<-chan replication.BinlogEvent, error)

StartBinlogDumpFromPosition requests a replication binlog dump from the master mysqld at the given Position and then sends binlog events to the provided channel. The stream will continue in the background, waiting for new events if necessary, until the connection is closed, either by the master or by canceling the context.

Note the context is valid and used until eventChan is closed.

type Status 

type Status struct {
	Position            replication.Position
	SlaveIORunning      bool
	SlaveSQLRunning     bool
	SecondsBehindMaster uint
	MasterHost          string
	MasterPort          int
	MasterConnectRetry  int
}

Status holds replication information from SHOW SLAVE STATUS.

func NewStatus 

func NewStatus(masterAddr string) (*Status, error)

NewStatus creates a Status pointing to masterAddr.

func ProtoToStatus 

func ProtoToStatus(r *replicationdatapb.Status) Status

ProtoToStatus translates a proto Status, or panics

func (*Status) MasterAddr 

func (rs *Status) MasterAddr() string

MasterAddr returns the host:port address of the master.

func (*Status) SlaveRunning 

func (rs *Status) SlaveRunning() bool

SlaveRunning returns true iff both the Slave IO and Slave SQL threads are running.

Source Files

View all Source files

Directories

Path Synopsis
Package backupstorage contains the interface and file system implementation of the backup system.
 Package backupstorage contains the interface and file system implementation of the backup system.
Package cephbackupstorage implements the BackupStorage interface for Ceph Cloud Storage.
 Package cephbackupstorage implements the BackupStorage interface for Ceph Cloud Storage.
Package filebackupstorage implements the BackupStorage interface for a local filesystem (which can be an NFS mount).
 Package filebackupstorage implements the BackupStorage interface for a local filesystem (which can be an NFS mount).
Package gcsbackupstorage implements the BackupStorage interface for Google Cloud Storage.
 Package gcsbackupstorage implements the BackupStorage interface for Google Cloud Storage.
Package grpcmysqlctlclient contains the gRPC1 version of the mysqlctl client protocol.
 Package grpcmysqlctlclient contains the gRPC1 version of the mysqlctl client protocol.
Package grpcmysqlctlserver contains the gRPC implementation of the server side of the remote execution of mysqlctl commands.
 Package grpcmysqlctlserver contains the gRPC implementation of the server side of the remote execution of mysqlctl commands.
Package mysqlctlclient contains the generic client side of the remote mysqlctl protocol.
 Package mysqlctlclient contains the generic client side of the remote mysqlctl protocol.
Package s3backupstorage implements the BackupStorage interface for AWS S3.
 Package s3backupstorage implements the BackupStorage interface for AWS S3.
Package tmutils contains helper methods to deal with the tabletmanagerdata proto3 structures.
 Package tmutils contains helper methods to deal with the tabletmanagerdata proto3 structures.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.