replication

package
v0.21.0-rc1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 15, 2024 License: Apache-2.0 Imports: 13 Imported by: 4

Documentation

Index

Constants

View Source 
const FilePosFlavorID = "FilePos"

FilePosFlavorID is the string identifier for the filePos flavor.

View Source 
const MariadbFlavorID = "MariaDB"

MariadbFlavorID is the string identifier for the MariaDB flavor.

View Source 
const (
	// MaximumPositionSize is the maximum size of a replication position.
	// It is used as the maximum column size in the reparent_journal table
	// and other related tables. A row has a maximum size of 65535 bytes.
	// So we want to stay under that. We use VARBINARY so the character
	// set doesn't matter, we only store ascii characters anyway.
	MaximumPositionSize = 64000
)
View Source 
const Mysql56FlavorID = "MySQL56"

Mysql56FlavorID is the string identifier for the Mysql56 flavor.

Variables

View Source 
var (
	ErrExpectMysql56Flavor = vterrors.Errorf(vtrpc.Code_INVALID_ARGUMENT, "expected MySQL GTID position but found a different or invalid format.")
)

Functions

func DecodePositionMySQL56 added in v0.20.0 

func DecodePositionMySQL56(s string) (rp Position, gtidSet Mysql56GTIDSet, err error)

DecodePositionMySQL56 converts a string into a Position value with the MySQL56 flavor. The function returns an error if the given string does not translate to a MySQL56 GTID set. The prefix "MySQL56/" is optional in the input string. Examples of inputs strings that produce valid result: - "MySQL56/16b1039f-22b6-11ed-b765-0a43f95f28a3:1-615" - "16b1039f-22b6-11ed-b765-0a43f95f28a3:1-615"

func EncodeGTID 

func EncodeGTID(gtid GTID) string

EncodeGTID returns a string that contains both the flavor and value of the GTID, so that the correct parser can be selected when that string is passed to DecodeGTID.

func EncodePosition 

func EncodePosition(rp Position) string

EncodePosition returns a string that contains both the flavor and value of the Position, so that the correct parser can be selected when that string is passed to DecodePosition.

func ErrantGTIDsOnReplica 

func ErrantGTIDsOnReplica(replicaPosition Position, primaryPosition Position) (string, error)

ErrantGTIDsOnReplica gets the errant GTIDs on the replica by comparing against the primary position and UUID.

func GTIDCount added in v0.21.0 

func GTIDCount(gtidStr string) (int64, error)

GTIDCount returns the number of GTIDs in a GTID set.

func PrimaryStatusToProto 

func PrimaryStatusToProto(s PrimaryStatus) *replicationdatapb.PrimaryStatus

PrimaryStatusToProto translates a PrimaryStatus to proto3.

func ReplicationStatusToProto 

func ReplicationStatusToProto(s ReplicationStatus) *replicationdatapb.Status

ReplicationStatusToProto translates a Status to proto3.

func Subtract 

func Subtract(lhs, rhs string) (string, error)

Subtract takes in two Mysql56GTIDSets as strings and subtracts the second from the first The result is also a string. An error is thrown if parsing is not possible for either GTIDSets

Types

type FilePosGTID 

type FilePosGTID struct {
	File string
	Pos  uint32
}

FilePosGTID implements GTID.

func (FilePosGTID) AddGTID 

func (gtid FilePosGTID) AddGTID(other GTID) GTIDSet

AddGTID implements GTIDSet.AddGTID().

func (FilePosGTID) Contains 

func (gtid FilePosGTID) Contains(other GTIDSet) bool

Contains implements GTIDSet.Contains().

func (FilePosGTID) ContainsGTID 

func (gtid FilePosGTID) ContainsGTID(other GTID) bool

ContainsGTID implements GTIDSet.ContainsGTID().

func (FilePosGTID) Equal 

func (gtid FilePosGTID) Equal(other GTIDSet) bool

Equal implements GTIDSet.Equal().

func (FilePosGTID) Flavor 

func (gtid FilePosGTID) Flavor() string

Flavor implements GTID.Flavor().

func (FilePosGTID) GTIDSet 

func (gtid FilePosGTID) GTIDSet() GTIDSet

GTIDSet implements GTID.GTIDSet().

func (FilePosGTID) Last 

func (gtid FilePosGTID) Last() string

Last returns last filePosition For filePos based GTID we have only one position here we will just return the current filePos

func (FilePosGTID) SequenceDomain 

func (gtid FilePosGTID) SequenceDomain() any

SequenceDomain implements GTID.SequenceDomain().

func (FilePosGTID) SequenceNumber 

func (gtid FilePosGTID) SequenceNumber() any

SequenceNumber implements GTID.SequenceNumber().

func (FilePosGTID) SourceServer 

func (gtid FilePosGTID) SourceServer() any

SourceServer implements GTID.SourceServer().

func (FilePosGTID) String 

func (gtid FilePosGTID) String() string

String implements GTID.String().

func (FilePosGTID) Union 

func (gtid FilePosGTID) Union(other GTIDSet) GTIDSet

Union implements GTIDSet.Union().

type GTID 

type GTID interface {
	// String returns the canonical printed form of the GTID as expected by a
	// particular flavor of MySQL.
	String() string

	// Flavor returns the key under which the corresponding GTID parser function
	// is registered in the gtidParsers map.
	Flavor() string

	// SourceServer returns the ID of the server that generated the transaction.
	SourceServer() any

	// SequenceNumber returns the ID number that increases with each transaction.
	// It is only valid to compare the sequence numbers of two GTIDs if they have
	// the same domain value.
	SequenceNumber() any

	// SequenceDomain returns the ID of the domain within which two sequence
	// numbers can be meaningfully compared.
	SequenceDomain() any

	// GTIDSet returns a GTIDSet of the same flavor as this GTID, containing only
	// this GTID.
	GTIDSet() GTIDSet
}

GTID represents a Global Transaction ID, also known as Transaction Group ID. Each flavor of MySQL has its own format for the GTID. This interface is used along with various MysqlFlavor implementations to abstract the differences.

Types that implement GTID should use a non-pointer receiver. This ensures that comparing GTID interface values with == has the expected semantics.

func DecodeGTID 

func DecodeGTID(s string) (GTID, error)

DecodeGTID converts a string in the format returned by EncodeGTID back into a GTID interface value with the correct underlying flavor.

func MustDecodeGTID 

func MustDecodeGTID(s string) GTID

MustDecodeGTID calls DecodeGTID and panics on error.

func MustParseGTID 

func MustParseGTID(flavor, value string) GTID

MustParseGTID calls ParseGTID and panics on error.

func ParseGTID 

func ParseGTID(flavor, value string) (GTID, error)

ParseGTID calls the GTID parser for the specified flavor.

type GTIDSet 

type GTIDSet interface {
	// String returns the canonical printed form of the set as expected by a
	// particular flavor of MySQL.
	String() string

	// Flavor returns the key under which the corresponding parser function is
	// registered in the transactionSetParsers map.
	Flavor() string

	// ContainsGTID returns true if the set contains the specified transaction.
	ContainsGTID(GTID) bool

	// Contains returns true if the set is a superset of another set. All implementations should return false if
	// other GTIDSet is not the right concrete type for that flavor.
	Contains(GTIDSet) bool

	// Equal returns true if the set is equal to another set.
	Equal(GTIDSet) bool

	// AddGTID returns a new GTIDSet that is expanded to contain the given GTID.
	AddGTID(GTID) GTIDSet

	// Union returns a union of the receiver GTIDSet and the supplied GTIDSet.
	Union(GTIDSet) GTIDSet

	// Union returns a union of the receiver GTIDSet and the supplied GTIDSet.
	Last() string
}

GTIDSet represents the set of transactions received or applied by a server. In some flavors, a single GTID is enough to specify the set of all transactions that came before it, but in others a more complex structure is required.

GTIDSet is wrapped by replication.Position, which is a concrete struct. When sending a GTIDSet over RPCs, encode/decode it as a string. Most code outside of this package should use replication.Position rather than GTIDSet.

func ParseFilePosGTIDSet 

func ParseFilePosGTIDSet(s string) (GTIDSet, error)

ParseFilePosGTIDSet is registered as a GTIDSet parser.

func ParseMariadbGTIDSet 

func ParseMariadbGTIDSet(s string) (GTIDSet, error)

ParseMariadbGTIDSet is registered as a GTIDSet parser.

type MariadbGTID 

type MariadbGTID struct {
	// Domain is the ID number of the domain within which sequence numbers apply.
	Domain uint32
	// Server is the ID of the server that generated the transaction.
	Server uint32
	// Sequence is the sequence number of the transaction within the domain.
	Sequence uint64
}

MariadbGTID implements GTID.

func (MariadbGTID) Flavor 

func (gtid MariadbGTID) Flavor() string

Flavor implements GTID.Flavor().

func (MariadbGTID) GTIDSet 

func (gtid MariadbGTID) GTIDSet() GTIDSet

GTIDSet implements GTID.GTIDSet().

func (MariadbGTID) SequenceDomain 

func (gtid MariadbGTID) SequenceDomain() any

SequenceDomain implements GTID.SequenceDomain().

func (MariadbGTID) SequenceNumber 

func (gtid MariadbGTID) SequenceNumber() any

SequenceNumber implements GTID.SequenceNumber().

func (MariadbGTID) SourceServer 

func (gtid MariadbGTID) SourceServer() any

SourceServer implements GTID.SourceServer().

func (MariadbGTID) String 

func (gtid MariadbGTID) String() string

String implements GTID.String().

type MariadbGTIDSet 

type MariadbGTIDSet map[uint32]MariadbGTID

MariadbGTIDSet implements GTIDSet.

func (MariadbGTIDSet) AddGTID 

func (gtidSet MariadbGTIDSet) AddGTID(other GTID) GTIDSet

AddGTID implements GTIDSet.AddGTID().

func (MariadbGTIDSet) Contains 

func (gtidSet MariadbGTIDSet) Contains(other GTIDSet) bool

Contains implements GTIDSet.Contains().

func (MariadbGTIDSet) ContainsGTID 

func (gtidSet MariadbGTIDSet) ContainsGTID(other GTID) bool

ContainsGTID implements GTIDSet.ContainsGTID().

func (MariadbGTIDSet) Equal 

func (gtidSet MariadbGTIDSet) Equal(other GTIDSet) bool

Equal implements GTIDSet.Equal().

func (MariadbGTIDSet) Flavor 

func (gtidSet MariadbGTIDSet) Flavor() string

Flavor implements GTIDSet.Flavor()

func (MariadbGTIDSet) Last 

func (gtidSet MariadbGTIDSet) Last() string

Last returns the last gtid

func (MariadbGTIDSet) String 

func (gtidSet MariadbGTIDSet) String() string

String implements GTIDSet.String()

func (MariadbGTIDSet) Union 

func (gtidSet MariadbGTIDSet) Union(other GTIDSet) GTIDSet

Union implements GTIDSet.Union(). This is a pure method, and does not mutate the receiver.

type Mysql56GTID 

type Mysql56GTID struct {
	// Server is the SID of the server that originally committed the transaction.
	Server SID
	// Sequence is the sequence number of the transaction within a given Server's
	// scope.
	Sequence int64
}

Mysql56GTID implements GTID

func (Mysql56GTID) Flavor 

func (gtid Mysql56GTID) Flavor() string

Flavor implements GTID.Flavor().

func (Mysql56GTID) GTIDSet 

func (gtid Mysql56GTID) GTIDSet() GTIDSet

GTIDSet implements GTID.GTIDSet().

func (Mysql56GTID) SequenceDomain 

func (gtid Mysql56GTID) SequenceDomain() any

SequenceDomain implements GTID.SequenceDomain().

func (Mysql56GTID) SequenceNumber 

func (gtid Mysql56GTID) SequenceNumber() any

SequenceNumber implements GTID.SequenceNumber().

func (Mysql56GTID) SourceServer 

func (gtid Mysql56GTID) SourceServer() any

SourceServer implements GTID.SourceServer().

func (Mysql56GTID) String 

func (gtid Mysql56GTID) String() string

String implements GTID.String().

type Mysql56GTIDSet 

type Mysql56GTIDSet map[SID][]interval

Mysql56GTIDSet implements GTIDSet for MySQL 5.6.

func NewMysql56GTIDSetFromSIDBlock 

func NewMysql56GTIDSetFromSIDBlock(data []byte) (Mysql56GTIDSet, error)

NewMysql56GTIDSetFromSIDBlock builds a Mysql56GTIDSet from parsing a SID Block. This is the reverse of the SIDBlock method.

Expected format: 

# bytes field
8       nSIDs

(nSIDs times) 

16      SID
8       nIntervals

(nIntervals times) 

8       start
8       end

func ParseMysql56GTIDSet 

func ParseMysql56GTIDSet(s string) (Mysql56GTIDSet, error)

ParseMysql56GTIDSet is registered as a GTIDSet parser.

https://dev.mysql.com/doc/refman/5.6/en/replication-gtids-concepts.html

func (Mysql56GTIDSet) AddGTID 

func (set Mysql56GTIDSet) AddGTID(gtid GTID) GTIDSet

AddGTID implements GTIDSet.

func (Mysql56GTIDSet) Contains 

func (set Mysql56GTIDSet) Contains(other GTIDSet) bool

Contains implements GTIDSet.

func (Mysql56GTIDSet) ContainsGTID 

func (set Mysql56GTIDSet) ContainsGTID(gtid GTID) bool

ContainsGTID implements GTIDSet.

func (Mysql56GTIDSet) Difference 

func (set Mysql56GTIDSet) Difference(other Mysql56GTIDSet) Mysql56GTIDSet

Difference will supply the difference between the receiver and supplied Mysql56GTIDSets, and supply the result as a Mysql56GTIDSet.

func (Mysql56GTIDSet) Equal 

func (set Mysql56GTIDSet) Equal(other GTIDSet) bool

Equal implements GTIDSet.

func (Mysql56GTIDSet) Flavor 

func (Mysql56GTIDSet) Flavor() string

Flavor implements GTIDSet.

func (Mysql56GTIDSet) Last 

func (set Mysql56GTIDSet) Last() string

Last returns the last gtid as string For gtidset having multiple SIDs or multiple intervals it just returns the last SID with last interval

func (Mysql56GTIDSet) RemoveUUID 

func (set Mysql56GTIDSet) RemoveUUID(uuid SID) Mysql56GTIDSet

RemoveUUID removes a specific UUID from the gtid set.

func (Mysql56GTIDSet) SIDBlock 

func (set Mysql56GTIDSet) SIDBlock() []byte

SIDBlock returns the binary encoding of a MySQL 5.6 GTID set as expected by internal commands that refer to an "SID block".

e.g. https://dev.mysql.com/doc/internals/en/com-binlog-dump-gtid.html

func (Mysql56GTIDSet) SIDs 

func (set Mysql56GTIDSet) SIDs() []SID

SIDs returns a sorted list of SIDs in the set.

func (Mysql56GTIDSet) String 

func (set Mysql56GTIDSet) String() string

String implements GTIDSet.

func (Mysql56GTIDSet) Union 

func (set Mysql56GTIDSet) Union(other GTIDSet) GTIDSet

Union implements GTIDSet.Union().

type Position 

type Position struct {

	// GTIDSet is the underlying GTID set. It must not be anonymous,
	// or else Position would itself also implement the GTIDSet interface.
	GTIDSet GTIDSet
	// contains filtered or unexported fields
}

Position represents the information necessary to describe which transactions a server has seen, so that it can request a replication stream from a new source that picks up where it left off.

This must be a concrete struct because custom Unmarshalers can't be registered on an interface.

The == operator should not be used with Position, because the underlying GTIDSet might use slices, which are not comparable. Using == in those cases will result in a run-time panic.

func AppendGTID 

func AppendGTID(rp Position, gtid GTID) Position

AppendGTID returns a new Position that represents the position after the given GTID is replicated.

func DecodePosition 

func DecodePosition(s string) (rp Position, err error)

DecodePosition converts a string in the format returned by EncodePosition back into a Position value with the correct underlying flavor.

func DecodePositionDefaultFlavor 

func DecodePositionDefaultFlavor(s string, flavor string) (rp Position, err error)

DecodePositionDefaultFlavor converts a string in the format returned by EncodePosition back into a Position value with the correct underlying flavor. If the string does not indicate a flavor, then the 'flavor' argument is used. For example: - DecodePositionDefaultFlavor("MySQL56/16b1039f-22b6-11ed-b765-0a43f95f28a3:1-615", "foo"): "MySQL56" explicitly indicated, this is the flavor. - DecodePositionDefaultFlavor("16b1039f-22b6-11ed-b765-0a43f95f28a3:1-615", "MySQL56"): No flavor indicated in `s`, therefore using "MySQL56"

func MustParsePosition 

func MustParsePosition(flavor, value string) Position

MustParsePosition calls ParsePosition and panics on error.

func ParsePosition 

func ParsePosition(flavor, value string) (rp Position, err error)

ParsePosition calls the parser for the specified flavor.

func (Position) AtLeast 

func (rp Position) AtLeast(other Position) bool

AtLeast returns true if this position is equal to or after another.

func (Position) Equal 

func (rp Position) Equal(other Position) bool

Equal returns true if this position is equal to another.

func (Position) IsZero 

func (rp Position) IsZero() bool

IsZero returns true if this is the zero value, Position{}.

func (Position) MarshalJSON 

func (rp Position) MarshalJSON() ([]byte, error)

MarshalJSON implements encoding/json.Marshaler.

func (*Position) MatchesFlavor 

func (rp *Position) MatchesFlavor(flavor string) bool

MatchesFlavor will take a flavor string, and return whether the positions GTIDSet matches the supplied flavor. The caller should use the constants Mysql56FlavorID, MariadbFlavorID, or FilePosFlavorID when supplying the flavor string.

func (Position) String 

func (rp Position) String() string

String returns a string representation of the underlying GTIDSet. If the set is nil, it returns "<nil>" in the style of Sprintf("%v", nil).

func (*Position) UnmarshalJSON 

func (rp *Position) UnmarshalJSON(buf []byte) error

UnmarshalJSON implements encoding/json.Unmarshaler.

type PrimaryStatus 

type PrimaryStatus struct {
	// Position represents the server's GTID based position.
	Position Position
	// FilePosition represents the server's file based position.
	FilePosition Position
}

PrimaryStatus holds replication information from SHOW BINARY LOG STATUS.

func ParseFilePosPrimaryStatus 

func ParseFilePosPrimaryStatus(resultMap map[string]string) (PrimaryStatus, error)

func ParseMysqlPrimaryStatus 

func ParseMysqlPrimaryStatus(resultMap map[string]string) (PrimaryStatus, error)

func ParsePrimaryStatus 

func ParsePrimaryStatus(fields map[string]string) PrimaryStatus

ParsePrimaryStatus parses the common fields of SHOW BINARY LOG STATUS.

type ReplicationState 

type ReplicationState int32
const (
	ReplicationStateUnknown ReplicationState = iota
	ReplicationStateStopped
	ReplicationStateConnecting
	ReplicationStateRunning
)

func ReplicationStatusToState 

func ReplicationStatusToState(s string) ReplicationState

ReplicationStatusToState converts a value you have for the IO thread(s) or SQL thread(s) or Group Replication applier thread(s) from MySQL or intermediate layers to a ReplicationState. on,yes,true == ReplicationStateRunning off,no,false == ReplicationStateStopped connecting == ReplicationStateConnecting anything else == ReplicationStateUnknown

type ReplicationStatus 

type ReplicationStatus struct {
	// Position is the current position of the replica. For GTID replication implementations
	// it is the executed GTID set. For file replication implementation, it is same as
	// FilePosition
	Position Position
	// RelayLogPosition is the Position that the replica would be at if it
	// were to finish executing everything that's currently in its relay log.
	// However, some MySQL flavors don't expose this information,
	// in which case RelayLogPosition.IsZero() will be true.
	// If ReplicationLagUnknown is true then we should not rely on the seconds
	// behind value and we can instead try to calculate the lag ourselves when
	// appropriate. For MySQL GTID replication implementation it is the union of
	// executed GTID set and retrieved GTID set. For file replication implementation,
	// it is same as RelayLogSourceBinlogEquivalentPosition
	RelayLogPosition Position
	// FilePosition stores the position of the source tablets binary log
	// upto which the SQL thread of the replica has run.
	FilePosition Position
	// RelayLogSourceBinlogEquivalentPosition stores the position of the source tablets binary log
	// upto which the IO thread has read and added to the relay log
	RelayLogSourceBinlogEquivalentPosition Position
	// RelayLogFilePosition stores the position in the relay log file
	RelayLogFilePosition  Position
	SourceServerID        uint32
	IOState               ReplicationState
	LastIOError           string
	SQLState              ReplicationState
	LastSQLError          string
	ReplicationLagSeconds uint32
	ReplicationLagUnknown bool
	SourceHost            string
	SourcePort            int32
	SourceUser            string
	ConnectRetry          int32
	SourceUUID            SID
	SQLDelay              uint32
	AutoPosition          bool
	UsingGTID             bool
	HasReplicationFilters bool
	SSLAllowed            bool
}

ReplicationStatus holds replication information from SHOW REPLICA STATUS.

func ParseFilePosReplicationStatus 

func ParseFilePosReplicationStatus(resultMap map[string]string) (ReplicationStatus, error)

func ParseMariadbReplicationStatus 

func ParseMariadbReplicationStatus(resultMap map[string]string) (ReplicationStatus, error)

func ParseMysqlReplicationStatus 

func ParseMysqlReplicationStatus(resultMap map[string]string, replicaTerminology bool) (ReplicationStatus, error)

func ParseReplicationStatus 

func ParseReplicationStatus(fields map[string]string, replica bool) ReplicationStatus

ParseReplicationStatus parses the common (non-flavor-specific) fields of ReplicationStatus

func ProtoToReplicationStatus 

func ProtoToReplicationStatus(s *replicationdatapb.Status) ReplicationStatus

ProtoToReplicationStatus translates a proto Status, or panics.

func (*ReplicationStatus) FindErrantGTIDs 

func (s *ReplicationStatus) FindErrantGTIDs(otherReplicaStatuses []*ReplicationStatus) (Mysql56GTIDSet, error)

FindErrantGTIDs can be used to find errant GTIDs in the receiver's relay log, by comparing it against all known replicas, provided as a list of ReplicationStatus's. This method only works if the flavor for all retrieved ReplicationStatus's is MySQL. The result is returned as a Mysql56GTIDSet, each of whose elements is a found errant GTID. This function is best effort in nature. If it marks something as errant, then it is for sure errant. But there may be cases of errant GTIDs, which aren't caught by this function.

func (*ReplicationStatus) Healthy 

func (s *ReplicationStatus) Healthy() bool

Healthy returns true if both the SQL IO components are healthy

func (*ReplicationStatus) IOHealthy 

func (s *ReplicationStatus) IOHealthy() bool

IOHealthy returns true if the IO thread is running OR, the IO thread is connecting AND there's no IO error from the last attempt to connect to the source.

func (*ReplicationStatus) Running 

func (s *ReplicationStatus) Running() bool

Running returns true if both the IO and SQL threads are running.

func (*ReplicationStatus) SQLHealthy 

func (s *ReplicationStatus) SQLHealthy() bool

SQLHealthy returns true if the SQLState is running. For consistency and to support altering this calculation in the future.

type SID 

type SID [16]byte

SID is the 16-byte unique ID of a MySQL 5.6 server.

func ParseSID 

func ParseSID(s string) (sid SID, err error)

ParseSID parses an SID in the form used by MySQL 5.6.

func (SID) String 

func (sid SID) String() string

String prints an SID in the form used by MySQL 5.6.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.