tabletmanager

package
v0.0.0-...-444a4ab Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 14, 2015 License: BSD-3-Clause Imports: 51 Imported by: 0

Documentation

Overview

Package tabletmanager exports the ActionAgent object. It keeps the local tablet state, starts / stops all associated services (query service, update stream, binlog players, ...), and handles tabletmanager RPCs to update the state.

The agent is responsible for maintaining the tablet record in the topology server. Only 'ScrapTablet -force' and 'DeleteTablet' should be run by other processes, everything else should ask the tablet server to make the change.

Most RPC calls lock the actionMutex, except the easy read-only ones. RPC calls that change the tablet record will also call updateState.

See rpc_server.go for all cases, and which actions take the actionMutex, and which run changeCallback.

Index

Constants

View Source
const (
	BPM_STATE_RUNNING int64 = iota
	BPM_STATE_STOPPED
)

Variables

View Source
var RegisterQueryServices []RegisterQueryService

RegisterQueryServices is a list of functions to call when the delayed registration is triggered.

Functions

func HttpHandleSnapshots

func HttpHandleSnapshots(mycnf *mysqlctl.Mycnf, uid uint32)

HttpHandleSnapshots handles the serving of files from the local tablet

func RegisterBinlogPlayerMap

func RegisterBinlogPlayerMap(blm *BinlogPlayerMap)

RegisterBinlogPlayerMap registers the varz for the players

func SetReparentFlags

func SetReparentFlags(fast bool, timeout time.Duration)

SetReparentFlags changes flag values. It should only be used in tests.

Types

type ActionAgent

type ActionAgent struct {
	// The following fields are set during creation
	QueryServiceControl tabletserver.QueryServiceControl
	HealthReporter      health.Reporter
	TopoServer          topo.Server
	TabletAlias         topo.TabletAlias
	Mysqld              *mysqlctl.Mysqld
	MysqlDaemon         mysqlctl.MysqlDaemon
	DBConfigs           *dbconfigs.DBConfigs
	SchemaOverrides     []tabletserver.SchemaOverride
	BinlogPlayerMap     *BinlogPlayerMap
	LockTimeout         time.Duration

	// This is the History of the health checks, public so status
	// pages can display it
	History *history.History
	// contains filtered or unexported fields
}

ActionAgent is the main class for the agent.

func NewActionAgent

func NewActionAgent(
	queryServiceControl tabletserver.QueryServiceControl,
	batchCtx context.Context,
	tabletAlias topo.TabletAlias,
	dbcfgs *dbconfigs.DBConfigs,
	mycnf *mysqlctl.Mycnf,
	port, securePort int,
	overridesFile string,
	lockTimeout time.Duration,
) (agent *ActionAgent, err error)

NewActionAgent creates a new ActionAgent and registers all the associated services.

batchCtx is the context that the agent will use for any background tasks it spawns.

func NewTestActionAgent

func NewTestActionAgent(batchCtx context.Context, ts topo.Server, tabletAlias topo.TabletAlias, port int, mysqlDaemon mysqlctl.MysqlDaemon) *ActionAgent

NewTestActionAgent creates an agent for test purposes. Only a subset of features are supported now, but we'll add more over time.

func (*ActionAgent) ApplySchema

func (agent *ActionAgent) ApplySchema(ctx context.Context, change *myproto.SchemaChange) (*myproto.SchemaChangeResult, error)

ApplySchema will apply a schema change Should be called under RPCWrapLockAction.

func (*ActionAgent) BlacklistedTables

func (agent *ActionAgent) BlacklistedTables() []string

BlacklistedTables reads the list of blacklisted tables from the TabletControl record (if any) stored in the agent, protected by mutex.

func (*ActionAgent) BreakSlaves

func (agent *ActionAgent) BreakSlaves(ctx context.Context) error

BreakSlaves will tinker with the replication stream in a way that will stop all the slaves. Should be called under RPCWrapLockAction.

func (*ActionAgent) BroadcastHealthStreamReply

func (agent *ActionAgent) BroadcastHealthStreamReply(hsr *actionnode.HealthStreamReply)

BroadcastHealthStreamReply will send the HealthStreamReply to all listening clients.

func (*ActionAgent) ChangeType

func (agent *ActionAgent) ChangeType(ctx context.Context, tabletType topo.TabletType) error

ChangeType changes the tablet type Should be called under RPCWrapLockAction.

func (*ActionAgent) DemoteMaster

func (agent *ActionAgent) DemoteMaster(ctx context.Context) error

DemoteMaster demotes the current master, and marks it read-only in the topo. Should be called under RPCWrapLockAction.

func (*ActionAgent) DisableQueryService

func (agent *ActionAgent) DisableQueryService() bool

DisableQueryService reads the DisableQueryService field from the TabletControl record (if any) stored in the agent, protected by mutex.

func (*ActionAgent) ExecuteFetch

func (agent *ActionAgent) ExecuteFetch(ctx context.Context, query string, maxrows int, wantFields, disableBinlogs bool, dbconfigName dbconfigs.DbConfigName) (*proto.QueryResult, error)

ExecuteFetch will execute the given query, possibly disabling binlogs. Should be called under RPCWrap.

func (*ActionAgent) ExecuteHook

func (agent *ActionAgent) ExecuteHook(ctx context.Context, hk *hook.Hook) *hook.HookResult

ExecuteHook executes the provided hook locally, and returns the result. Should be called under RPCWrapLockAction.

func (*ActionAgent) GetPermissions

func (agent *ActionAgent) GetPermissions(ctx context.Context) (*myproto.Permissions, error)

GetPermissions returns the db permissions. Should be called under RPCWrap.

func (*ActionAgent) GetSchema

func (agent *ActionAgent) GetSchema(ctx context.Context, tables, excludeTables []string, includeViews bool) (*myproto.SchemaDefinition, error)

GetSchema returns the schema. Should be called under RPCWrap.

func (*ActionAgent) GetSlaves

func (agent *ActionAgent) GetSlaves(ctx context.Context) ([]string, error)

GetSlaves returns the address of all the slaves Should be called under RPCWrap.

func (*ActionAgent) HealthStreamMapSize

func (agent *ActionAgent) HealthStreamMapSize() int

HealthStreamMapSize returns the size of the healthStreamMap (used for tests).

func (*ActionAgent) Healthy

func (agent *ActionAgent) Healthy() (time.Duration, error)

Healthy reads the result of the latest healthcheck, protected by mutex.

func (*ActionAgent) InitTablet

func (agent *ActionAgent) InitTablet(port, securePort int) error

InitTablet initializes the tablet record if necessary.

func (*ActionAgent) IsRunningHealthCheck

func (agent *ActionAgent) IsRunningHealthCheck() bool

IsRunningHealthCheck indicates if the agent is configured to run healthchecks.

func (*ActionAgent) MasterPosition

func (agent *ActionAgent) MasterPosition(ctx context.Context) (myproto.ReplicationPosition, error)

MasterPosition returns the master position Should be called under RPCWrap.

func (*ActionAgent) Ping

func (agent *ActionAgent) Ping(ctx context.Context, args string) string

Ping makes sure RPCs work, and refreshes the tablet record. Should be called under RPCWrap.

func (*ActionAgent) PreflightSchema

func (agent *ActionAgent) PreflightSchema(ctx context.Context, change string) (*myproto.SchemaChangeResult, error)

PreflightSchema will try out the schema change Should be called under RPCWrapLockAction.

func (*ActionAgent) PromoteSlave

func (agent *ActionAgent) PromoteSlave(ctx context.Context) (*actionnode.RestartSlaveData, error)

PromoteSlave transforms the current tablet from a slave to a master. It returns the data needed for other tablets to become a slave. Should be called under RPCWrapLockAction.

func (*ActionAgent) RPCWrap

func (agent *ActionAgent) RPCWrap(ctx context.Context, name string, args, reply interface{}, f func() error) error

RPCWrap is for read-only actions that can be executed concurrently. verbose is forced to false.

func (*ActionAgent) RPCWrapLock

func (agent *ActionAgent) RPCWrapLock(ctx context.Context, name string, args, reply interface{}, verbose bool, f func() error) error

RPCWrapLock is for actions that should not run concurrently with each other.

func (*ActionAgent) RPCWrapLockAction

func (agent *ActionAgent) RPCWrapLockAction(ctx context.Context, name string, args, reply interface{}, verbose bool, f func() error) error

RPCWrapLockAction is the same as RPCWrapLock, plus it will call refreshTablet after the action returns.

func (*ActionAgent) RefreshState

func (agent *ActionAgent) RefreshState(ctx context.Context)

RefreshState reload the tablet record from the topo server. Should be called under RPCWrapLockAction, so it actually works.

func (*ActionAgent) RegisterHealthStream

func (agent *ActionAgent) RegisterHealthStream(c chan<- *actionnode.HealthStreamReply) (int, error)

RegisterHealthStream adds a health stream channel to our list

func (*ActionAgent) ReloadSchema

func (agent *ActionAgent) ReloadSchema(ctx context.Context)

ReloadSchema will reload the schema Should be called under RPCWrapLockAction.

func (*ActionAgent) ReparentPosition

ReparentPosition returns the RestartSlaveData for the provided ReplicationPosition. Should be called under RPCWrap.

func (*ActionAgent) ReserveForRestore

func (agent *ActionAgent) ReserveForRestore(ctx context.Context, args *actionnode.ReserveForRestoreArgs) error

ReserveForRestore reserves the current tablet for an upcoming restore operation. Should be called under RPCWrapLockAction.

func (*ActionAgent) RestartSlave

func (agent *ActionAgent) RestartSlave(ctx context.Context, rsd *actionnode.RestartSlaveData) error

RestartSlave tells the tablet it has a new master Should be called under RPCWrapLockAction.

func (*ActionAgent) Restore

func (agent *ActionAgent) Restore(ctx context.Context, args *actionnode.RestoreArgs, logger logutil.Logger) error

Restore stops the tablet's mysqld, replaces its data folder with a snapshot, and then restarts it.

Check that the SnapshotManifest is valid and the master has not changed. Shutdown mysqld. Load the snapshot from source tablet. Restart mysqld and replication. Put tablet into the replication graph as a spare. Should be called under RPCWrapLockAction.

func (*ActionAgent) RunBlpUntil

func (agent *ActionAgent) RunBlpUntil(ctx context.Context, bpl *blproto.BlpPositionList, waitTime time.Duration) (*myproto.ReplicationPosition, error)

RunBlpUntil runs the binlog player server until the position is reached, and returns the current mysql master replication position.

func (*ActionAgent) RunHealthCheck

func (agent *ActionAgent) RunHealthCheck(ctx context.Context, targetTabletType topo.TabletType)

RunHealthCheck will manually run the health check on the tablet Should be called under RPCWrap.

func (*ActionAgent) Scrap

func (agent *ActionAgent) Scrap(ctx context.Context) error

Scrap scraps the live running tablet Should be called under RPCWrapLockAction.

func (*ActionAgent) SetReadOnly

func (agent *ActionAgent) SetReadOnly(ctx context.Context, rdonly bool) error

SetReadOnly makes the mysql instance read-only or read-write Should be called under RPCWrapLockAction.

func (*ActionAgent) SlaveStatus

func (agent *ActionAgent) SlaveStatus(ctx context.Context) (*myproto.ReplicationStatus, error)

SlaveStatus returns the replication status Should be called under RPCWrap.

func (*ActionAgent) SlaveWasPromoted

func (agent *ActionAgent) SlaveWasPromoted(ctx context.Context) error

SlaveWasPromoted promotes a slave to master, no questions asked. Should be called under RPCWrapLockAction.

func (*ActionAgent) SlaveWasRestarted

func (agent *ActionAgent) SlaveWasRestarted(ctx context.Context, swrd *actionnode.SlaveWasRestartedArgs) error

SlaveWasRestarted updates the parent record for a tablet. Should be called under RPCWrapLockAction.

func (*ActionAgent) Sleep

func (agent *ActionAgent) Sleep(ctx context.Context, duration time.Duration)

Sleep sleeps for the duration Should be called under RPCWrapLockAction.

func (*ActionAgent) Snapshot

Snapshot takes a db snapshot Should be called under RPCWrapLockAction.

func (*ActionAgent) SnapshotSourceEnd

func (agent *ActionAgent) SnapshotSourceEnd(ctx context.Context, args *actionnode.SnapshotSourceEndArgs) error

SnapshotSourceEnd restores the state of the server after a Snapshot(server_mode =true) Should be called under RPCWrapLockAction.

func (*ActionAgent) Start

func (agent *ActionAgent) Start(mysqlPort, vtPort, vtsPort int) error

Start validates and updates the topology records for the tablet, and performs the initial state change callback to start tablet services.

func (*ActionAgent) StartBlp

func (agent *ActionAgent) StartBlp(ctx context.Context) error

StartBlp starts the binlog players Should be called under RPCWrapLockAction.

func (*ActionAgent) StartSlave

func (agent *ActionAgent) StartSlave(ctx context.Context) error

StartSlave will start the replication Should be called under RPCWrapLock.

func (*ActionAgent) Stop

func (agent *ActionAgent) Stop()

Stop shutdowns this agent.

func (*ActionAgent) StopBlp

func (agent *ActionAgent) StopBlp(ctx context.Context) (*blproto.BlpPositionList, error)

StopBlp stops the binlog players, and return their positions. Should be called under RPCWrapLockAction.

func (*ActionAgent) StopSlave

func (agent *ActionAgent) StopSlave(ctx context.Context) error

StopSlave will stop the replication Should be called under RPCWrapLock.

func (*ActionAgent) StopSlaveMinimum

func (agent *ActionAgent) StopSlaveMinimum(ctx context.Context, position myproto.ReplicationPosition, waitTime time.Duration) (*myproto.ReplicationStatus, error)

StopSlaveMinimum will stop the slave after it reaches at least the provided position.

func (*ActionAgent) Tablet

func (agent *ActionAgent) Tablet() *topo.TabletInfo

Tablet reads the stored TabletInfo from the agent, protected by mutex.

func (*ActionAgent) TabletExternallyReparented

func (agent *ActionAgent) TabletExternallyReparented(ctx context.Context, externalID string) error

TabletExternallyReparented updates all topo records so the current tablet is the new master for this shard. Should be called under RPCWrapLock.

func (*ActionAgent) UnregisterHealthStream

func (agent *ActionAgent) UnregisterHealthStream(id int) error

UnregisterHealthStream removes a health stream channel from our list

func (*ActionAgent) WaitBlpPosition

func (agent *ActionAgent) WaitBlpPosition(ctx context.Context, blpPosition *blproto.BlpPosition, waitTime time.Duration) error

WaitBlpPosition waits until a specific filtered replication position is reached. Should be called under RPCWrapLock.

func (*ActionAgent) WaitSlavePosition

func (agent *ActionAgent) WaitSlavePosition(ctx context.Context, position myproto.ReplicationPosition, waitTimeout time.Duration) (*myproto.ReplicationStatus, error)

WaitSlavePosition waits until we reach the provided position, and returns the current position Should be called under RPCWrapLock.

type BinlogPlayerController

type BinlogPlayerController struct {
	// contains filtered or unexported fields
}

BinlogPlayerController controls one player

func (*BinlogPlayerController) BlpPosition

func (bpc *BinlogPlayerController) BlpPosition(vtClient *binlogplayer.DBClient) (*blproto.BlpPosition, string, error)

BlpPosition returns the current position for a controller, as read from the database.

func (*BinlogPlayerController) Iteration

func (bpc *BinlogPlayerController) Iteration() (err error)

Iteration is a single iteration for the player: get the current status, try to play, and plays until interrupted, or until an error occurs.

func (*BinlogPlayerController) Loop

func (bpc *BinlogPlayerController) Loop()

Loop runs the main player loop: try to play, and in case of error, sleep for 5 seconds and try again.

func (*BinlogPlayerController) Start

func (bpc *BinlogPlayerController) Start()

Start will start the player in the background and run forever

func (*BinlogPlayerController) StartUntil

func (bpc *BinlogPlayerController) StartUntil(stopPos myproto.ReplicationPosition) error

StartUntil will start the Player until we reach the given position.

func (*BinlogPlayerController) Stop

func (bpc *BinlogPlayerController) Stop()

Stop will ask the controller to stop playing, and wait until it is stopped.

func (*BinlogPlayerController) String

func (bpc *BinlogPlayerController) String() string

func (*BinlogPlayerController) WaitForStop

func (bpc *BinlogPlayerController) WaitForStop(waitTimeout time.Duration) error

WaitForStop will wait until the player is stopped. Use this after StartUntil.

type BinlogPlayerControllerStatus

type BinlogPlayerControllerStatus struct {
	// configuration values
	Index        uint32
	SourceShard  topo.SourceShard
	StopPosition myproto.ReplicationPosition

	// stats and current values
	LastPosition        myproto.ReplicationPosition
	SecondsBehindMaster int64
	Counts              map[string]int64
	Rates               map[string][]float64
	State               string
	SourceTablet        topo.TabletAlias
	LastError           string
}

BinlogPlayerControllerStatus is the status of an individual controller

type BinlogPlayerControllerStatusList

type BinlogPlayerControllerStatusList []*BinlogPlayerControllerStatus

BinlogPlayerControllerStatusList is the list of statuses

func (BinlogPlayerControllerStatusList) Len

Len is part of sort.Interface

func (BinlogPlayerControllerStatusList) Less

func (bpcsl BinlogPlayerControllerStatusList) Less(i, j int) bool

Less is part of sort.Interface

func (BinlogPlayerControllerStatusList) Swap

func (bpcsl BinlogPlayerControllerStatusList) Swap(i, j int)

Swap is part of sort.Interface

type BinlogPlayerMap

type BinlogPlayerMap struct {
	// contains filtered or unexported fields
}

BinlogPlayerMap controls all the players. It can be stopped and restarted.

func NewBinlogPlayerMap

func NewBinlogPlayerMap(ts topo.Server, dbConfig *mysql.ConnectionParams, mysqld *mysqlctl.Mysqld) *BinlogPlayerMap

NewBinlogPlayerMap creates a new map of players

func (*BinlogPlayerMap) BlpPositionList

func (blm *BinlogPlayerMap) BlpPositionList() (*blproto.BlpPositionList, error)

BlpPositionList returns the current position of all the players

func (*BinlogPlayerMap) RefreshMap

func (blm *BinlogPlayerMap) RefreshMap(tablet *topo.Tablet, keyspaceInfo *topo.KeyspaceInfo, shardInfo *topo.ShardInfo)

RefreshMap reads the right data from topo.Server and makes sure we're playing the right logs.

func (*BinlogPlayerMap) RunUntil

func (blm *BinlogPlayerMap) RunUntil(blpPositionList *blproto.BlpPositionList, waitTimeout time.Duration) error

RunUntil will run all the players until they reach the given position. Holds the map lock during that exercise, shouldn't take long at all.

func (*BinlogPlayerMap) Start

func (blm *BinlogPlayerMap) Start()

Start restarts the current players

func (*BinlogPlayerMap) Status

func (blm *BinlogPlayerMap) Status() *BinlogPlayerMapStatus

Status returns the BinlogPlayerMapStatus for the BinlogPlayerMap

func (*BinlogPlayerMap) Stop

func (blm *BinlogPlayerMap) Stop()

Stop stops the current players, but does not remove them from the map. Call 'Start' to restart the playback.

func (*BinlogPlayerMap) StopAllPlayersAndReset

func (blm *BinlogPlayerMap) StopAllPlayersAndReset()

StopAllPlayersAndReset stops all the binlog players, and reset the map of players.

type BinlogPlayerMapStatus

type BinlogPlayerMapStatus struct {
	State       string
	Controllers BinlogPlayerControllerStatusList
}

BinlogPlayerMapStatus is the complete player status

type HealthRecord

type HealthRecord struct {
	Error            error
	ReplicationDelay time.Duration
	Time             time.Time
}

HealthRecord records one run of the health checker

func (*HealthRecord) Class

func (r *HealthRecord) Class() string

Class returns a human-readable one word version of the health state.

func (*HealthRecord) HTML

func (r *HealthRecord) HTML() template.HTML

HTML returns a html version to be displayed on UIs

func (*HealthRecord) IsDuplicate

func (r *HealthRecord) IsDuplicate(other interface{}) bool

IsDuplicate implements history.Deduplicable

type RPCAgent

type RPCAgent interface {
	Ping(ctx context.Context, args string) string

	GetSchema(ctx context.Context, tables, excludeTables []string, includeViews bool) (*myproto.SchemaDefinition, error)

	GetPermissions(ctx context.Context) (*myproto.Permissions, error)

	SetReadOnly(ctx context.Context, rdonly bool) error

	ChangeType(ctx context.Context, tabletType topo.TabletType) error

	Scrap(ctx context.Context) error

	Sleep(ctx context.Context, duration time.Duration)

	ExecuteHook(ctx context.Context, hk *hook.Hook) *hook.HookResult

	RefreshState(ctx context.Context)

	RunHealthCheck(ctx context.Context, targetTabletType topo.TabletType)

	RegisterHealthStream(chan<- *actionnode.HealthStreamReply) (int, error)
	UnregisterHealthStream(int) error

	ReloadSchema(ctx context.Context)

	PreflightSchema(ctx context.Context, change string) (*myproto.SchemaChangeResult, error)

	ApplySchema(ctx context.Context, change *myproto.SchemaChange) (*myproto.SchemaChangeResult, error)

	ExecuteFetch(ctx context.Context, query string, maxrows int, wantFields, disableBinlogs bool, dbconfigName dbconfigs.DbConfigName) (*proto.QueryResult, error)

	SlaveStatus(ctx context.Context) (*myproto.ReplicationStatus, error)

	WaitSlavePosition(ctx context.Context, position myproto.ReplicationPosition, waitTimeout time.Duration) (*myproto.ReplicationStatus, error)

	MasterPosition(ctx context.Context) (myproto.ReplicationPosition, error)

	ReparentPosition(ctx context.Context, rp *myproto.ReplicationPosition) (*actionnode.RestartSlaveData, error)

	StopSlave(ctx context.Context) error

	StopSlaveMinimum(ctx context.Context, position myproto.ReplicationPosition, waitTime time.Duration) (*myproto.ReplicationStatus, error)

	StartSlave(ctx context.Context) error

	TabletExternallyReparented(ctx context.Context, externalID string) error

	GetSlaves(ctx context.Context) ([]string, error)

	WaitBlpPosition(ctx context.Context, blpPosition *blproto.BlpPosition, waitTime time.Duration) error

	StopBlp(ctx context.Context) (*blproto.BlpPositionList, error)

	StartBlp(ctx context.Context) error

	RunBlpUntil(ctx context.Context, bpl *blproto.BlpPositionList, waitTime time.Duration) (*myproto.ReplicationPosition, error)

	DemoteMaster(ctx context.Context) error

	PromoteSlave(ctx context.Context) (*actionnode.RestartSlaveData, error)

	SlaveWasPromoted(ctx context.Context) error

	RestartSlave(ctx context.Context, rsd *actionnode.RestartSlaveData) error

	SlaveWasRestarted(ctx context.Context, swrd *actionnode.SlaveWasRestartedArgs) error

	BreakSlaves(ctx context.Context) error

	Snapshot(ctx context.Context, args *actionnode.SnapshotArgs, logger logutil.Logger) (*actionnode.SnapshotReply, error)

	SnapshotSourceEnd(ctx context.Context, args *actionnode.SnapshotSourceEndArgs) error

	ReserveForRestore(ctx context.Context, args *actionnode.ReserveForRestoreArgs) error

	Restore(ctx context.Context, args *actionnode.RestoreArgs, logger logutil.Logger) error

	// RPC helpers
	RPCWrap(ctx context.Context, name string, args, reply interface{}, f func() error) error
	RPCWrapLock(ctx context.Context, name string, args, reply interface{}, verbose bool, f func() error) error
	RPCWrapLockAction(ctx context.Context, name string, args, reply interface{}, verbose bool, f func() error) error
}

RPCAgent defines the interface implemented by the Agent for RPCs. It is useful for RPC implementations to test their full stack.

type RegisterQueryService

type RegisterQueryService func(*ActionAgent)

RegisterQueryService is used to delay registration of RPC servers until we have all the objects.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL