wrangler

package
v0.0.0-...-bc668d9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 3, 2015 License: BSD-3-Clause Imports: 31 Imported by: 0

Documentation

Overview

Package wrangler contains the Wrangler object to manage complex topology actions.

Index

Constants

View Source 
const ChangeSlaveTypeActionName = "ChangeSlaveTypeAction"

ChangeSlaveTypeActionName is the name of the action to change a slave type (can be used to find such an action by name)

View Source 
const StartBlpActionName = "StartBlpAction"

StartBlpActionName is the name of the action to start binlog player

View Source 
const StartSlaveActionName = "StartSlaveAction"

StartSlaveActionName is the name of the slave start action

View Source 
const TabletTagActionName = "TabletTagAction"

TabletTagActionName is the name of the Tag action

Variables

View Source 
var (
	// DefaultActionTimeout is a good default for interactive
	// remote actions. We usually take a lock then do an action,
	// so basing this to be greater than DefaultLockTimeout is good.
	// Use this as the default value for Context that need a deadline.
	DefaultActionTimeout = actionnode.DefaultLockTimeout * 4
)

Functions

func RecordChangeSlaveTypeAction 

func RecordChangeSlaveTypeAction(cleaner *Cleaner, tabletAlias topo.TabletAlias, tabletType topo.TabletType)

RecordChangeSlaveTypeAction records a new ChangeSlaveTypeAction into the specified Cleaner

func RecordStartBlpAction 

func RecordStartBlpAction(cleaner *Cleaner, tabletInfo *topo.TabletInfo)

RecordStartBlpAction records a new StartBlpAction into the specified Cleaner

func RecordStartSlaveAction 

func RecordStartSlaveAction(cleaner *Cleaner, tabletInfo *topo.TabletInfo)

RecordStartSlaveAction records a new StartSlaveAction into the specified Cleaner

func RecordTabletTagAction 

func RecordTabletTagAction(cleaner *Cleaner, tabletAlias topo.TabletAlias, name, value string)

RecordTabletTagAction records a new TabletTagAction into the specified Cleaner

Types

type ChangeSlaveTypeAction 

type ChangeSlaveTypeAction struct {
	TabletAlias topo.TabletAlias
	TabletType  topo.TabletType
}

ChangeSlaveTypeAction will change a server type to another type

func FindChangeSlaveTypeActionByTarget 

func FindChangeSlaveTypeActionByTarget(cleaner *Cleaner, tabletAlias topo.TabletAlias) (*ChangeSlaveTypeAction, error)

FindChangeSlaveTypeActionByTarget finds the first action for the target

func (ChangeSlaveTypeAction) CleanUp 

func (csta ChangeSlaveTypeAction) CleanUp(ctx context.Context, wr *Wrangler) error

CleanUp is part of CleanerAction interface.

type Cleaner 

type Cleaner struct {
	// contains filtered or unexported fields
}

Cleaner remembers a list of cleanup steps to perform. Just record action cleanup steps, and execute them at the end in reverse order, with various guarantees.

func (*Cleaner) CleanUp 

func (cleaner *Cleaner) CleanUp(wr *Wrangler) error

CleanUp will run the recorded actions. If an action on a target fails, it will not run the next action on the same target. We return the aggregate errors for all cleanups. CleanUp uses its own context, with a timeout of 5 minutes, so that clean up action will run even if the original context times out. TODO(alainjobart) Actions should run concurrently on a per target basis. They are then serialized on each target.

func (*Cleaner) GetActionByName 

func (cleaner *Cleaner) GetActionByName(name, target string) (CleanerAction, error)

GetActionByName returns the first action in the list with the given name and target

func (*Cleaner) Record 

func (cleaner *Cleaner) Record(name, target string, action CleanerAction)

Record will add a cleaning action to the list

func (*Cleaner) RemoveActionByName 

func (cleaner *Cleaner) RemoveActionByName(name, target string) error

RemoveActionByName removes an action from the cleaner list

type CleanerAction 

type CleanerAction interface {
	CleanUp(context.Context, *Wrangler) error
}

CleanerAction is the interface that clean-up actions need to implement

type LegacyZknsAddrs 

type LegacyZknsAddrs struct {
	Endpoints []string `json:"endpoints"`
}

LegacyZknsAddrs is what we write to ZK to use for zkns

type StartBlpAction 

type StartBlpAction struct {
	TabletInfo *topo.TabletInfo
}

StartBlpAction will restart binlog replication on a server

func (StartBlpAction) CleanUp 

func (sba StartBlpAction) CleanUp(ctx context.Context, wr *Wrangler) error

CleanUp is part of CleanerAction interface.

type StartSlaveAction 

type StartSlaveAction struct {
	TabletInfo *topo.TabletInfo
}

StartSlaveAction will restart binlog replication on a server

func (StartSlaveAction) CleanUp 

func (sba StartSlaveAction) CleanUp(ctx context.Context, wr *Wrangler) error

CleanUp is part of CleanerAction interface.

type TabletTagAction 

type TabletTagAction struct {
	TabletAlias topo.TabletAlias
	Name        string
	Value       string
}

TabletTagAction will add / remove a tag to a tablet. If Value is empty, will remove the tag.

func (TabletTagAction) CleanUp 

func (tta TabletTagAction) CleanUp(ctx context.Context, wr *Wrangler) error

CleanUp is part of CleanerAction interface.

type Wrangler 

type Wrangler struct {
	// contains filtered or unexported fields
}

Wrangler manages complex actions on the topology, like reparents, snapshots, restores, ...

FIXME(alainjobart) take the context out of this structure. We want the context to come from the outside on every call.

Multiple go routines can use the same Wrangler at the same time, provided they want to share the same logger / topo server / lock timeout.

func New 

func New(logger logutil.Logger, ts topo.Server, lockTimeout time.Duration) *Wrangler

New creates a new Wrangler object.

lockTimeout: how long should we wait for the initial lock to start a complex action? This is distinct from the context timeout because most of the time, we want to immediately know that our action will fail. However, automated action will need some time to arbitrate the locks.

func (*Wrangler) ApplySchema 

func (wr *Wrangler) ApplySchema(ctx context.Context, tabletAlias topo.TabletAlias, sc *myproto.SchemaChange) (*myproto.SchemaChangeResult, error)

ApplySchema will apply a schema change on the remote tablet.

func (*Wrangler) ApplySchemaKeyspace 

func (wr *Wrangler) ApplySchemaKeyspace(ctx context.Context, keyspace string, change string, simple, force bool, waitSlaveTimeout time.Duration) (*myproto.SchemaChangeResult, error)

ApplySchemaKeyspace applies a schema change to an entire keyspace. take a keyspace lock to do this. first we will validate the Preflight works the same on all shard masters and fail if not (unless force is specified) if simple, we just do it on all masters. if complex, we do the shell game in parallel on all shards

func (*Wrangler) ApplySchemaShard 

func (wr *Wrangler) ApplySchemaShard(ctx context.Context, keyspace, shard, change string, newParentTabletAlias topo.TabletAlias, simple, force bool, waitSlaveTimeout time.Duration) (*myproto.SchemaChangeResult, error)

ApplySchemaShard applies a shcema change on a shard. Note for 'complex' mode (the 'simple' mode is easy enough that we don't need to handle recovery that much): this method is able to recover if interrupted in the middle, because it knows which server has the schema change already applied, and will just pass through them very quickly.

func (*Wrangler) ChangeType 

func (wr *Wrangler) ChangeType(ctx context.Context, tabletAlias topo.TabletAlias, tabletType topo.TabletType, force bool) error

ChangeType changes the type of tablet and recompute all necessary derived paths in the serving graph. If force is true, it will bypass the RPC action system and make the data change directly, and not run the remote hooks.

Note we don't update the master record in the Shard here, as we can't ChangeType from and out of master anyway.

func (*Wrangler) ChangeTypeNoRebuild 

func (wr *Wrangler) ChangeTypeNoRebuild(ctx context.Context, tabletAlias topo.TabletAlias, tabletType topo.TabletType, force bool) (rebuildRequired bool, cell, keyspace, shard string, err error)

ChangeTypeNoRebuild changes a tablet's type, and returns whether there's a shard that should be rebuilt, along with its cell, keyspace, and shard. If force is true, it will bypass the RPC action system and make the data change directly, and not run the remote hooks.

Note we don't update the master record in the Shard here, as we can't ChangeType from and out of master anyway.

func (*Wrangler) Clone 

func (wr *Wrangler) Clone(ctx context.Context, srcTabletAlias topo.TabletAlias, dstTabletAliases []topo.TabletAlias, forceMasterSnapshot bool, snapshotConcurrency, fetchConcurrency, fetchRetryCount int, serverMode bool) error

Clone will do all the necessary actions to copy all the data from a source to a set of destinations.

func (*Wrangler) CopySchemaShard 

func (wr *Wrangler) CopySchemaShard(ctx context.Context, srcTabletAlias topo.TabletAlias, tables, excludeTables []string, includeViews bool, keyspace, shard string) error

CopySchemaShard copies the schema from a source tablet to the specified shard. The schema is applied directly on the master of the destination shard, and is propogated to the replicas through binlogs.

func (*Wrangler) DeleteShard 

func (wr *Wrangler) DeleteShard(ctx context.Context, keyspace, shard string) error

DeleteShard will do all the necessary changes in the topology server to entirely remove a shard. It can only work if there are no tablets in that shard.

func (*Wrangler) DeleteTablet 

func (wr *Wrangler) DeleteTablet(tabletAlias topo.TabletAlias) error

DeleteTablet will get the tablet record, and if it's scrapped, will delete the record from the topology.

func (*Wrangler) ExecuteFetch 

func (wr *Wrangler) ExecuteFetch(ctx context.Context, tabletAlias topo.TabletAlias, query string, maxRows int, wantFields, disableBinlogs bool) (*mproto.QueryResult, error)

ExecuteFetch will get data from a remote tablet

func (*Wrangler) ExecuteHook 

func (wr *Wrangler) ExecuteHook(ctx context.Context, tabletAlias topo.TabletAlias, hook *hk.Hook) (hookResult *hk.HookResult, err error)

ExecuteHook will run the hook on the tablet

func (*Wrangler) ExecuteOptionalTabletInfoHook 

func (wr *Wrangler) ExecuteOptionalTabletInfoHook(ctx context.Context, ti *topo.TabletInfo, hook *hk.Hook) (err error)

ExecuteOptionalTabletInfoHook executes a hook and returns an error only if the hook failed, not if the hook doesn't exist.

func (*Wrangler) ExecuteTabletInfoHook 

func (wr *Wrangler) ExecuteTabletInfoHook(ctx context.Context, ti *topo.TabletInfo, hook *hk.Hook) (hookResult *hk.HookResult, err error)

ExecuteTabletInfoHook will run the hook on the tablet described by TabletInfo

func (*Wrangler) ExportZkns 

func (wr *Wrangler) ExportZkns(cell string) error

ExportZkns exports addresses from the VT serving graph to a legacy zkns server. Note these functions only work with a zktopo.

func (*Wrangler) ExportZknsForKeyspace 

func (wr *Wrangler) ExportZknsForKeyspace(ctx context.Context, keyspace string) error

ExportZknsForKeyspace exports addresses from the VT serving graph to a legacy zkns server.

func (*Wrangler) GetPermissions 

func (wr *Wrangler) GetPermissions(ctx context.Context, tabletAlias topo.TabletAlias) (*myproto.Permissions, error)

GetPermissions returns the permissions set on a remote tablet

func (*Wrangler) GetSchema 

func (wr *Wrangler) GetSchema(ctx context.Context, tabletAlias topo.TabletAlias, tables, excludeTables []string, includeViews bool) (*myproto.SchemaDefinition, error)

GetSchema uses an RPC to get the schema from a remote tablet

func (*Wrangler) GetVersion 

func (wr *Wrangler) GetVersion(tabletAlias topo.TabletAlias) (string, error)

GetVersion returns the version string from a tablet

func (*Wrangler) InitTablet 

func (wr *Wrangler) InitTablet(ctx context.Context, tablet *topo.Tablet, force, createShardAndKeyspace, update bool) error

InitTablet creates or updates a tablet. If no parent is specified in the tablet, and the tablet has a slave type, we will find the appropriate parent. If createShardAndKeyspace is true and the parent keyspace or shard don't exist, they will be created. If update is true, and a tablet with the same ID exists, update it. If Force is true, and a tablet with the same ID already exists, it will be scrapped and deleted, and then recreated.

func (*Wrangler) Logger 

func (wr *Wrangler) Logger() logutil.Logger

Logger returns the logger associated with this wrangler.

func (*Wrangler) MigrateServedFrom 

func (wr *Wrangler) MigrateServedFrom(ctx context.Context, keyspace, shard string, servedType topo.TabletType, cells []string, reverse bool, filteredReplicationWaitTime time.Duration) error

MigrateServedFrom is used during vertical splits to migrate a served type from a keyspace to another.

func (*Wrangler) MigrateServedTypes 

func (wr *Wrangler) MigrateServedTypes(ctx context.Context, keyspace, shard string, cells []string, servedType topo.TabletType, reverse, skipReFreshState bool, filteredReplicationWaitTime time.Duration) error

MigrateServedTypes is used during horizontal splits to migrate a served type from a list of shards to another.

func (*Wrangler) PreflightSchema 

func (wr *Wrangler) PreflightSchema(ctx context.Context, tabletAlias topo.TabletAlias, change string) (*myproto.SchemaChangeResult, error)

PreflightSchema will try a schema change on the remote tablet.

func (*Wrangler) RebuildKeyspaceGraph 

func (wr *Wrangler) RebuildKeyspaceGraph(ctx context.Context, keyspace string, cells []string) error

RebuildKeyspaceGraph rebuilds the serving graph data while locking out other changes. If some shards were recently read / updated, pass them in the cache so we don't read them again (and possible get stale replicated data)

func (*Wrangler) RebuildReplicationGraph 

func (wr *Wrangler) RebuildReplicationGraph(ctx context.Context, cells []string, keyspaces []string) error

RebuildReplicationGraph is a quick and dirty tool to resurrect the TopologyServer data from the canonical data stored in the tablet nodes.

cells: local vt cells to scan for all tablets keyspaces: list of keyspaces to rebuild

func (*Wrangler) RebuildShardGraph 

func (wr *Wrangler) RebuildShardGraph(ctx context.Context, keyspace, shard string, cells []string) (*topo.ShardInfo, error)

RebuildShardGraph rebuilds the serving and replication rollup data data while locking out other changes.

func (*Wrangler) RefreshTablesByShard 

func (wr *Wrangler) RefreshTablesByShard(ctx context.Context, si *topo.ShardInfo, tabletType topo.TabletType, cells []string) error

RefreshTablesByShard calls RefreshState on all the tables of a given type in a shard. It would work for the master, but the discovery wouldn't be very efficient.

func (*Wrangler) ReloadSchema 

func (wr *Wrangler) ReloadSchema(ctx context.Context, tabletAlias topo.TabletAlias) error

ReloadSchema forces the remote tablet to reload its schema.

func (*Wrangler) RemoveShardCell 

func (wr *Wrangler) RemoveShardCell(ctx context.Context, keyspace, shard, cell string, force bool) error

RemoveShardCell will remove a cell from the Cells list in a shard. It will first check the shard has no tablets there. if 'force' is specified, it will remove the cell even when the tablet map cannot be retrieved. This is intended to be used when a cell is completely down and its topology server cannot even be reached.

func (*Wrangler) ReparentShard 

func (wr *Wrangler) ReparentShard(ctx context.Context, keyspace, shard string, masterElectTabletAlias topo.TabletAlias, leaveMasterReadOnly, forceReparentToCurrentMaster bool, waitSlaveTimeout time.Duration) error

ReparentShard creates the reparenting action and launches a goroutine to coordinate the procedure.

leaveMasterReadOnly: leave the master in read-only mode, even 

though all the other necessary updates have been made.

forceReparentToCurrentMaster: mostly for test setups, this can 

cause data loss.

func (*Wrangler) ReparentTablet 

func (wr *Wrangler) ReparentTablet(ctx context.Context, tabletAlias topo.TabletAlias) error

ReparentTablet attempts to reparent this tablet to the current master, based on the current replication position. If there is no match, it will fail.

func (*Wrangler) ReserveForRestore 

func (wr *Wrangler) ReserveForRestore(ctx context.Context, srcTabletAlias, dstTabletAlias topo.TabletAlias) (err error)

ReserveForRestore will make sure a tablet is ready to be used as a restore target.

func (*Wrangler) Restore 

func (wr *Wrangler) Restore(ctx context.Context, srcTabletAlias topo.TabletAlias, srcFilePath string, dstTabletAlias, parentAlias topo.TabletAlias, fetchConcurrency, fetchRetryCount int, wasReserved, dontWaitForSlaveStart bool) error

Restore actually performs the restore action on a tablet.

func (*Wrangler) Scrap 

func (wr *Wrangler) Scrap(ctx context.Context, tabletAlias topo.TabletAlias, force, skipRebuild bool) error

Scrap a tablet. If force is used, we write to topo.Server directly and don't remote-execute the command.

If we scrap the master for a shard, we will clear its record from the Shard object (only if that was the right master)

func (*Wrangler) SetKeyspaceServedFrom 

func (wr *Wrangler) SetKeyspaceServedFrom(ctx context.Context, keyspace string, servedType topo.TabletType, cells []string, sourceKeyspace string, remove bool) error

SetKeyspaceServedFrom locks a keyspace and changes its ServerFromMap

func (*Wrangler) SetKeyspaceShardingInfo 

func (wr *Wrangler) SetKeyspaceShardingInfo(ctx context.Context, keyspace, shardingColumnName string, shardingColumnType key.KeyspaceIdType, splitShardCount int32, force bool) error

SetKeyspaceShardingInfo locks a keyspace and sets its ShardingColumnName and ShardingColumnType

func (*Wrangler) SetLogger 

func (wr *Wrangler) SetLogger(logger logutil.Logger)

SetLogger can be used to change the current logger. Not synchronized, no calls to this wrangler should be in progress.

func (*Wrangler) SetShardServedTypes 

func (wr *Wrangler) SetShardServedTypes(ctx context.Context, keyspace, shard string, cells []string, servedType topo.TabletType, remove bool) error

SetShardServedTypes changes the ServedTypes parameter of a shard. It does not rebuild any serving graph or do any consistency check (yet).

func (*Wrangler) SetShardTabletControl 

func (wr *Wrangler) SetShardTabletControl(ctx context.Context, keyspace, shard string, tabletType topo.TabletType, cells []string, remove, disableQueryService bool, tables []string) error

SetShardTabletControl changes the TabletControl records for a shard. It does not rebuild any serving graph or do cross-shard consistency check.

  • if disableQueryService is set, tables has to be empty
  • if disableQueryService is not set, and tables is empty, we remove the TabletControl record for the cells

func (*Wrangler) SetSourceShards 

func (wr *Wrangler) SetSourceShards(ctx context.Context, keyspace, shard string, sources []topo.TabletAlias, tables []string) error

SetSourceShards is a utility function to override the SourceShards fields on a Shard.

func (*Wrangler) ShardReplicationStatuses 

func (wr *Wrangler) ShardReplicationStatuses(ctx context.Context, keyspace, shard string) ([]*topo.TabletInfo, []*myproto.ReplicationStatus, error)

ShardReplicationStatuses returns the ReplicationStatus for each tablet in a shard.

func (*Wrangler) Snapshot 

func (wr *Wrangler) Snapshot(ctx context.Context, tabletAlias topo.TabletAlias, forceMasterSnapshot bool, snapshotConcurrency int, serverMode bool) (*actionnode.SnapshotReply, topo.TabletType, error)

Snapshot takes a tablet snapshot.

forceMasterSnapshot: Normally a master is not a viable tablet to snapshot. However, there are degenerate cases where you need to override this, for instance the initial clone of a new master.

serverMode: if specified, the server will stop its mysqld, and be ready to serve the data files directly. Slaves can just download these and use them directly. Call SnapshotSourceEnd to return into serving mode. If not specified, the server will create an archive of the files, store them locally, and restart.

If error is nil, returns the SnapshotReply from the remote host, and the original type the server was before the snapshot.

func (*Wrangler) SnapshotSourceEnd 

func (wr *Wrangler) SnapshotSourceEnd(ctx context.Context, tabletAlias topo.TabletAlias, slaveStartRequired, readWrite bool, originalType topo.TabletType) (err error)

SnapshotSourceEnd will change the tablet back to its original type once it's done serving backups.

func (*Wrangler) SourceShardAdd 

func (wr *Wrangler) SourceShardAdd(ctx context.Context, keyspace, shard string, uid uint32, skeyspace, sshard string, keyRange key.KeyRange, tables []string) error

SourceShardAdd will add a new SourceShard inside a shard

func (*Wrangler) SourceShardDelete 

func (wr *Wrangler) SourceShardDelete(ctx context.Context, keyspace, shard string, uid uint32) error

SourceShardDelete will delete a SourceShard inside a shard, by index.

func (*Wrangler) TabletManagerClient 

func (wr *Wrangler) TabletManagerClient() tmclient.TabletManagerClient

TabletManagerClient returns the tmclient.TabletManagerClient this wrangler is using.

func (*Wrangler) TopoServer 

func (wr *Wrangler) TopoServer() topo.Server

TopoServer returns the topo.Server this wrangler is using.

func (*Wrangler) UnreserveForRestore 

func (wr *Wrangler) UnreserveForRestore(ctx context.Context, dstTabletAlias topo.TabletAlias) (err error)

UnreserveForRestore switches the tablet back to its original state, the restore won't happen.

func (*Wrangler) UnreserveForRestoreMulti 

func (wr *Wrangler) UnreserveForRestoreMulti(ctx context.Context, dstTabletAliases []topo.TabletAlias)

UnreserveForRestoreMulti calls UnreserveForRestore on all targets.

func (*Wrangler) Validate 

func (wr *Wrangler) Validate(ctx context.Context, pingTablets bool) error

Validate a whole TopologyServer tree

func (*Wrangler) ValidateKeyspace 

func (wr *Wrangler) ValidateKeyspace(ctx context.Context, keyspace string, pingTablets bool) error

ValidateKeyspace will validate a bunch of information in a keyspace is correct.

func (*Wrangler) ValidatePermissionsKeyspace 

func (wr *Wrangler) ValidatePermissionsKeyspace(ctx context.Context, keyspace string) error

ValidatePermissionsKeyspace validates all the permissions are the same in a keyspace

func (*Wrangler) ValidatePermissionsShard 

func (wr *Wrangler) ValidatePermissionsShard(ctx context.Context, keyspace, shard string) error

ValidatePermissionsShard validates all the permissions are the same in a shard

func (*Wrangler) ValidateSchemaKeyspace 

func (wr *Wrangler) ValidateSchemaKeyspace(ctx context.Context, keyspace string, excludeTables []string, includeViews bool) error

ValidateSchemaKeyspace will diff the schema from all the tablets in the keyspace.

func (*Wrangler) ValidateSchemaShard 

func (wr *Wrangler) ValidateSchemaShard(ctx context.Context, keyspace, shard string, excludeTables []string, includeViews bool) error

ValidateSchemaShard will diff the schema from all the tablets in the shard.

func (*Wrangler) ValidateShard 

func (wr *Wrangler) ValidateShard(ctx context.Context, keyspace, shard string, pingTablets bool) error

ValidateShard will validate a bunch of information in a shard is correct.

func (*Wrangler) ValidateVersionKeyspace 

func (wr *Wrangler) ValidateVersionKeyspace(ctx context.Context, keyspace string) error

ValidateVersionKeyspace validates all versions are the same in all tablets in a keyspace

func (*Wrangler) ValidateVersionShard 

func (wr *Wrangler) ValidateVersionShard(ctx context.Context, keyspace, shard string) error

ValidateVersionShard validates all versions are the same in all tablets in a shard

Source Files

View all Source files

Directories

Path Synopsis
Package testlib contains utility methods to include in unit tests to deal with topology common tasks, like fake tablets and action loops.
 Package testlib contains utility methods to include in unit tests to deal with topology common tasks, like fake tablets and action loops.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.