topotools

Documentation

Overview

Package topotools contains high level functions based on vt/topo and vt/actionnode. It should not depend on anything else that's higher level. In particular, it cannot depend on:

• vt/wrangler: much higher level, wrangler depends on topotools.
• vt/tabletmanager/initiator: we don't want the various remote protocol dependencies here.

topotools is used by wrangler, so it ends up in all tools using wrangler (vtctl, vtctld, ...). It is also included by vttablet, so it contains:

• most of the logic to create a shard / keyspace (tablet's init code)
• some of the logic to perform a TabletExternallyReparented (RPC call to primary vttablet to let it know it's the primary).

Index

• func ApplyVSchemaDDL(ksName string, ks *vschemapb.Keyspace, alterVschema *sqlparser.AlterVschema) (*vschemapb.Keyspace, error)
• func ChangeType(ctx context.Context, ts *topo.Server, tabletAlias *topodatapb.TabletAlias, ...) (*topodatapb.Tablet, error)
• func CheckOwnership(oldTablet, newTablet *topodatapb.Tablet) error
• func ConfigureTabletHook(hk *hook.Hook, tabletAlias *topodatapb.TabletAlias)
• func DeleteTablet(ctx context.Context, ts *topo.Server, tablet *topodatapb.Tablet) error
• func DoCellsHaveRdonlyTablets(ctx context.Context, ts *topo.Server, cells []string) (bool, error)
• func FindTabletByHostAndPort(tabletMap map[string]*topo.TabletInfo, addr, portName string, port int32) (*topodatapb.TabletAlias, error)
• func GetAllTabletsAcrossCells(ctx context.Context, ts *topo.Server) ([]*topo.TabletInfo, error)
• func GetKeyspaceRoutingRules(ctx context.Context, ts *topo.Server) (map[string]string, error)
• func GetKeyspaceRoutingRulesMap(rules *vschemapb.KeyspaceRoutingRules) map[string]string
• func GetRoutingRules(ctx context.Context, ts *topo.Server) (map[string][]string, error)
• func GetRoutingRulesMap(rules *vschemapb.RoutingRules) map[string][]string
• func GetShardPrimaryForTablet(ctx context.Context, ts *topo.Server, tablet *topodatapb.Tablet) (*topo.TabletInfo, error)
• func GetShardRoutingRuleKey(fromKeyspace, shard string) string
• func GetShardRoutingRules(ctx context.Context, ts *topo.Server) (map[string]string, error)
• func GetShardRoutingRulesMap(rules *vschemapb.ShardRoutingRules) map[string]string
• func GetTabletMapForCell(ctx context.Context, ts *topo.Server, cell string) (map[string]*topo.TabletInfo, error)
• func IsPrimaryTablet(ctx context.Context, ts *topo.Server, ti *topo.TabletInfo) (bool, error)
• func KeyspaceEquality(left, right *topodatapb.Keyspace) bool
• func ParseShardRoutingRuleKey(key string) (string, string)
• func RebuildKeyspace(ctx context.Context, log logutil.Logger, ts *topo.Server, keyspace string, ...) (err error)
• func RebuildKeyspaceLocked(ctx context.Context, log logutil.Logger, ts *topo.Server, keyspace string, ...) error
• func RefreshTabletsByShard(ctx context.Context, ts *topo.Server, tmc tmclient.TabletManagerClient, ...) (isPartialRefresh bool, partialRefreshDetails string, err error)
• func SaveRoutingRules(ctx context.Context, ts *topo.Server, rules map[string][]string) error
• func SaveShardRoutingRules(ctx context.Context, ts *topo.Server, srr map[string]string) error
• func SortedTabletMap(tabletMap map[string]*topo.TabletInfo) (map[string]*topo.TabletInfo, map[string]*topo.TabletInfo)
• func TabletEquality(left, right *topodatapb.Tablet) bool
• func TabletIdent(tablet *topodatapb.Tablet) string
• func UpdateKeyspaceRoutingRules(ctx context.Context, ts *topo.Server, reason string, ...) (err error)
• func UpdateShardRecords(ctx context.Context, ts *topo.Server, tmc tmclient.TabletManagerClient, ...) error
• func ValidateForReshard(sourceShards, targetShards []*topo.ShardInfo) error

Functions

func ApplyVSchemaDDL

func ApplyVSchemaDDL(ksName string, ks *vschemapb.Keyspace, alterVschema *sqlparser.AlterVschema) (*vschemapb.Keyspace, error)

ApplyVSchemaDDL applies the given DDL statement to the vschema keyspace definition and returns the modified keyspace object.

func ChangeType

func ChangeType(ctx context.Context, ts *topo.Server, tabletAlias *topodatapb.TabletAlias, newType topodatapb.TabletType, PrimaryTermStartTime *vttime.Time) (*topodatapb.Tablet, error)

ChangeType changes the type of the tablet. Make this external, since these transitions need to be forced from time to time.

If successful, the updated tablet record is returned.

func CheckOwnership

func CheckOwnership(oldTablet, newTablet *topodatapb.Tablet) error

CheckOwnership returns nil iff the Hostname and port match on oldTablet and newTablet, which implies that no other tablet process has taken over the record.

func ConfigureTabletHook

func ConfigureTabletHook(hk *hook.Hook, tabletAlias *topodatapb.TabletAlias)

ConfigureTabletHook configures the right parameters for a hook running locally on a tablet.

func DeleteTablet

func DeleteTablet(ctx context.Context, ts *topo.Server, tablet *topodatapb.Tablet) error

DeleteTablet removes a tablet record from the topology: - the replication data record if any - the tablet record

func DoCellsHaveRdonlyTablets

func DoCellsHaveRdonlyTablets(ctx context.Context, ts *topo.Server, cells []string) (bool, error)

DoCellsHaveRdonlyTablets returns true if any of the cells has at least one tablet with type RDONLY. If the slice of cells to search over is empty, it checks all cells in the topo.

func FindTabletByHostAndPort

func FindTabletByHostAndPort(tabletMap map[string]*topo.TabletInfo, addr, portName string, port int32) (*topodatapb.TabletAlias, error)

FindTabletByHostAndPort searches within a tablet map for tablets.

func GetAllTabletsAcrossCells

func GetAllTabletsAcrossCells(ctx context.Context, ts *topo.Server) ([]*topo.TabletInfo, error)

GetAllTabletsAcrossCells returns all tablets from known cells. If it returns topo.ErrPartialResult, then the list is valid, but partial.

func GetKeyspaceRoutingRules

func GetKeyspaceRoutingRules(ctx context.Context, ts *topo.Server) (map[string]string, error)

GetKeyspaceRoutingRules fetches keyspace routing rules from the topology server and returns a map of fromKeyspace=>toKeyspace.

func GetKeyspaceRoutingRulesMap

func GetKeyspaceRoutingRulesMap(rules *vschemapb.KeyspaceRoutingRules) map[string]string

GetKeyspaceRoutingRulesMap returns a map of fromKeyspace=>toKeyspace from a vschemapb.KeyspaceRoutingRules

func GetRoutingRules

func GetRoutingRules(ctx context.Context, ts *topo.Server) (map[string][]string, error)

GetRoutingRules fetches routing rules from the topology server and returns a mapping of fromTable=>[]toTables.

func GetRoutingRulesMap

func GetRoutingRulesMap(rules *vschemapb.RoutingRules) map[string][]string

func GetShardPrimaryForTablet

func GetShardPrimaryForTablet(ctx context.Context, ts *topo.Server, tablet *topodatapb.Tablet) (*topo.TabletInfo, error)

GetShardPrimaryForTablet returns the TabletInfo of the given tablet's shard's primary.

It returns an error if: - The shard does not exist in the topo. - The shard has no primary in the topo. - The shard primary does not think it is PRIMARY. - The shard primary tablet record does not match the keyspace and shard of the replica.

func GetShardRoutingRuleKey

func GetShardRoutingRuleKey(fromKeyspace, shard string) string

func GetShardRoutingRules

func GetShardRoutingRules(ctx context.Context, ts *topo.Server) (map[string]string, error)

GetShardRoutingRules fetches shard routing rules from the topology server and returns a mapping of fromKeyspace.Shard=>toKeyspace.

func GetShardRoutingRulesMap

func GetShardRoutingRulesMap(rules *vschemapb.ShardRoutingRules) map[string]string

func GetTabletMapForCell

func GetTabletMapForCell(ctx context.Context, ts *topo.Server, cell string) (map[string]*topo.TabletInfo, error)

GetTabletMapForCell returns a map of TabletInfo keyed by alias as string

func IsPrimaryTablet

func IsPrimaryTablet(ctx context.Context, ts *topo.Server, ti *topo.TabletInfo) (bool, error)

IsPrimaryTablet is a helper function to determine whether the current tablet is a primary before we allow its tablet record to be deleted. The canonical way to determine the only true primary in a shard is to list all the tablets and find the one with the highest PrimaryTermStartTime among the ones that claim to be primary.

We err on the side of caution here, i.e. we should never return false for a true primary tablet, but it is okay to return true for a tablet that isn't the true primary. This can occur if someone issues a DeleteTablet while the system is in transition (a reparenting event is in progress and parts of the topo have not yet been updated).

func KeyspaceEquality

func KeyspaceEquality(left, right *topodatapb.Keyspace) bool

KeyspaceEquality returns true iff two Keyspace fields are identical for testing purposes.

func ParseShardRoutingRuleKey

func ParseShardRoutingRuleKey(key string) (string, string)

func RebuildKeyspace

func RebuildKeyspace(ctx context.Context, log logutil.Logger, ts *topo.Server, keyspace string, cells []string, allowPartial bool) (err error)

RebuildKeyspace rebuilds the serving graph data while locking out other changes.

func RebuildKeyspaceLocked

func RebuildKeyspaceLocked(ctx context.Context, log logutil.Logger, ts *topo.Server, keyspace string, cells []string, allowPartial bool) error

RebuildKeyspaceLocked should only be used with an action lock on the keyspace - otherwise the consistency of the serving graph data can't be guaranteed.

Take data from the global keyspace and rebuild the local serving copies in each cell.

func RefreshTabletsByShard

func RefreshTabletsByShard(ctx context.Context, ts *topo.Server, tmc tmclient.TabletManagerClient, si *topo.ShardInfo, cells []string, logger logutil.Logger) (isPartialRefresh bool, partialRefreshDetails string, err error)

RefreshTabletsByShard calls RefreshState on all the tablets in a given shard.

It only returns errors from looking up the tablet map from the topology; errors returned from any RefreshState RPCs are logged and then ignored. Also, any tablets without a .Hostname set in the topology are skipped.

However, on partial errors from the topology, or errors from a RefreshState RPC will cause a boolean flag to be returned indicating only partial success along with a string detailing why we had a partial refresh.

func SaveRoutingRules

func SaveRoutingRules(ctx context.Context, ts *topo.Server, rules map[string][]string) error

SaveRoutingRules converts a mapping of fromTable=>[]toTables into a vschemapb.RoutingRules protobuf message and saves it in the topology.

func SaveShardRoutingRules

func SaveShardRoutingRules(ctx context.Context, ts *topo.Server, srr map[string]string) error

SaveShardRoutingRules converts a mapping of fromKeyspace.Shard=>toKeyspace into a vschemapb.ShardRoutingRules protobuf message and saves it in the topology.

func SortedTabletMap

func SortedTabletMap(tabletMap map[string]*topo.TabletInfo) (map[string]*topo.TabletInfo, map[string]*topo.TabletInfo)

SortedTabletMap returns two maps:

• The replicaMap contains all the non-primary non-scrapped hosts. This can be used as a list of replicas to fix up for reparenting
• The primaryMap contains all the tablets without parents (scrapped or not). This can be used to special case the old primary, and any tablet in a weird state, left over, ...

func TabletEquality

func TabletEquality(left, right *topodatapb.Tablet) bool

TabletEquality returns true iff two Tablets are identical for testing purposes

func TabletIdent

func TabletIdent(tablet *topodatapb.Tablet) string

TabletIdent returns a concise string representation of this tablet.

func UpdateKeyspaceRoutingRules

func UpdateKeyspaceRoutingRules(ctx context.Context, ts *topo.Server, reason string,
	update func(ctx context.Context, rules *map[string]string) error) (err error)

UpdateKeyspaceRoutingRules updates the keyspace routing rules in the topo server. If the keyspace routing rules do not yet exist, it will create them. If multiple callers are racing to create the initial keyspace routing rules then the first writer will win and the other callers can immediately retry when getting the resulting topo.NodeExists error. When the routing rules already exist, it will acquire a RoutingRulesLock and then modify the keyspace routing rules in-place.

func UpdateShardRecords

func UpdateShardRecords(
	ctx context.Context,
	ts *topo.Server,
	tmc tmclient.TabletManagerClient,
	keyspace string,
	shards []*topo.ShardInfo,
	cells []string,
	servedType topodatapb.TabletType,
	isFrom bool,
	clearSourceShards bool,
	logger logutil.Logger,
) error

UpdateShardRecords updates the shard records based on 'from' or 'to' direction.

func ValidateForReshard

func ValidateForReshard(sourceShards, targetShards []*topo.ShardInfo) error

ValidateForReshard returns an error if sourceShards cannot reshard into targetShards.