topotools

Documentation

Overview

Package topotools contains high level functions based on vt/topo and vt/actionnode. It should not depend on anything else that's higher level. In particular, it cannot depend on:

• vt/wrangler: much higher level, wrangler depends on topotools.
• vt/tabletmanager/initiator: we don't want the various remote protocol dependencies here.

topotools is used by wrangler, so it ends up in all tools using wrangler (vtctl, vtctld, ...). It is also included by vttablet, so it contains:

• most of the logic to rebuild a shard serving graph (helthcheck module)
• some of the logic to perform a TabletExternallyReparented (RPC call to master vttablet to let it know it's the master).

Index

• func ChangeType(ctx context.Context, ts topo.Server, tabletAlias *pb.TabletAlias, ...) error
• func ConfigureTabletHook(hk *hook.Hook, tabletAlias *pb.TabletAlias)
• func CopyMapKeys(m interface{}, typeHint interface{}) interface{}
• func CopyMapValues(m interface{}, typeHint interface{}) interface{}
• func CreateShard(ctx context.Context, ts topo.Server, keyspace, shard string) error
• func FindTabletByIPAddrAndPort(tabletMap map[pb.TabletAlias]*topo.TabletInfo, addr, portName string, ...) (pb.TabletAlias, error)
• func GetAllTablets(ctx context.Context, ts topo.Server, cell string) ([]*topo.TabletInfo, error)
• func GetAllTabletsAcrossCells(ctx context.Context, ts topo.Server) ([]*topo.TabletInfo, error)
• func GetOrCreateShard(ctx context.Context, ts topo.Server, keyspace, shard string) (*topo.ShardInfo, error)
• func MapKeys(m interface{}) []interface{}
• func RebuildShard(ctx context.Context, log logutil.Logger, ts topo.Server, ...) (*topo.ShardInfo, error)
• func RestartSlavesExternal(ts topo.Server, log logutil.Logger, ...)
• func Scrap(ctx context.Context, ts topo.Server, tabletAlias *pb.TabletAlias, force bool) error
• func SortedTabletMap(tabletMap map[pb.TabletAlias]*topo.TabletInfo) (map[pb.TabletAlias]*topo.TabletInfo, map[pb.TabletAlias]*topo.TabletInfo)
• func UpdateAllSrvShards(ctx context.Context, ts topo.Server, si *topo.ShardInfo) error
• func UpdateSrvShard(ctx context.Context, ts topo.Server, cell string, si *topo.ShardInfo) error
• func UpdateTabletEndpoints(ctx context.Context, ts topo.Server, tablet *pb.Tablet) (err error)
• type KeyspaceNodes
  • func (ks KeyspaceNodes) HasType(tabletType pb.TabletType) bool
  • func (ks KeyspaceNodes) TabletTypes() []pb.TabletType
• type OverlappingShards
  • func FindOverlappingShards(ctx context.Context, ts topo.Server, keyspace string) ([]*OverlappingShards, error)
  • func OverlappingShardsForShard(os []*OverlappingShards, shardName string) *OverlappingShards
  • func (os *OverlappingShards) ContainsShard(shardName string) bool
• type ServingGraph
  • func DbServingGraph(ctx context.Context, ts topo.Server, cell string) (servingGraph *ServingGraph)
• type ShardNodes
• type TabletNode
  • func (tn *TabletNode) ShortName() string
• type TabletNodesByType
• type Topology
  • func DbTopology(ctx context.Context, ts topo.Server) (*Topology, error)

Functions

func ChangeType

func ChangeType(ctx context.Context, ts topo.Server, tabletAlias *pb.TabletAlias, newType pb.TabletType, health map[string]string) error

ChangeType changes the type of the tablet and possibly also updates the health informaton for it. Make this external, since these transitions need to be forced from time to time.

- if health is nil, we don't touch the Tablet's Health record. - if health is an empty map, we clear the Tablet's Health record. - if health has values, we overwrite the Tablet's Health record.

func ConfigureTabletHook

func ConfigureTabletHook(hk *hook.Hook, tabletAlias *pb.TabletAlias)

ConfigureTabletHook configures the right parameters for a hook running locally on a tablet.

func CopyMapKeys

func CopyMapKeys(m interface{}, typeHint interface{}) interface{}

CopyMapKeys copies keys from from map m into a new slice with the type specified by typeHint. Reflection can't make a new slice type just based on the key type AFAICT.

func CopyMapValues

func CopyMapValues(m interface{}, typeHint interface{}) interface{}

CopyMapValues copies values from from map m into a new slice with the type specified by typeHint. Reflection can't make a new slice type just based on the key type AFAICT.

func CreateShard

func CreateShard(ctx context.Context, ts topo.Server, keyspace, shard string) error

CreateShard will create the shard, while holding the keyspace lock

func FindTabletByIPAddrAndPort

func FindTabletByIPAddrAndPort(tabletMap map[pb.TabletAlias]*topo.TabletInfo, addr, portName string, port int32) (pb.TabletAlias, error)

FindTabletByIPAddrAndPort searches within a tablet map for tablets

func GetAllTablets

func GetAllTablets(ctx context.Context, ts topo.Server, cell string) ([]*topo.TabletInfo, error)

GetAllTablets returns a sorted list of tablets.

func GetAllTabletsAcrossCells

func GetAllTabletsAcrossCells(ctx context.Context, ts topo.Server) ([]*topo.TabletInfo, error)

GetAllTabletsAcrossCells returns all tablets from known cells. If it returns topo.ErrPartialResult, then the list is valid, but partial.

func GetOrCreateShard

func GetOrCreateShard(ctx context.Context, ts topo.Server, keyspace, shard string) (*topo.ShardInfo, error)

GetOrCreateShard will return the shard object, or create one if it doesn't already exist. Note the shard creation is protected by a keyspace Lock.

func MapKeys

func MapKeys(m interface{}) []interface{}

MapKeys returns an array with th provided map keys.

func RebuildShard

func RebuildShard(ctx context.Context, log logutil.Logger, ts topo.Server, keyspace, shard string, cells []string, lockTimeout time.Duration) (*topo.ShardInfo, error)

RebuildShard updates the SrvShard objects and underlying serving graph.

Re-read from TopologyServer to make sure we are using the side effects of all actions.

This function will start each cell over from the beginning on ErrBadVersion, so it doesn't need a lock on the shard.

func RestartSlavesExternal

func RestartSlavesExternal(ts topo.Server, log logutil.Logger, slaveTabletMap, masterTabletMap map[pb.TabletAlias]*topo.TabletInfo, masterElectTabletAlias *pb.TabletAlias, slaveWasRestarted func(*topo.TabletInfo, *actionnode.SlaveWasRestartedArgs) error)

RestartSlavesExternal will tell all the slaves in the provided list that they have a new master, and also tell all the masters. The masters will be scrapped if they don't answer. We execute all the actions in parallel.

func Scrap

func Scrap(ctx context.Context, ts topo.Server, tabletAlias *pb.TabletAlias, force bool) error

Scrap will update the tablet type to 'Scrap', and remove it from the serving graph.

'force' means we are not on the tablet being scrapped, so it is probably dead. So if 'force' is true, we will also remove pending remote actions. And if 'force' is false, we also run an optional hook.

func SortedTabletMap

func SortedTabletMap(tabletMap map[pb.TabletAlias]*topo.TabletInfo) (map[pb.TabletAlias]*topo.TabletInfo, map[pb.TabletAlias]*topo.TabletInfo)

SortedTabletMap returns two maps:

• The slaveMap contains all the non-master non-scrapped hosts. This can be used as a list of slaves to fix up for reparenting
• The masterMap contains all the tablets without parents (scrapped or not). This can be used to special case the old master, and any tablet in a weird state, left over, ...

func UpdateAllSrvShards

func UpdateAllSrvShards(ctx context.Context, ts topo.Server, si *topo.ShardInfo) error

UpdateAllSrvShards calls UpdateSrvShard for all cells concurrently.

func UpdateSrvShard

func UpdateSrvShard(ctx context.Context, ts topo.Server, cell string, si *topo.ShardInfo) error

UpdateSrvShard creates the SrvShard object based on the global ShardInfo, and writes it to the given cell.

func UpdateTabletEndpoints

func UpdateTabletEndpoints(ctx context.Context, ts topo.Server, tablet *pb.Tablet) (err error)

UpdateTabletEndpoints fixes up any entries in the serving graph that relate to a given tablet.

Types

type KeyspaceNodes

type KeyspaceNodes struct {
	ShardNodes []*ShardNodes // sorted by shard name
	ServedFrom map[topo.TabletType]string
}

KeyspaceNodes represents all tablet nodes in a keyspace.

func (KeyspaceNodes) HasType

func (ks KeyspaceNodes) HasType(tabletType pb.TabletType) bool

HasType returns true if ks has any tablets with the named type.

func (KeyspaceNodes) TabletTypes

func (ks KeyspaceNodes) TabletTypes() []pb.TabletType

TabletTypes returns a slice of tablet type names this ks contains.

type OverlappingShards

type OverlappingShards struct {
	Left  []*topo.ShardInfo
	Right []*topo.ShardInfo
}

OverlappingShards contains sets of shards that overlap which each-other. With this library, there is no guarantee of which set will be left or right.

func FindOverlappingShards

func FindOverlappingShards(ctx context.Context, ts topo.Server, keyspace string) ([]*OverlappingShards, error)

FindOverlappingShards will return an array of OverlappingShards for the provided keyspace. We do not support more than two overlapping shards (for instance, having 40-80, 40-60 and 40-50 in the same keyspace is not supported and will return an error). If shards don't perfectly overlap, they are not returned.

func OverlappingShardsForShard

func OverlappingShardsForShard(os []*OverlappingShards, shardName string) *OverlappingShards

OverlappingShardsForShard returns the OverlappingShards object from the list that has he provided shard, or nil

func (*OverlappingShards) ContainsShard

func (os *OverlappingShards) ContainsShard(shardName string) bool

ContainsShard returns true if either Left or Right lists contain the provided Shard.

type ServingGraph

type ServingGraph struct {
	Cell      string
	Keyspaces map[string]*KeyspaceNodes // indexed by keyspace name
	Errors    []string                  // collected during creation
}

ServingGraph contains the representation of the serving graph for a given cell.

func DbServingGraph

func DbServingGraph(ctx context.Context, ts topo.Server, cell string) (servingGraph *ServingGraph)

DbServingGraph returns the ServingGraph for the given cell.

type ShardNodes

type ShardNodes struct {
	Name        string
	TabletNodes []*TabletNodesByType
	ServedTypes []pb.TabletType
	Tag         interface{} // Tag is an arbitrary value manageable by a plugin.
}

ShardNodes represents all tablet nodes for a shard, indexed by tablet type.

type TabletNode

type TabletNode struct {
	Host  string
	Alias *pb.TabletAlias
	Port  int32
}

TabletNode is the representation of a tablet in the db topology. It can be constructed from a Tablet object, or from an EndPoint.

func (*TabletNode) ShortName

func (tn *TabletNode) ShortName() string

ShortName returns a displayable representation of the host name. If the host is an IP address instead of a name, it is not shortened.

type TabletNodesByType

type TabletNodesByType struct {
	TabletType pb.TabletType
	Nodes      []*TabletNode
}

TabletNodesByType maps tablet types to slices of tablet nodes.

type Topology

type Topology struct {
	Assigned map[string]*KeyspaceNodes // indexed by keyspace name
	Idle     []*TabletNode
	Scrap    []*TabletNode
	Partial  bool
}

Topology is the entire set of tablets in the topology.

func DbTopology

func DbTopology(ctx context.Context, ts topo.Server) (*Topology, error)

DbTopology returns the Topology for the topo server.