topo

package
v2.0.0-alpha1+incompat... Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2015 License: BSD-3-Clause Imports: 25 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// According to docs, the tablet uid / (mysql server id) is uint32.
	// However, zero appears to be a sufficiently degenerate value to use
	// as a marker for not having a parent server id.
	// http://dev.mysql.com/doc/refman/5.1/en/replication-options.html
	NO_TABLET = 0

	// ReplicationLag is the key in the health map to indicate high
	// replication lag
	ReplicationLag = "replication_lag"

	// ReplicationLagHigh is the value in the health map to indicate high
	// replication lag
	ReplicationLagHigh = "high"
)
View Source 
const (
	// idle -  no keyspace, shard or type assigned
	TYPE_IDLE = TabletType("idle")

	// primary copy of data
	TYPE_MASTER = TabletType("master")

	// a slaved copy of the data ready to be promoted to master
	TYPE_REPLICA = TabletType("replica")

	// a slaved copy of the data for olap load patterns.
	// too many aliases for olap - need to pick one
	TYPE_RDONLY = TabletType("rdonly")
	TYPE_BATCH  = TabletType("batch")

	// a slaved copy of the data ready, but not serving query traffic
	// could be a potential master.
	TYPE_SPARE = TabletType("spare")

	// a slaved copy of the data ready, but not serving query traffic
	// implies something abnormal about the setup - don't consider it
	// a potential master and don't worry about lag when reparenting.
	TYPE_EXPERIMENTAL = TabletType("experimental")

	// a slaved copy of the data that was serving but is now applying
	// a schema change. Will go bak to its serving type after the
	// upgrade
	TYPE_SCHEMA_UPGRADE = TabletType("schema_apply")

	// a slaved copy of the data, but offline to queries other than backup
	// replication sql thread may be stopped
	TYPE_BACKUP = TabletType("backup")

	// A tablet that has not been in the replication graph and is restoring
	// from a snapshot.
	TYPE_RESTORE = TabletType("restore")

	// A tablet that is used by a worker process. It is probably
	// lagging in replication.
	TYPE_WORKER = TabletType("worker")

	// a machine with data that needs to be wiped
	TYPE_SCRAP = TabletType("scrap")
)
View Source 
const (
	// DefaultPortName is the port named used by SrvEntries
	// if "" is given as the named port.
	DefaultPortName = "vt"
)

Variables

View Source 
var (
	// ErrNodeExists is returned by functions to specify the
	// requested resource already exists.
	ErrNodeExists = errors.New("node already exists")

	// ErrNoNode is returned by functions to specify the requested
	// resource does not exist.
	ErrNoNode = errors.New("node doesn't exist")

	// ErrNotEmpty is returned by functions to specify a child of the
	// resource is still present and prevents the action from completing.
	ErrNotEmpty = errors.New("node not empty")

	// ErrTimeout is returned by functions that wait for a result
	// when the timeout value is reached.
	ErrTimeout = errors.New("deadline exceeded")

	// ErrInterrupted is returned by functions that wait for a result
	// when they are interrupted.
	ErrInterrupted = errors.New("interrupted")

	// ErrBadVersion is returned by an update function that
	// failed to update the data because the version was different
	ErrBadVersion = errors.New("bad node version")

	// ErrPartialResult is returned by a function that could only
	// get a subset of its results
	ErrPartialResult = errors.New("partial result")
)

Functions

func CloseServers 

func CloseServers()

CloseServers closes all registered Server.

func DeleteTabletReplicationData 

func DeleteTabletReplicationData(ctx context.Context, ts Server, tablet *pb.Tablet) error

DeleteTabletReplicationData deletes replication data.

func EndPointEquality 

func EndPointEquality(left, right *pb.EndPoint) bool

EndPointEquality returns true iff two EndPoint are representing the same data

func FixShardReplication 

func FixShardReplication(ctx context.Context, ts Server, logger logutil.Logger, cell, keyspace, shard string) error

FixShardReplication will fix the first problem it encounters within a ShardReplication object

func InCellList 

func InCellList(cell string, cells []string) bool

InCellList returns true if the cell list is empty, or if the passed cell is in the cell list.

func IsHealthEqual 

func IsHealthEqual(left, right map[string]string) bool

IsHealthEqual compares the two health maps, and returns true if they're equivalent.

func IsInReplicationGraph 

func IsInReplicationGraph(tt pb.TabletType) bool

IsInReplicationGraph returns if this tablet appears in the replication graph Only IDLE and SCRAP are not in the replication graph. The other non-obvious types are BACKUP, SNAPSHOT_SOURCE, RESTORE: these have had a master at some point (or were the master), so they are in the graph.

func IsInServingGraph 

func IsInServingGraph(tt pb.TabletType) bool

IsInServingGraph returns if a tablet appears in the serving graph

func IsRunningQueryService 

func IsRunningQueryService(tt pb.TabletType) bool

IsRunningQueryService returns if a tablet is running the query service

func IsRunningUpdateStream 

func IsRunningUpdateStream(tt pb.TabletType) bool

IsRunningUpdateStream returns if a tablet is running the update stream RPC service.

func IsShardUsingRangeBasedSharding 

func IsShardUsingRangeBasedSharding(shard string) bool

IsShardUsingRangeBasedSharding returns true if the shard name implies it is using range based sharding.

func IsSlaveType 

func IsSlaveType(tt pb.TabletType) bool

IsSlaveType returns if this type should be connected to a master db and actively replicating? MASTER is not obviously (only support one level replication graph) IDLE and SCRAP are not either BACKUP, RESTORE, TYPE_WORKER may or may not be, but we don't know for sure

func IsTrivialTypeChange 

func IsTrivialTypeChange(oldTabletType, newTabletType pb.TabletType) bool

IsTrivialTypeChange returns if this db type be trivially reassigned without changes to the replication graph

func LookupVtName 

func LookupVtName(ctx context.Context, ts Server, cell, keyspace, shard string, tabletType pb.TabletType, namedPort string) ([]*net.SRV, error)

LookupVtName gets the list of EndPoints for a cell/keyspace/shard/tablet type and converts the list to net.SRV records

func NewEndPoint 

func NewEndPoint(uid uint32, host string) *pb.EndPoint

NewEndPoint returns a new empty EndPoint

func NewEndPoints 

func NewEndPoints() *pb.EndPoints

NewEndPoints creates a EndPoints with a pre-allocated slice for Entries.

func RegisterServer 

func RegisterServer(name string, ts Impl)

RegisterServer adds an implementation for a Server. If an implementation with that name already exists, panics. Call this in the 'init' function in your module.

func RemoveShardReplicationRecord 

func RemoveShardReplicationRecord(ctx context.Context, ts Server, cell, keyspace, shard string, tabletAlias *pb.TabletAlias) error

RemoveShardReplicationRecord is a low level function to remove an entry from the ShardReplication object.

func ResolveKeyspaceWildcard 

func ResolveKeyspaceWildcard(ctx context.Context, server WildcardBackend, param string) ([]string, error)

ResolveKeyspaceWildcard will resolve keyspace wildcards.

  • If the param is not a wildcard, it will just be returned (if the keyspace doesn't exist, it is still returned).
  • If the param is a wildcard, it will get all keyspaces and returns the ones which match the wildcard (which may be an empty list).

func SrvAddr 

func SrvAddr(srv *net.SRV) string

SrvAddr prints a net.SRV

func SrvEntries 

func SrvEntries(addrs *pb.EndPoints, namedPort string) (srvs []*net.SRV, err error)

SrvEntries converts EndPoints to net.SRV for a given port. FIXME(msolomon) merge with zkns

func SrvKeyspaceToProto 

func SrvKeyspaceToProto(s *SrvKeyspace) *pb.SrvKeyspace

SrvKeyspaceToProto turns a Tablet into a proto

func TabletComplete 

func TabletComplete(tablet *pb.Tablet) error

TabletComplete validates and normalizes the tablet. If the shard name contains a '-' it is going to try to infer the keyrange from it.

func TabletEndPoint 

func TabletEndPoint(tablet *pb.Tablet) (*pb.EndPoint, error)

TabletEndPoint returns an EndPoint associated with the tablet record

func TabletTypeToProto 

func TabletTypeToProto(t TabletType) pb.TabletType

TabletTypeToProto turns a TabletType into a proto

func TabletValidatePortMap 

func TabletValidatePortMap(tablet *pb.Tablet) error

TabletValidatePortMap returns an error if the tablet's portmap doesn't contain all the necessary ports for the tablet to be fully operational. We only care about vt port now, as mysql may not even be running.

func UpdateEndPoints 

func UpdateEndPoints(ctx context.Context, ts Server, cell, keyspace, shard string, tabletType pb.TabletType, addrs *pb.EndPoints, existingVersion int64) error

UpdateEndPoints is a high level wrapper for TopoServer.UpdateEndPoints. It generates trace spans.

func UpdateShardReplicationRecord 

func UpdateShardReplicationRecord(ctx context.Context, ts Server, keyspace, shard string, tabletAlias *pb.TabletAlias) error

UpdateShardReplicationRecord is a low level function to add / update an entry to the ShardReplication object.

func UpdateTabletReplicationData 

func UpdateTabletReplicationData(ctx context.Context, ts Server, tablet *pb.Tablet) error

UpdateTabletReplicationData creates or updates the replication graph data for a tablet

func Validate 

func Validate(ctx context.Context, ts Server, tabletAlias *pb.TabletAlias) error

Validate makes sure a tablet is represented correctly in the topology server.

func ValidateShardName 

func ValidateShardName(shard string) (string, *pb.KeyRange, error)

ValidateShardName takes a shard name and sanitizes it, and also returns the KeyRange.

Types

type GetEndPointsArgs 

type GetEndPointsArgs struct {
	Cell       string
	Keyspace   string
	Shard      string
	TabletType TabletType
}

GetEndPointsArgs is the parameters for TopoReader.GetEndPoints

func (*GetEndPointsArgs) MarshalBson 

func (getEndPointsArgs *GetEndPointsArgs) MarshalBson(buf *bytes2.ChunkedWriter, key string)

MarshalBson bson-encodes GetEndPointsArgs.

func (*GetEndPointsArgs) UnmarshalBson 

func (getEndPointsArgs *GetEndPointsArgs) UnmarshalBson(buf *bytes.Buffer, kind byte)

UnmarshalBson bson-decodes into GetEndPointsArgs.

type GetSrvKeyspaceArgs 

type GetSrvKeyspaceArgs struct {
	Cell     string
	Keyspace string
}

GetSrvKeyspaceArgs is the parameters for TopoReader.GetSrvKeyspace

func (*GetSrvKeyspaceArgs) MarshalBson 

func (getSrvKeyspaceArgs *GetSrvKeyspaceArgs) MarshalBson(buf *bytes2.ChunkedWriter, key string)

MarshalBson bson-encodes GetSrvKeyspaceArgs.

func (*GetSrvKeyspaceArgs) UnmarshalBson 

func (getSrvKeyspaceArgs *GetSrvKeyspaceArgs) UnmarshalBson(buf *bytes.Buffer, kind byte)

UnmarshalBson bson-decodes into GetSrvKeyspaceArgs.

type GetSrvKeyspaceNamesArgs 

type GetSrvKeyspaceNamesArgs struct {
	Cell string
}

GetSrvKeyspaceNamesArgs is the parameters for TopoReader.GetSrvKeyspaceNames

func (*GetSrvKeyspaceNamesArgs) MarshalBson 

func (getSrvKeyspaceNamesArgs *GetSrvKeyspaceNamesArgs) MarshalBson(buf *bytes2.ChunkedWriter, key string)

MarshalBson bson-encodes GetSrvKeyspaceNamesArgs.

func (*GetSrvKeyspaceNamesArgs) UnmarshalBson 

func (getSrvKeyspaceNamesArgs *GetSrvKeyspaceNamesArgs) UnmarshalBson(buf *bytes.Buffer, kind byte)

UnmarshalBson bson-decodes into GetSrvKeyspaceNamesArgs.

type GetSrvShardArgs 

type GetSrvShardArgs struct {
	Cell     string
	Keyspace string
	Shard    string
}

GetSrvShardArgs is the parameters for TopoReader.GetSrvShard

func (*GetSrvShardArgs) MarshalBson 

func (getSrvShardArgs *GetSrvShardArgs) MarshalBson(buf *bytes2.ChunkedWriter, key string)

MarshalBson bson-encodes GetSrvShardArgs.

func (*GetSrvShardArgs) UnmarshalBson 

func (getSrvShardArgs *GetSrvShardArgs) UnmarshalBson(buf *bytes.Buffer, kind byte)

UnmarshalBson bson-decodes into GetSrvShardArgs.

type Impl 

type Impl interface {
	// topo.Server management interface.
	Close()

	// GetKnownCells returns the list of known cells running our processes.
	// It is possible to find all tablets in the entire system
	// by then calling GetTabletsByCell on every cell, for instance.
	// They shall be sorted.
	GetKnownCells(ctx context.Context) ([]string, error)

	// CreateKeyspace creates the given keyspace, assuming it doesn't exist
	// yet. Can return ErrNodeExists if it already exists.
	CreateKeyspace(ctx context.Context, keyspace string, value *pb.Keyspace) error

	// UpdateKeyspace updates the keyspace information
	// pointed at by ki.keyspace to the *ki value.
	// This will only be called with a lock on the keyspace.
	// Can return ErrNoNode if the keyspace doesn't exist yet,
	// or ErrBadVersion if the version has changed.
	//
	// Do not use directly, but instead use topo.UpdateKeyspace.
	UpdateKeyspace(ctx context.Context, keyspace string, value *pb.Keyspace, existingVersion int64) (newVersion int64, err error)

	// DeleteKeyspace deletes the specified keyspace.
	// Can return ErrNoNode if the keyspace doesn't exist.
	DeleteKeyspace(ctx context.Context, keyspace string) error

	// GetKeyspace reads a keyspace and returns it, along with its version.
	// Can return ErrNoNode
	GetKeyspace(ctx context.Context, keyspace string) (*pb.Keyspace, int64, error)

	// GetKeyspaces returns the known keyspace names. They shall be sorted.
	GetKeyspaces(ctx context.Context) ([]string, error)

	// DeleteKeyspaceShards deletes all the shards in a keyspace.
	// Use with caution.
	DeleteKeyspaceShards(ctx context.Context, keyspace string) error

	// CreateShard creates an empty shard, assuming it doesn't exist
	// yet. The contents of the shard will be a new Shard{} object,
	// with KeyRange populated by the result of ValidateShardName().
	// Can return ErrNodeExists if it already exists.
	CreateShard(ctx context.Context, keyspace, shard string, value *pb.Shard) error

	// UpdateShard updates the shard information
	// pointed at by si.keyspace / si.shard to the *si value.
	// This will only be called with a lock on the shard.
	// Can return ErrNoNode if the shard doesn't exist yet,
	// or ErrBadVersion if the version has changed.
	//
	// Do not use directly, but instead use topo.UpdateShard.
	UpdateShard(ctx context.Context, keyspace, shard string, value *pb.Shard, existingVersion int64) (newVersion int64, err error)

	// ValidateShard performs routine checks on the shard.
	ValidateShard(ctx context.Context, keyspace, shard string) error

	// GetShard reads a shard and returns it, along with its version.
	// Can return ErrNoNode
	GetShard(ctx context.Context, keyspace, shard string) (*pb.Shard, int64, error)

	// GetShardNames returns the known shards in a keyspace.
	// Can return ErrNoNode if the keyspace wasn't created,
	// or if DeleteKeyspaceShards was called. They shall be sorted.
	GetShardNames(ctx context.Context, keyspace string) ([]string, error)

	// DeleteShard deletes the provided shard.
	// Can return ErrNoNode if the shard doesn't exist.
	DeleteShard(ctx context.Context, keyspace, shard string) error

	// CreateTablet creates the given tablet, assuming it doesn't exist
	// yet. It does *not* create the tablet replication paths.
	// Can return ErrNodeExists if it already exists.
	CreateTablet(ctx context.Context, tablet *pb.Tablet) error

	// UpdateTablet updates a given tablet. The version is used
	// for atomic updates. UpdateTablet will return ErrNoNode if
	// the tablet doesn't exist and ErrBadVersion if the version
	// has changed.
	//
	// Do not use directly, but instead use topo.UpdateTablet.
	UpdateTablet(ctx context.Context, tablet *pb.Tablet, existingVersion int64) (newVersion int64, err error)

	// UpdateTabletFields updates the current tablet record
	// with new values, independently of the version
	// Can return ErrNoNode if the tablet doesn't exist.
	UpdateTabletFields(ctx context.Context, tabletAlias *pb.TabletAlias, update func(*pb.Tablet) error) (*pb.Tablet, error)

	// DeleteTablet removes a tablet from the system.
	// We assume no RPC is currently running to it.
	// TODO(alainjobart) verify this assumption, link with RPC code.
	// Can return ErrNoNode if the tablet doesn't exist.
	DeleteTablet(ctx context.Context, alias *pb.TabletAlias) error

	// GetTablet returns the tablet data (includes the current version).
	// Can return ErrNoNode if the tablet doesn't exist.
	GetTablet(ctx context.Context, alias *pb.TabletAlias) (*pb.Tablet, int64, error)

	// GetTabletsByCell returns all the tablets in the given cell.
	// Can return ErrNoNode if no tablet was ever created in that cell.
	GetTabletsByCell(ctx context.Context, cell string) ([]*pb.TabletAlias, error)

	// UpdateShardReplicationFields updates the current
	// ShardReplication record with new values. If the
	// ShardReplication object does not exist, an empty one will
	// be passed to the update function. All necessary directories
	// need to be created by this method, if applicable.
	UpdateShardReplicationFields(ctx context.Context, cell, keyspace, shard string, update func(*pb.ShardReplication) error) error

	// GetShardReplication returns the replication data.
	// Can return ErrNoNode if the object doesn't exist.
	GetShardReplication(ctx context.Context, cell, keyspace, shard string) (*ShardReplicationInfo, error)

	// DeleteShardReplication deletes the replication data.
	// Can return ErrNoNode if the object doesn't exist.
	DeleteShardReplication(ctx context.Context, cell, keyspace, shard string) error

	// DeleteKeyspaceReplication deletes the replication data for all shards.
	// Can return ErrNoNode if the object doesn't exist.
	DeleteKeyspaceReplication(ctx context.Context, cell, keyspace string) error

	// LockSrvShardForAction locks the serving shard in order to
	// perform the action described by contents. It will wait for
	// the lock until at most ctx.Done(). The wait can be interrupted
	// by cancelling the context. It returns the lock path.
	//
	// Can return ErrTimeout or ErrInterrupted.
	LockSrvShardForAction(ctx context.Context, cell, keyspace, shard, contents string) (string, error)

	// UnlockSrvShardForAction unlocks a serving shard.
	UnlockSrvShardForAction(ctx context.Context, cell, keyspace, shard, lockPath, results string) error

	// GetSrvTabletTypesPerShard returns the existing serving types
	// for a shard.
	// Can return ErrNoNode.
	GetSrvTabletTypesPerShard(ctx context.Context, cell, keyspace, shard string) ([]pb.TabletType, error)

	// CreateEndPoints creates and sets the serving records for a cell,
	// keyspace, shard, tabletType.
	// It returns ErrNodeExists if the record already exists.
	CreateEndPoints(ctx context.Context, cell, keyspace, shard string, tabletType pb.TabletType, addrs *pb.EndPoints) error

	// UpdateEndPoints updates the serving records for a cell,
	// keyspace, shard, tabletType.
	// If existingVersion is -1, it will set the value unconditionally,
	// creating it if necessary.
	// Otherwise, it will Compare-And-Set only if the version matches.
	// Can return ErrBadVersion.
	// Can return ErrNoNode only if existingVersion is not -1.
	UpdateEndPoints(ctx context.Context, cell, keyspace, shard string, tabletType pb.TabletType, addrs *pb.EndPoints, existingVersion int64) error

	// GetEndPoints returns the EndPoints list of serving addresses
	// for a TabletType inside a shard, as well as the node version.
	// Can return ErrNoNode.
	GetEndPoints(ctx context.Context, cell, keyspace, shard string, tabletType pb.TabletType) (ep *pb.EndPoints, version int64, err error)

	// DeleteEndPoints deletes the serving records for a cell,
	// keyspace, shard, tabletType.
	// If existingVersion is -1, it will delete the records unconditionally.
	// Otherwise, it will Compare-And-Delete only if the version matches.
	// Can return ErrNoNode or ErrBadVersion.
	DeleteEndPoints(ctx context.Context, cell, keyspace, shard string, tabletType pb.TabletType, existingVersion int64) error

	// WatchSrvKeyspace returns a channel that receives notifications
	// every time the SrvKeyspace for the given keyspace / cell changes.
	// It should receive a notification with the initial value fairly
	// quickly after this is set. A value of nil means the SrvKeyspace
	// object doesn't exist or is empty. To stop watching this
	// SrvKeyspace object, close the stopWatching channel.
	// If the underlying topo.Server encounters an error watching the node,
	// it should retry on a regular basis until it can succeed.
	// The initial error returned by this method is meant to catch
	// the obvious bad cases (invalid cell, invalid tabletType, ...)
	// that are never going to work. Mutiple notifications with the
	// same contents may be sent (for instance when the serving graph
	// is rebuilt, but the content hasn't changed).
	WatchSrvKeyspace(ctx context.Context, cell, keyspace string) (notifications <-chan *SrvKeyspace, stopWatching chan<- struct{}, err error)

	// UpdateSrvShard updates the serving records for a cell,
	// keyspace, shard.
	UpdateSrvShard(ctx context.Context, cell, keyspace, shard string, srvShard *pb.SrvShard) error

	// GetSrvShard reads a SrvShard record.
	// Can return ErrNoNode.
	GetSrvShard(ctx context.Context, cell, keyspace, shard string) (*pb.SrvShard, error)

	// DeleteSrvShard deletes a SrvShard record.
	// Can return ErrNoNode.
	DeleteSrvShard(ctx context.Context, cell, keyspace, shard string) error

	// UpdateSrvKeyspace updates the serving records for a cell, keyspace.
	UpdateSrvKeyspace(ctx context.Context, cell, keyspace string, srvKeyspace *SrvKeyspace) error

	// DeleteSrvKeyspace deletes the cell-local serving records for a keyspace.
	// Can return ErrNoNode.
	DeleteSrvKeyspace(ctx context.Context, cell, keyspace string) error

	// GetSrvKeyspace reads a SrvKeyspace record.
	// Can return ErrNoNode.
	GetSrvKeyspace(ctx context.Context, cell, keyspace string) (*SrvKeyspace, error)

	// GetSrvKeyspaceNames returns the list of visible Keyspaces
	// in this cell. They shall be sorted.
	GetSrvKeyspaceNames(ctx context.Context, cell string) ([]string, error)

	// LockKeyspaceForAction locks the keyspace in order to
	// perform the action described by contents. It will wait for
	// the lock until at most ctx.Done(). The wait can be interrupted
	// by cancelling the context. It returns the lock path.
	//
	// Can return ErrTimeout or ErrInterrupted
	LockKeyspaceForAction(ctx context.Context, keyspace, contents string) (string, error)

	// UnlockKeyspaceForAction unlocks a keyspace.
	UnlockKeyspaceForAction(ctx context.Context, keyspace, lockPath, results string) error

	// LockShardForAction locks the shard in order to
	// perform the action described by contents. It will wait for
	// the lock until at most ctx.Done(). The wait can be interrupted
	// by cancelling the context. It returns the lock path.
	//
	// Can return ErrTimeout or ErrInterrupted
	LockShardForAction(ctx context.Context, keyspace, shard, contents string) (string, error)

	// UnlockShardForAction unlocks a shard.
	UnlockShardForAction(ctx context.Context, keyspace, shard, lockPath, results string) error

	// SaveVSchema saves the provided schema in the topo server.
	SaveVSchema(context.Context, string) error

	// GetVSchema retrieves the schema from the topo server.
	//
	// If no schema has been previously saved, it should return "{}"
	GetVSchema(ctx context.Context) (string, error)
}

Impl is the interface used to talk to a persistent backend storage server and locking service.

Zookeeper is a good example of this, and zktopo contains the implementation for this using zookeeper.

Inside Google, we use Chubby.

type KeyspaceInfo 

type KeyspaceInfo struct {
	*pb.Keyspace
	// contains filtered or unexported fields
}

KeyspaceInfo is a meta struct that contains metadata to give the data more context and convenience. This is the main way we interact with a keyspace.

func (*KeyspaceInfo) CheckServedFromMigration 

func (ki *KeyspaceInfo) CheckServedFromMigration(tabletType pb.TabletType, cells []string, keyspace string, remove bool) error

CheckServedFromMigration makes sure a requested migration is safe

func (*KeyspaceInfo) ComputeCellServedFrom 

func (ki *KeyspaceInfo) ComputeCellServedFrom(cell string) map[TabletType]string

ComputeCellServedFrom returns the ServedFrom map for a cell

func (*KeyspaceInfo) GetServedFrom 

func (ki *KeyspaceInfo) GetServedFrom(tabletType pb.TabletType) *pb.Keyspace_ServedFrom

GetServedFrom returns a Keyspace_ServedFrom record if it exists.

func (*KeyspaceInfo) KeyspaceName 

func (ki *KeyspaceInfo) KeyspaceName() string

KeyspaceName returns the keyspace name

func (*KeyspaceInfo) UpdateServedFromMap 

func (ki *KeyspaceInfo) UpdateServedFromMap(tabletType pb.TabletType, cells []string, keyspace string, remove bool, allCells []string) error

UpdateServedFromMap handles ServedFromMap. It can add or remove records, cells, ...

type KeyspacePartition 

type KeyspacePartition struct {
	// List of non-overlapping continuous shard references sorted by range.
	ShardReferences []ShardReference
}

KeyspacePartition represents a continuous set of shards to serve an entire data set.

func (*KeyspacePartition) HasShard 

func (kp *KeyspacePartition) HasShard(name string) bool

HasShard returns true if this KeyspacePartition has the shard with the given name in it.

func (*KeyspacePartition) MarshalBson 

func (keyspacePartition *KeyspacePartition) MarshalBson(buf *bytes2.ChunkedWriter, key string)

MarshalBson bson-encodes KeyspacePartition.

func (*KeyspacePartition) UnmarshalBson 

func (keyspacePartition *KeyspacePartition) UnmarshalBson(buf *bytes.Buffer, kind byte)

UnmarshalBson bson-decodes into KeyspacePartition.

type KeyspaceShard 

type KeyspaceShard struct {
	Keyspace string
	Shard    string
}

KeyspaceShard is a type used by ResolveShardWildcard

func ResolveShardWildcard 

func ResolveShardWildcard(ctx context.Context, server WildcardBackend, param string) ([]KeyspaceShard, error)

ResolveShardWildcard will resolve shard wildcards. Both keyspace and shard names can use wildcard. Errors talking to the topology server are returned. ErrNoNode is ignored if it's the result of resolving a wildcard. Examples:

  • */* returns all keyspace/shard pairs, or empty list if none.
  • user/* returns all shards in user keyspace (or error if user keyspace doesn't exist)
  • us*/* returns all shards in all keyspaces that start with 'us'. If no such keyspace exists, list is empty (it is not an error).

type Server 

type Server struct {
	Impl
}

Server is a wrapper type that can have extra methods. Outside modules should just use the Server object.

func GetServer 

func GetServer() Server

GetServer returns 'our' Server, going down this list:

  • If only one is registered, that's the one.
  • If more than one are registered, use the 'topo_implementation' flag (which defaults to zookeeper).
  • Then panics.

func GetServerByName 

func GetServerByName(name string) Server

GetServerByName returns a specific Server by name, or nil.

func (Server) CreateKeyspace 

func (ts Server) CreateKeyspace(ctx context.Context, keyspace string, value *pb.Keyspace) error

CreateKeyspace wraps the underlying Impl.DeleteKeyspaceShards and dispatches the event.

func (Server) CreateShard 

func (ts Server) CreateShard(ctx context.Context, keyspace, shard string) error

CreateShard creates a new shard and tries to fill in the right information. This should be called while holding the keyspace lock for the shard. (call topotools.CreateShard to do that for you). In unit tests (that are not parallel), this function can be called directly.

func (Server) CreateTablet 

func (ts Server) CreateTablet(ctx context.Context, tablet *pb.Tablet) error

CreateTablet creates a new tablet and all associated paths for the replication graph.

func (Server) DeleteKeyspace 

func (ts Server) DeleteKeyspace(ctx context.Context, keyspace string) error

DeleteKeyspace wraps the underlying Impl.DeleteKeyspace and dispatches the event.

func (Server) DeleteKeyspaceShards 

func (ts Server) DeleteKeyspaceShards(ctx context.Context, keyspace string) error

DeleteKeyspaceShards wraps the underlying Impl.DeleteKeyspaceShards and dispatches the event.

func (Server) DeleteShard 

func (ts Server) DeleteShard(ctx context.Context, keyspace, shard string) error

DeleteShard wraps the underlying Impl.DeleteShard and dispatches the event.

func (Server) DeleteTablet 

func (ts Server) DeleteTablet(ctx context.Context, tabletAlias *pb.TabletAlias) error

DeleteTablet wraps the underlying Impl.DeleteTablet and dispatches the event.

func (Server) FindAllShardsInKeyspace 

func (ts Server) FindAllShardsInKeyspace(ctx context.Context, keyspace string) (map[string]*ShardInfo, error)

FindAllShardsInKeyspace reads and returns all the existing shards in a keyspace. It doesn't take any lock.

func (Server) FindAllTabletAliasesInShard 

func (ts Server) FindAllTabletAliasesInShard(ctx context.Context, keyspace, shard string) ([]*pb.TabletAlias, error)

FindAllTabletAliasesInShard uses the replication graph to find all the tablet aliases in the given shard.

It can return ErrPartialResult if some cells were not fetched, in which case the result only contains the cells that were fetched.

The tablet aliases are sorted by cell, then by UID.

func (Server) FindAllTabletAliasesInShardByCell 

func (ts Server) FindAllTabletAliasesInShardByCell(ctx context.Context, keyspace, shard string, cells []string) ([]*pb.TabletAlias, error)

FindAllTabletAliasesInShardByCell uses the replication graph to find all the tablet aliases in the given shard.

It can return ErrPartialResult if some cells were not fetched, in which case the result only contains the cells that were fetched.

The tablet aliases are sorted by cell, then by UID.

func (Server) GetKeyspace 

func (ts Server) GetKeyspace(ctx context.Context, keyspace string) (*KeyspaceInfo, error)

GetKeyspace reads the given keyspace and returns it

func (Server) GetShard 

func (ts Server) GetShard(ctx context.Context, keyspace, shard string) (*ShardInfo, error)

GetShard is a high level function to read shard data. It generates trace spans.

func (Server) GetTablet 

func (ts Server) GetTablet(ctx context.Context, alias *pb.TabletAlias) (*TabletInfo, error)

GetTablet is a high level function to read tablet data. It generates trace spans.

func (Server) GetTabletMap 

func (ts Server) GetTabletMap(ctx context.Context, tabletAliases []*pb.TabletAlias) (map[pb.TabletAlias]*TabletInfo, error)

GetTabletMap tries to read all the tablets in the provided list, and returns them all in a map. If error is ErrPartialResult, the results in the dictionary are incomplete, meaning some tablets couldn't be read.

func (Server) GetTabletMapForShard 

func (ts Server) GetTabletMapForShard(ctx context.Context, keyspace, shard string) (map[pb.TabletAlias]*TabletInfo, error)

GetTabletMapForShard returns the tablets for a shard. It can return ErrPartialResult if it couldn't read all the cells, or all the individual tablets, in which case the map is valid, but partial.

func (Server) GetTabletMapForShardByCell 

func (ts Server) GetTabletMapForShardByCell(ctx context.Context, keyspace, shard string, cells []string) (map[pb.TabletAlias]*TabletInfo, error)

GetTabletMapForShardByCell returns the tablets for a shard. It can return ErrPartialResult if it couldn't read all the cells, or all the individual tablets, in which case the map is valid, but partial.

func (Server) UpdateKeyspace 

func (ts Server) UpdateKeyspace(ctx context.Context, ki *KeyspaceInfo) error

UpdateKeyspace updates the keyspace data, with the right version

func (Server) UpdateShard 

func (ts Server) UpdateShard(ctx context.Context, si *ShardInfo) error

UpdateShard updates the shard data, with the right version. It also creates a span, and dispatches the event.

func (Server) UpdateShardFields 

func (ts Server) UpdateShardFields(ctx context.Context, keyspace, shard string, update func(*pb.Shard) error) (*ShardInfo, error)

UpdateShardFields is a high level helper to read a shard record, call an update function on it, and then write it back. If the write fails due to a version mismatch, it will re-read the record and retry the update. If the update succeeds, it returns the updated ShardInfo.

func (Server) UpdateTablet 

func (ts Server) UpdateTablet(ctx context.Context, tablet *TabletInfo) error

UpdateTablet updates the tablet data only - not associated replication paths. It also uses a span, and sends the event.

func (Server) UpdateTabletFields 

func (ts Server) UpdateTabletFields(ctx context.Context, alias *pb.TabletAlias, update func(*pb.Tablet) error) error

UpdateTabletFields is a high level wrapper for TopoServer.UpdateTabletFields that generates trace spans.

type ShardInfo 

type ShardInfo struct {
	*pb.Shard
	// contains filtered or unexported fields
}

ShardInfo is a meta struct that contains metadata to give the data more context and convenience. This is the main way we interact with a shard.

func NewShardInfo 

func NewShardInfo(keyspace, shard string, value *pb.Shard, version int64) *ShardInfo

NewShardInfo returns a ShardInfo basing on shard with the keyspace / shard. This function should be only used by Server implementations.

func (*ShardInfo) CheckServedTypesMigration 

func (si *ShardInfo) CheckServedTypesMigration(tabletType pb.TabletType, cells []string, remove bool) error

CheckServedTypesMigration makes sure the provided migration is possible

func (*ShardInfo) GetServedType 

func (si *ShardInfo) GetServedType(tabletType pb.TabletType) *pb.Shard_ServedType

GetServedType returns the Shard_ServedType for a TabletType, or nil

func (*ShardInfo) GetServedTypesPerCell 

func (si *ShardInfo) GetServedTypesPerCell(cell string) []TabletType

GetServedTypesPerCell returns the list of types this shard is serving in the provided cell.

func (*ShardInfo) GetTabletControl 

func (si *ShardInfo) GetTabletControl(tabletType pb.TabletType) *pb.Shard_TabletControl

GetTabletControl returns the Shard_TabletControl for the given tablet type, or nil if it is not in the map.

func (*ShardInfo) HasCell 

func (si *ShardInfo) HasCell(cell string) bool

HasCell returns true if the cell is listed in the Cells for the shard.

func (*ShardInfo) HasMaster 

func (si *ShardInfo) HasMaster() bool

HasMaster returns true if the Shard has an assigned Master.

func (*ShardInfo) Keyspace 

func (si *ShardInfo) Keyspace() string

Keyspace returns the keyspace a shard belongs to.

func (*ShardInfo) ShardName 

func (si *ShardInfo) ShardName() string

ShardName returns the shard name for a shard.

func (*ShardInfo) UpdateDisableQueryService 

func (si *ShardInfo) UpdateDisableQueryService(tabletType pb.TabletType, cells []string, disableQueryService bool) error

UpdateDisableQueryService will make sure the disableQueryService is set appropriately in the shard record. Note we don't support a lot of the corner cases:

  • we don't support DisableQueryService at the same time as BlacklistedTables, because it's not used in the same context (vertical vs horizontal sharding)

func (*ShardInfo) UpdateServedTypesMap 

func (si *ShardInfo) UpdateServedTypesMap(tabletType pb.TabletType, cells []string, remove bool) error

UpdateServedTypesMap handles ServedTypesMap. It can add or remove records, cells, ...

func (*ShardInfo) UpdateSourceBlacklistedTables 

func (si *ShardInfo) UpdateSourceBlacklistedTables(tabletType pb.TabletType, cells []string, remove bool, tables []string) error

UpdateSourceBlacklistedTables will add or remove the listed tables in the shard record's TabletControl structures. Note we don't support a lot of the corner cases:

  • only support one table list per shard. If we encounter a different table list that the provided one, we error out.
  • we don't support DisableQueryService at the same time as BlacklistedTables, because it's not used in the same context (vertical vs horizontal sharding)

func (*ShardInfo) Version 

func (si *ShardInfo) Version() int64

Version returns the shard version from last time it was read or updated.

type ShardReference 

type ShardReference struct {
	// Copied / inferred from Shard
	Name     string
	KeyRange key.KeyRange
}

ShardReference is the structure used by SrvKeyspace to point to a Shard

func (*ShardReference) MarshalBson 

func (shardReference *ShardReference) MarshalBson(buf *bytes2.ChunkedWriter, key string)

MarshalBson bson-encodes ShardReference.

func (*ShardReference) UnmarshalBson 

func (shardReference *ShardReference) UnmarshalBson(buf *bytes.Buffer, kind byte)

UnmarshalBson bson-decodes into ShardReference.

type ShardReferenceArray 

type ShardReferenceArray []ShardReference

ShardReferenceArray is used for sorting ShardReference arrays

func (ShardReferenceArray) Len 

func (sra ShardReferenceArray) Len() int

Len implements sort.Interface

func (ShardReferenceArray) Less 

func (sra ShardReferenceArray) Less(i, j int) bool

Len implements sort.Interface

func (ShardReferenceArray) Sort 

func (sra ShardReferenceArray) Sort()

Sort will sort the list according to KeyRange.Start

func (ShardReferenceArray) Swap 

func (sra ShardReferenceArray) Swap(i, j int)

Len implements sort.Interface

type ShardReplicationInfo 

type ShardReplicationInfo struct {
	*pb.ShardReplication
	// contains filtered or unexported fields
}

ShardReplicationInfo is the companion structure for ShardReplication.

func NewShardReplicationInfo 

func NewShardReplicationInfo(sr *pb.ShardReplication, cell, keyspace, shard string) *ShardReplicationInfo

NewShardReplicationInfo is for topo.Server implementations to create the structure

func (*ShardReplicationInfo) Cell 

func (sri *ShardReplicationInfo) Cell() string

Cell returns the cell for a ShardReplicationInfo

func (*ShardReplicationInfo) GetShardReplicationNode 

func (sri *ShardReplicationInfo) GetShardReplicationNode(tabletAlias *pb.TabletAlias) (*pb.ShardReplication_Node, error)

GetShardReplicationNode finds a node for a given tablet.

func (*ShardReplicationInfo) Keyspace 

func (sri *ShardReplicationInfo) Keyspace() string

Keyspace returns the keyspace for a ShardReplicationInfo

func (*ShardReplicationInfo) Shard 

func (sri *ShardReplicationInfo) Shard() string

Shard returns the shard for a ShardReplicationInfo

type SrvKeyspace 

type SrvKeyspace struct {
	// Shards to use per type, only contains complete partitions.
	Partitions map[TabletType]*KeyspacePartition

	// Copied from Keyspace
	ShardingColumnName string
	ShardingColumnType key.KeyspaceIdType
	ServedFrom         map[TabletType]string
	SplitShardCount    int32
}

SrvKeyspace is a distilled serving copy of keyspace detail stored in the local cell for fast access. Derived from the global keyspace, shards and local details. By design, it should not contain details about the Shards themselves, but just which shards to use for serving. In zk, it is in /zk/<cell>/vt/ns/<keyspace>

func ProtoToSrvKeyspace 

func ProtoToSrvKeyspace(s *pb.SrvKeyspace) *SrvKeyspace

ProtoToSrvKeyspace turns a proto to a Tablet

func (*SrvKeyspace) MarshalBson 

func (srvKeyspace *SrvKeyspace) MarshalBson(buf *bytes2.ChunkedWriter, key string)

MarshalBson bson-encodes SrvKeyspace.

func (*SrvKeyspace) UnmarshalBson 

func (srvKeyspace *SrvKeyspace) UnmarshalBson(buf *bytes.Buffer, kind byte)

UnmarshalBson bson-decodes into SrvKeyspace.

type SrvKeyspaceNames 

type SrvKeyspaceNames struct {
	Entries []string
}

SrvKeyspaceNames is the response for TopoReader.GetSrvKeyspaceNames

func (*SrvKeyspaceNames) MarshalBson 

func (srvKeyspaceNames *SrvKeyspaceNames) MarshalBson(buf *bytes2.ChunkedWriter, key string)

MarshalBson bson-encodes SrvKeyspaceNames.

func (*SrvKeyspaceNames) UnmarshalBson 

func (srvKeyspaceNames *SrvKeyspaceNames) UnmarshalBson(buf *bytes.Buffer, kind byte)

UnmarshalBson bson-decodes into SrvKeyspaceNames.

type TabletInfo 

type TabletInfo struct {
	*pb.Tablet
	// contains filtered or unexported fields
}

TabletInfo is the container for a Tablet, read from the topology server.

func NewTabletInfo 

func NewTabletInfo(tablet *pb.Tablet, version int64) *TabletInfo

NewTabletInfo returns a TabletInfo basing on tablet with the version set. This function should be only used by Server implementations.

func (*TabletInfo) Addr 

func (ti *TabletInfo) Addr() string

Addr returns hostname:vt port.

func (*TabletInfo) AliasString 

func (ti *TabletInfo) AliasString() string

AliasString returns the string representation of the tablet alias

func (*TabletInfo) DbName 

func (ti *TabletInfo) DbName() string

DbName is usually implied by keyspace. Having the shard information in the database name complicates mysql replication.

func (*TabletInfo) IsAssigned 

func (ti *TabletInfo) IsAssigned() bool

IsAssigned returns if this tablet ever assigned data? A "scrap" node will show up as assigned even though its data cannot be used for serving.

func (*TabletInfo) IsInReplicationGraph 

func (ti *TabletInfo) IsInReplicationGraph() bool

IsInReplicationGraph returns if this tablet is in the replication graph.

func (*TabletInfo) IsInServingGraph 

func (ti *TabletInfo) IsInServingGraph() bool

IsInServingGraph returns if this tablet is in the serving graph

func (*TabletInfo) IsSlaveType 

func (ti *TabletInfo) IsSlaveType() bool

IsSlaveType returns if this tablet's type is a slave

func (*TabletInfo) MysqlAddr 

func (ti *TabletInfo) MysqlAddr() string

MysqlAddr returns hostname:mysql port.

func (*TabletInfo) String 

func (ti *TabletInfo) String() string

String returns a string describing the tablet.

func (*TabletInfo) Version 

func (ti *TabletInfo) Version() int64

Version returns the version of this tablet from last time it was read or updated.

type TabletType 

type TabletType string

TabletType is the main type for a tablet. It has an implication on: - the replication graph - the services run by vttablet on a tablet - the uptime expectancy

DEPRECATED: use the proto3 topodata.TabletType enum instead.

func ProtoToTabletType 

func ProtoToTabletType(t pb.TabletType) TabletType

ProtoToTabletType turns a proto to a TabletType

func (TabletType) MarshalBson 

func (tabletType TabletType) MarshalBson(buf *bytes2.ChunkedWriter, key string)

MarshalBson bson-encodes TabletType.

func (*TabletType) UnmarshalBson 

func (tabletType *TabletType) UnmarshalBson(buf *bytes.Buffer, kind byte)

UnmarshalBson bson-decodes into TabletType.

type TopoReader 

type TopoReader interface {
	// GetSrvKeyspaces returns the names of all the keyspaces in
	// the topology for the cell.
	GetSrvKeyspaceNames(context.Context, *GetSrvKeyspaceNamesArgs, *SrvKeyspaceNames) error

	// GetSrvKeyspace returns information about a keyspace in a
	// particular cell (as specified by the GetSrvKeyspaceArgs).
	GetSrvKeyspace(context.Context, *GetSrvKeyspaceArgs, *SrvKeyspace) error

	// GetSrvShard returns information about a shard in a
	// particular cell and keyspace (as specified by the GetSrvShardArgs).
	GetSrvShard(context.Context, *GetSrvShardArgs, *pb.SrvShard) error

	// GetEndPoints returns addresses for a tablet type in a shard
	// in a keyspace (as specified in GetEndPointsArgs).
	GetEndPoints(context.Context, *GetEndPointsArgs, *pb.EndPoints) error
}

TopoReader returns read only information about the topology.

type WildcardBackend 

type WildcardBackend interface {
	// GetKeyspaces returns the known keyspaces. They shall be sorted.
	GetKeyspaces(ctx context.Context) ([]string, error)

	// GetShard reads a shard and returns it.
	// Can return ErrNoNode
	GetShard(ctx context.Context, keyspace, shard string) (*ShardInfo, error)

	// GetShardNames returns the known shards in a keyspace.
	// Can return ErrNoNode
	GetShardNames(ctx context.Context, keyspace string) ([]string, error)
}

WildcardBackend is a subset of Server for the methods used by the wildcard code. This lets us test with a very simple fake topo server.

Source Files

View all Source files

Directories

Path Synopsis
Package events defines the structures used for events relating directly to individual topology elements, like keyspaces, shards, and tablets.
 Package events defines the structures used for events relating directly to individual topology elements, like keyspaces, shards, and tablets.
Package helpers contains a few utility classes to handle topo.Server objects, and transitions from one topo implementation to another.
 Package helpers contains a few utility classes to handle topo.Server objects, and transitions from one topo implementation to another.
Package test contains utilities to test topo.Impl implementations.
 Package test contains utilities to test topo.Impl implementations.
faketopo
Package faketopo contains utitlities for tests that have to interact with a Vitess topology.
 Package faketopo contains utitlities for tests that have to interact with a Vitess topology.
Package topoproto contains utility functions to deal with the proto3 structures defined in proto/topodata.
 Package topoproto contains utility functions to deal with the proto3 structures defined in proto/topodata.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.