topo

package
v2.1.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 25, 2017 License: BSD-3-Clause Imports: 29 Imported by: 92

Documentation

Index

Constants

View Source 
const (
	CellInfoFile         = "CellInfo"
	KeyspaceFile         = "Keyspace"
	ShardFile            = "Shard"
	VSchemaFile          = "VSchema"
	ShardReplicationFile = "ShardReplication"
	TabletFile           = "Tablet"
	SrvVSchemaFile       = "SrvVSchema"
	SrvKeyspaceFile      = "SrvKeyspace"
)

Filenames for all object types.

View Source 
const (
	// GlobalCell is the name of the global cell.  It is special
	// as it contains the global topology, and references the other cells.
	GlobalCell = "global"
)

Variables

View Source 
var (
	// DefaultLockTimeout is a good value to use as a default for
	// locking a shard / keyspace.
	DefaultLockTimeout = 30 * time.Second

	// LockTimeout is the command line flag that introduces a shorter
	// timeout for locking topology structures.
	LockTimeout = flag.Duration("lock_timeout", DefaultLockTimeout, "timeout for acquiring topology locks")
)
View Source 
var (
	// ErrNodeExists is returned by functions to specify the
	// requested resource already exists.
	ErrNodeExists = errors.New("node already exists")

	// ErrNoNode is returned by functions to specify the requested
	// resource does not exist.
	ErrNoNode = errors.New("node doesn't exist")

	// ErrNotEmpty is returned by functions to specify a child of the
	// resource is still present and prevents the action from completing.
	ErrNotEmpty = errors.New("node not empty")

	// ErrTimeout is returned by functions that wait for a result
	// when the timeout value is reached.
	ErrTimeout = errors.New("deadline exceeded")

	// ErrInterrupted is returned by functions that wait for a result
	// when they are interrupted.
	ErrInterrupted = errors.New("interrupted")

	// ErrBadVersion is returned by an update function that
	// failed to update the data because the version was different
	ErrBadVersion = errors.New("bad node version")

	// ErrPartialResult is returned by a function that could only
	// get a subset of its results
	ErrPartialResult = errors.New("partial result")

	// ErrNoUpdateNeeded can be returned by an 'UpdateFields' method
	// to skip any update.
	ErrNoUpdateNeeded = errors.New("no update needed")
)

Functions

func CheckKeyspaceLocked 

func CheckKeyspaceLocked(ctx context.Context, keyspace string) error

CheckKeyspaceLocked can be called on a context to make sure we have the lock for a given keyspace.

func CheckShardLocked 

func CheckShardLocked(ctx context.Context, keyspace, shard string) error

CheckShardLocked can be called on a context to make sure we have the lock for a given shard.

func DeleteTabletReplicationData 

func DeleteTabletReplicationData(ctx context.Context, ts Server, tablet *topodatapb.Tablet) error

DeleteTabletReplicationData deletes replication data.

func FixShardReplication 

func FixShardReplication(ctx context.Context, ts Server, logger logutil.Logger, cell, keyspace, shard string) error

FixShardReplication will fix the first problem it encounters within a ShardReplication object.

func InCellList 

func InCellList(cell string, cells []string) bool

InCellList returns true if the cell list is empty, or if the passed cell is in the cell list.

func IsInServingGraph 

func IsInServingGraph(tt topodatapb.TabletType) bool

IsInServingGraph returns if a tablet appears in the serving graph

func IsRunningQueryService 

func IsRunningQueryService(tt topodatapb.TabletType) bool

IsRunningQueryService returns if a tablet is running the query service

func IsRunningUpdateStream 

func IsRunningUpdateStream(tt topodatapb.TabletType) bool

IsRunningUpdateStream returns if a tablet is running the update stream RPC service.

func IsShardUsingRangeBasedSharding 

func IsShardUsingRangeBasedSharding(shard string) bool

IsShardUsingRangeBasedSharding returns true if the shard name implies it is using range based sharding.

func IsSlaveType 

func IsSlaveType(tt topodatapb.TabletType) bool

IsSlaveType returns if this type should be connected to a master db and actively replicating? MASTER is not obviously (only support one level replication graph) BACKUP, RESTORE, DRAINED may or may not be, but we don't know for sure

func IsSubjectToLameduck 

func IsSubjectToLameduck(tt topodatapb.TabletType) bool

IsSubjectToLameduck returns if a tablet is subject to being lameduck. Lameduck is a transition period where we are still allowed to serve, but we tell the clients we are going away soon. Typically, a vttablet will still serve, but broadcast a non-serving state through its health check. then vtgate will ctahc that non-serving state, and stop sending queries.

Masters are not subject to lameduck, as we usually want to transition them as fast as possible.

Replica and rdonly will use lameduck when going from healthy to unhealhty (either because health check fails, or they're shutting down).

Other types are probably not serving user visible traffic, so they need to transition as fast as possible too.

func IsTrivialTypeChange 

func IsTrivialTypeChange(oldTabletType, newTabletType topodatapb.TabletType) bool

IsTrivialTypeChange returns if this db type be trivially reassigned without changes to the replication graph

func NewTablet 

func NewTablet(uid uint32, cell, host string) *topodatapb.Tablet

NewTablet create a new Tablet record with the given id, cell, and hostname.

func RegisterFactory 

func RegisterFactory(name string, factory Factory)

RegisterFactory registers a Factory for an implementation for a Server. If an implementation with that name already exists, it log.Fatals out. Call this in the 'init' function in your topology implementation module.

func RemoveShardReplicationRecord 

func RemoveShardReplicationRecord(ctx context.Context, ts Server, cell, keyspace, shard string, tabletAlias *topodatapb.TabletAlias) error

RemoveShardReplicationRecord is a low level function to remove an entry from the ShardReplication object.

func ResolveKeyspaceWildcard 

func ResolveKeyspaceWildcard(ctx context.Context, server WildcardBackend, param string) ([]string, error)

ResolveKeyspaceWildcard will resolve keyspace wildcards.

  • If the param is not a wildcard, it will just be returned (if the keyspace doesn't exist, it is still returned).
  • If the param is a wildcard, it will get all keyspaces and returns the ones which match the wildcard (which may be an empty list).

func TabletComplete 

func TabletComplete(tablet *topodatapb.Tablet) error

TabletComplete validates and normalizes the tablet. If the shard name contains a '-' it is going to try to infer the keyrange from it.

func TabletEquality 

func TabletEquality(left, right *topodatapb.Tablet) bool

TabletEquality returns true iff two Tablet are representing the same tablet process: same uid/cell, running on the same host / ports.

func UpdateShardReplicationRecord 

func UpdateShardReplicationRecord(ctx context.Context, ts Server, keyspace, shard string, tabletAlias *topodatapb.TabletAlias) error

UpdateShardReplicationRecord is a low level function to add / update an entry to the ShardReplication object.

func UpdateTabletReplicationData 

func UpdateTabletReplicationData(ctx context.Context, ts Server, tablet *topodatapb.Tablet) error

UpdateTabletReplicationData creates or updates the replication graph data for a tablet

func Validate 

func Validate(ctx context.Context, ts Server, tabletAlias *topodatapb.TabletAlias) error

Validate makes sure a tablet is represented correctly in the topology server.

func ValidateShardName 

func ValidateShardName(shard string) (string, *topodatapb.KeyRange, error)

ValidateShardName takes a shard name and sanitizes it, and also returns the KeyRange.

Types

type Backend 

type Backend interface {

	// ListDir returns the entries in a directory.  The returned
	// list should be sorted (by sort.Strings for instance).
	// If there are no files under the provided path, returns ErrNoNode.
	// dirPath is a path relative to the root directory of the cell.
	ListDir(ctx context.Context, cell, dirPath string) ([]string, error)

	// Create creates the initial version of a file.
	// Returns ErrNodeExists if the file exists.
	// filePath is a path relative to the root directory of the cell.
	Create(ctx context.Context, cell, filePath string, contents []byte) (Version, error)

	// Update updates the file with the provided filename with the
	// new content.
	// If version is nil, it is an unconditional update
	// (which is then the same as a Create is the file doesn't exist).
	// filePath is a path relative to the root directory of the cell.
	// It returns the new Version of the file after update.
	// Returns ErrBadVersion if the provided version is not current.
	Update(ctx context.Context, cell, filePath string, contents []byte, version Version) (Version, error)

	// Get returns the content and version of a file.
	// filePath is a path relative to the root directory of the cell.
	// Can return ErrNoNode if the file doesn't exist.
	Get(ctx context.Context, cell, filePath string) ([]byte, Version, error)

	// Delete deletes the provided file.
	// If version is nil, it is an unconditional delete.
	// If the last entry of a directory is deleted, using ListDir
	// on its parent directory should not return the directory.
	// For instance, when deleting /keyspaces/aaa/Keyspace, and if
	// there is no other file in /keyspaces/aaa, then aaa should not
	// appear any more when listing /keyspaces.
	// filePath is a path relative to the root directory of the cell.
	//
	// Delete will never be called on a directory.
	// Returns ErrNodeExists if the file doesn't exist.
	// Returns ErrBadVersion if the provided version is not current.
	Delete(ctx context.Context, cell, filePath string, version Version) error

	// Watch starts watching a file in the provided cell.  It
	// returns the current value, a 'changes' channel to read the
	// changes from, and a 'cancel' function to call to stop the
	// watch.  If the initial read fails, or the file doesn't
	// exist, current.Err is set, and 'changes'/'cancel' are nil.
	// Otherwise current.Err is nil, and current.Contents /
	// current.Version are accurate. The provided context is only
	// used to setup the current watch, and not after Watch()
	// returns.
	//
	// To stop the watch, just call the returned 'cancel' function.
	// This will eventually result in a final WatchData result with Err =
	// ErrInterrupted. It should be safe to call the 'cancel' function
	// multiple times, or after the Watch already errored out.
	//
	// The 'changes' channel may return a record with Err != nil.
	// In that case, the channel will also be closed right after
	// that record.  In any case, 'changes' has to be drained of
	// all events, even when 'stop' is closed.
	//
	// Note the 'changes' channel can return twice the same
	// Version/Contents (for instance, if the watch is interrupted
	// and restarted within the Backend implementation).
	// Similarly, the 'changes' channel may skip versions / changes
	// (that is, if value goes [A, B, C, D, E, F], the watch may only
	// receive [A, B, F]). This should only happen for rapidly
	// changing values though. Usually, the initial value will come
	// back right away. And a stable value (that hasn't changed for
	// a while) should be seen shortly.
	//
	// The Watch call is not guaranteed to return exactly up to
	// date data right away. For instance, if a file is created
	// and saved, and then a watch is set on that file, it may
	// return ErrNoNode (as the underlying configuration service
	// may use asynchronous caches that are not up to date
	// yet). The only guarantee is that the watch data will
	// eventually converge. Vitess doesn't explicitly depend on the data
	// being correct quickly, as long as it eventually gets there.
	//
	// filePath is a path relative to the root directory of the cell.
	Watch(ctx context.Context, cell, filePath string) (current *WatchData, changes <-chan *WatchData, cancel CancelFunc)

	// NewMasterParticipation creates a MasterParticipation
	// object, used to become the Master in an election for the
	// provided group name.  Id is the name of the local process,
	// passing in the hostname:port of the current process as id
	// is the common usage. Id must be unique for each process
	// calling this, for a given name. Calling this function does
	// not make the current process a candidate for the election.
	NewMasterParticipation(name, id string) (MasterParticipation, error)
}

Backend defines the interface that must be implemented by topology plug-ins to be used with Vitess.

Zookeeper is a good example of an implementation, as defined in go/vt/topo/zk2topo.

This API is very generic, and key/value store oriented. We use regular paths for object names, and we can list all immediate children of a path. All paths sent through this API are relative paths, from the root directory of the cell.

FIXME(alainjobart) add all parts of the API, implement them all for all our current systems, and convert the higher levels to talk to this API. This is a long-term project.

type CancelFunc 

type CancelFunc func()

CancelFunc is returned by the Watch method.

type Factory 

type Factory func(serverAddr, root string) (Impl, error)

Factory is a factory method to create Impl objects.

type Impl 

type Impl interface {
	// Impl will eventually be entirely replaced with Backend, and
	// just disappear.
	Backend

	// topo.Server management interface.
	Close()

	// GetKnownCells returns the list of known cells running our processes.
	// It is possible to find all tablets in the entire system
	// by then calling GetTabletsByCell on every cell, for instance.
	// They shall be sorted.
	GetKnownCells(ctx context.Context) ([]string, error)

	// CreateKeyspace creates the given keyspace, assuming it doesn't exist
	// yet. Can return ErrNodeExists if it already exists.
	CreateKeyspace(ctx context.Context, keyspace string, value *topodatapb.Keyspace) error

	// UpdateKeyspace updates the keyspace information
	// pointed at by ki.keyspace to the *ki value.
	// This will only be called with a lock on the keyspace.
	// Can return ErrNoNode if the keyspace doesn't exist yet,
	// or ErrBadVersion if the version has changed.
	//
	// Do not use directly, but instead use Server.UpdateKeyspace.
	UpdateKeyspace(ctx context.Context, keyspace string, value *topodatapb.Keyspace, existingVersion int64) (newVersion int64, err error)

	// DeleteKeyspace deletes the specified keyspace.
	// Can return ErrNoNode if the keyspace doesn't exist.
	DeleteKeyspace(ctx context.Context, keyspace string) error

	// GetKeyspace reads a keyspace and returns it, along with its version.
	// Can return ErrNoNode
	GetKeyspace(ctx context.Context, keyspace string) (*topodatapb.Keyspace, int64, error)

	// GetKeyspaces returns the known keyspace names. They shall be sorted.
	GetKeyspaces(ctx context.Context) ([]string, error)

	// CreateShard creates a shard, assuming it doesn't exist yet.
	// Can return ErrNodeExists if it already exists.
	CreateShard(ctx context.Context, keyspace, shard string, value *topodatapb.Shard) error

	// UpdateShard updates the shard information
	// pointed at by si.keyspace / si.shard to the *si value.
	// Can return ErrNoNode if the shard doesn't exist yet,
	// or ErrBadVersion if the version has changed.
	//
	// Do not use directly, but instead use topo.UpdateShardFields.
	UpdateShard(ctx context.Context, keyspace, shard string, value *topodatapb.Shard, existingVersion int64) (newVersion int64, err error)

	// GetShard reads a shard and returns it, along with its version.
	// Can return ErrNoNode
	GetShard(ctx context.Context, keyspace, shard string) (*topodatapb.Shard, int64, error)

	// GetShardNames returns the known shards in a keyspace.
	// Can return ErrNoNode if the keyspace wasn't created.
	// Will return an empty list if the keyspace has no shard.
	// They shall be sorted.
	GetShardNames(ctx context.Context, keyspace string) ([]string, error)

	// DeleteShard deletes the provided shard.
	// Can return ErrNoNode if the shard doesn't exist.
	DeleteShard(ctx context.Context, keyspace, shard string) error

	// CreateTablet creates the given tablet, assuming it doesn't exist
	// yet. It does *not* create the tablet replication paths.
	// Can return ErrNodeExists if it already exists.
	CreateTablet(ctx context.Context, tablet *topodatapb.Tablet) error

	// UpdateTablet updates a given tablet. The version is used
	// for atomic updates. UpdateTablet will return ErrNoNode if
	// the tablet doesn't exist and ErrBadVersion if the version
	// has changed.
	//
	// Do not use directly, but instead use topo.UpdateTablet.
	UpdateTablet(ctx context.Context, tablet *topodatapb.Tablet, existingVersion int64) (newVersion int64, err error)

	// DeleteTablet removes a tablet from the system.
	// We assume no RPC is currently running to it.
	// TODO(alainjobart) verify this assumption, link with RPC code.
	// Can return ErrNoNode if the tablet doesn't exist.
	DeleteTablet(ctx context.Context, alias *topodatapb.TabletAlias) error

	// GetTablet returns the tablet data (includes the current version).
	// Can return ErrNoNode if the tablet doesn't exist.
	GetTablet(ctx context.Context, alias *topodatapb.TabletAlias) (*topodatapb.Tablet, int64, error)

	// GetTabletsByCell returns all the tablets in the given cell.
	// Can return ErrNoNode if no tablet was ever created in that cell.
	GetTabletsByCell(ctx context.Context, cell string) ([]*topodatapb.TabletAlias, error)

	// UpdateShardReplicationFields updates the current
	// ShardReplication record with new values. If the
	// ShardReplication object does not exist, an empty one will
	// be passed to the update function. All necessary directories
	// need to be created by this method, if applicable.
	UpdateShardReplicationFields(ctx context.Context, cell, keyspace, shard string, update func(*topodatapb.ShardReplication) error) error

	// GetShardReplication returns the replication data.
	// Can return ErrNoNode if the object doesn't exist.
	GetShardReplication(ctx context.Context, cell, keyspace, shard string) (*ShardReplicationInfo, error)

	// DeleteShardReplication deletes the replication data.
	// Can return ErrNoNode if the object doesn't exist.
	DeleteShardReplication(ctx context.Context, cell, keyspace, shard string) error

	// DeleteKeyspaceReplication deletes the replication data for all shards.
	// Can return ErrNoNode if the object doesn't exist.
	DeleteKeyspaceReplication(ctx context.Context, cell, keyspace string) error

	// GetSrvKeyspaceNames returns the list of visible Keyspaces
	// in this cell. They shall be sorted.
	GetSrvKeyspaceNames(ctx context.Context, cell string) ([]string, error)

	// UpdateSrvKeyspace updates the serving records for a cell, keyspace.
	UpdateSrvKeyspace(ctx context.Context, cell, keyspace string, srvKeyspace *topodatapb.SrvKeyspace) error

	// DeleteSrvKeyspace deletes the cell-local serving records for a keyspace.
	// Can return ErrNoNode.
	DeleteSrvKeyspace(ctx context.Context, cell, keyspace string) error

	// GetSrvKeyspace reads a SrvKeyspace record.
	// Can return ErrNoNode.
	GetSrvKeyspace(ctx context.Context, cell, keyspace string) (*topodatapb.SrvKeyspace, error)

	// UpdateSrvVSchema updates the serving records for a cell.
	UpdateSrvVSchema(ctx context.Context, cell string, srvVSchema *vschemapb.SrvVSchema) error

	// GetSrvVSchema reads a SrvVSchema record.
	// Can return ErrNoNode.
	GetSrvVSchema(ctx context.Context, cell string) (*vschemapb.SrvVSchema, error)

	// LockKeyspaceForAction locks the keyspace in order to
	// perform the action described by contents. It will wait for
	// the lock until at most ctx.Done(). The wait can be interrupted
	// by cancelling the context. It returns the lock path.
	//
	// Can return ErrTimeout or ErrInterrupted
	LockKeyspaceForAction(ctx context.Context, keyspace, contents string) (string, error)

	// UnlockKeyspaceForAction unlocks a keyspace.
	UnlockKeyspaceForAction(ctx context.Context, keyspace, lockPath, results string) error

	// LockShardForAction locks the shard in order to
	// perform the action described by contents. It will wait for
	// the lock until at most ctx.Done(). The wait can be interrupted
	// by cancelling the context. It returns the lock path.
	//
	// Can return ErrTimeout or ErrInterrupted
	LockShardForAction(ctx context.Context, keyspace, shard, contents string) (string, error)

	// UnlockShardForAction unlocks a shard.
	UnlockShardForAction(ctx context.Context, keyspace, shard, lockPath, results string) error

	// SaveVSchema saves the provided schema in the topo server.
	SaveVSchema(ctx context.Context, keyspace string, vschema *vschemapb.Keyspace) error

	// GetVSchema retrieves the schema from the topo server.
	//
	// Can return ErrNoNode
	GetVSchema(ctx context.Context, keyspace string) (*vschemapb.Keyspace, error)
}

Impl is the interface used to talk to a persistent backend storage server and locking service.

Zookeeper is a good example of this, and go/vt/topo/zk2topo contains the implementation for this using zookeeper.

Inside Google, we use Chubby.

FIXME(alainjobart) we are deprecating this interface, to be replaced with a lower level interface defined by Backend.

type KeyspaceInfo 

type KeyspaceInfo struct {
	*topodatapb.Keyspace
	// contains filtered or unexported fields
}

KeyspaceInfo is a meta struct that contains metadata to give the data more context and convenience. This is the main way we interact with a keyspace.

func (*KeyspaceInfo) CheckServedFromMigration 

func (ki *KeyspaceInfo) CheckServedFromMigration(tabletType topodatapb.TabletType, cells []string, keyspace string, remove bool) error

CheckServedFromMigration makes sure a requested migration is safe

func (*KeyspaceInfo) ComputeCellServedFrom 

func (ki *KeyspaceInfo) ComputeCellServedFrom(cell string) []*topodatapb.SrvKeyspace_ServedFrom

ComputeCellServedFrom returns the ServedFrom list for a cell

func (*KeyspaceInfo) GetServedFrom 

func (ki *KeyspaceInfo) GetServedFrom(tabletType topodatapb.TabletType) *topodatapb.Keyspace_ServedFrom

GetServedFrom returns a Keyspace_ServedFrom record if it exists.

func (*KeyspaceInfo) KeyspaceName 

func (ki *KeyspaceInfo) KeyspaceName() string

KeyspaceName returns the keyspace name

func (*KeyspaceInfo) UpdateServedFromMap 

func (ki *KeyspaceInfo) UpdateServedFromMap(tabletType topodatapb.TabletType, cells []string, keyspace string, remove bool, allCells []string) error

UpdateServedFromMap handles ServedFromMap. It can add or remove records, cells, ...

type KeyspaceShard 

type KeyspaceShard struct {
	Keyspace string
	Shard    string
}

KeyspaceShard is a type used by ResolveShardWildcard

func ResolveShardWildcard 

func ResolveShardWildcard(ctx context.Context, server WildcardBackend, param string) ([]KeyspaceShard, error)

ResolveShardWildcard will resolve shard wildcards. Both keyspace and shard names can use wildcard. Errors talking to the topology server are returned. ErrNoNode is ignored if it's the result of resolving a wildcard. Examples:

  • */* returns all keyspace/shard pairs, or empty list if none.
  • user/* returns all shards in user keyspace (or error if user keyspace doesn't exist)
  • us*/* returns all shards in all keyspaces that start with 'us'. If no such keyspace exists, list is empty (it is not an error).

type Lock 

type Lock struct {
	// Action and the following fields are set at construction time.
	Action   string
	HostName string
	UserName string
	Time     string

	// Status is the current status of the Lock.
	Status string
}

Lock describes a long-running lock on a keyspace or a shard. It needs to be public as we JSON-serialize it.

func (*Lock) ToJSON 

func (l *Lock) ToJSON() (string, error)

ToJSON returns a JSON representation of the object.

type LockDescriptor 

type LockDescriptor interface {
	// String returns a text representation of the lock.
	String() string
}

LockDescriptor is an interface that describes a lock.

type MasterParticipation 

type MasterParticipation interface {
	// WaitForMastership makes the current process a candidate
	// for election, and waits until this process is the master.
	// After we become the master, we may lose mastership. In that case,
	// the returned context will be canceled. If Stop was called,
	// WaitForMastership will return nil, ErrInterrupted.
	WaitForMastership() (context.Context, error)

	// Stop is called when we don't want to participate in the
	// master election any more. Typically, that is when the
	// hosting process is terminating.  We will relinquish
	// mastership at that point, if we had it. Stop should
	// not return until everything has been done.
	// The MasterParticipation object should be discarded
	// after Stop has been called. Any call to WaitForMastership
	// after Stop() will return nil, ErrInterrupted.
	// If WaitForMastership() was running, it will return
	// nil, ErrInterrupted as soon as possible.
	Stop()

	// GetCurrentMasterID returns the current master id.
	// This may not work after Stop has been called.
	GetCurrentMasterID(ctx context.Context) (string, error)
}

MasterParticipation is the object returned by NewMasterParticipation. Sample usage:

mp := server.NewMasterParticipation("vtctld", "hostname:8080") job := NewJob() 

go func() {
  for {
    ctx, err := mp.WaitForMastership()
    switch err {
    case nil:
      job.RunUntilContextDone(ctx)
    case topo.ErrInterrupted:
      return
    default:
      log.Errorf("Got error while waiting for master, will retry in 5s: %v", err)
      time.Sleep(5 * time.Second)
    }
  }
}()

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
  if job.Running() {
    job.WriteStatus(w, r)
  } else {
    http.Redirect(w, r, mp.GetCurrentMasterID(context.Background()), http.StatusFound)
  }
})

servenv.OnTermSync(func() {
  mp.Stop()
})

type Server 

type Server struct {
	Impl
}

Server is a wrapper type that can have extra methods. Outside modules should just use the Server object.

func Open 

func Open() Server

Open returns a Server using the command line parameter flags for implementation, address and root. It log.Fatals out if an error occurs.

func OpenServer 

func OpenServer(implementation, serverAddress, root string) (Server, error)

OpenServer returns a Server using the provided implementation, address and root.

func (Server) CreateCellInfo 

func (ts Server) CreateCellInfo(ctx context.Context, cell string, ci *topodatapb.CellInfo) error

CreateCellInfo creates a new CellInfo with the provided content.

func (Server) CreateKeyspace 

func (ts Server) CreateKeyspace(ctx context.Context, keyspace string, value *topodatapb.Keyspace) error

CreateKeyspace wraps the underlying Impl.CreateKeyspace and dispatches the event.

func (Server) CreateShard 

func (ts Server) CreateShard(ctx context.Context, keyspace, shard string) (err error)

CreateShard creates a new shard and tries to fill in the right information. This will lock the Keyspace, as we may be looking at other shard servedTypes. Using GetOrCreateShard is probably a better idea for most use cases.

func (Server) CreateTablet 

func (ts Server) CreateTablet(ctx context.Context, tablet *topodatapb.Tablet) error

CreateTablet creates a new tablet and all associated paths for the replication graph.

func (Server) CreateWorkflow 

func (ts Server) CreateWorkflow(ctx context.Context, w *workflowpb.Workflow) (*WorkflowInfo, error)

CreateWorkflow creates the given workflow, and returns the initial WorkflowInfo.

func (Server) DeleteCellInfo 

func (ts Server) DeleteCellInfo(ctx context.Context, cell string) error

DeleteCellInfo deletes the specified CellInfo. We first make sure no Shard record points to the cell.

func (Server) DeleteKeyspace 

func (ts Server) DeleteKeyspace(ctx context.Context, keyspace string) error

DeleteKeyspace wraps the underlying Impl.DeleteKeyspace and dispatches the event.

func (Server) DeleteShard 

func (ts Server) DeleteShard(ctx context.Context, keyspace, shard string) error

DeleteShard wraps the underlying Impl.DeleteShard and dispatches the event.

func (Server) DeleteTablet 

func (ts Server) DeleteTablet(ctx context.Context, tabletAlias *topodatapb.TabletAlias) error

DeleteTablet wraps the underlying Impl.DeleteTablet and dispatches the event.

func (Server) DeleteWorkflow 

func (ts Server) DeleteWorkflow(ctx context.Context, wi *WorkflowInfo) error

DeleteWorkflow deletes the specified workflow. After this, the WorkflowInfo object should not be used any more.

func (Server) FindAllShardsInKeyspace 

func (ts Server) FindAllShardsInKeyspace(ctx context.Context, keyspace string) (map[string]*ShardInfo, error)

FindAllShardsInKeyspace reads and returns all the existing shards in a keyspace. It doesn't take any lock.

func (Server) FindAllTabletAliasesInShard 

func (ts Server) FindAllTabletAliasesInShard(ctx context.Context, keyspace, shard string) ([]*topodatapb.TabletAlias, error)

FindAllTabletAliasesInShard uses the replication graph to find all the tablet aliases in the given shard.

It can return ErrPartialResult if some cells were not fetched, in which case the result only contains the cells that were fetched.

The tablet aliases are sorted by cell, then by UID.

func (Server) FindAllTabletAliasesInShardByCell 

func (ts Server) FindAllTabletAliasesInShardByCell(ctx context.Context, keyspace, shard string, cells []string) ([]*topodatapb.TabletAlias, error)

FindAllTabletAliasesInShardByCell uses the replication graph to find all the tablet aliases in the given shard.

It can return ErrPartialResult if some cells were not fetched, in which case the result only contains the cells that were fetched.

The tablet aliases are sorted by cell, then by UID.

func (Server) GetCellInfo 

func (ts Server) GetCellInfo(ctx context.Context, cell string) (*topodatapb.CellInfo, error)

GetCellInfo reads a CellInfo from the Backend.

func (Server) GetCellInfoNames 

func (ts Server) GetCellInfoNames(ctx context.Context) ([]string, error)

GetCellInfoNames returns the names of the existing cells. They are sorted by name.

func (Server) GetKeyspace 

func (ts Server) GetKeyspace(ctx context.Context, keyspace string) (*KeyspaceInfo, error)

GetKeyspace reads the given keyspace and returns it

func (Server) GetOrCreateShard 

func (ts Server) GetOrCreateShard(ctx context.Context, keyspace, shard string) (si *ShardInfo, err error)

GetOrCreateShard will return the shard object, or create one if it doesn't already exist. Note the shard creation is protected by a keyspace Lock.

func (Server) GetShard 

func (ts Server) GetShard(ctx context.Context, keyspace, shard string) (*ShardInfo, error)

GetShard is a high level function to read shard data. It generates trace spans.

func (Server) GetTablet 

func (ts Server) GetTablet(ctx context.Context, alias *topodatapb.TabletAlias) (*TabletInfo, error)

GetTablet is a high level function to read tablet data. It generates trace spans.

func (Server) GetTabletMap 

func (ts Server) GetTabletMap(ctx context.Context, tabletAliases []*topodatapb.TabletAlias) (map[topodatapb.TabletAlias]*TabletInfo, error)

GetTabletMap tries to read all the tablets in the provided list, and returns them all in a map. If error is ErrPartialResult, the results in the dictionary are incomplete, meaning some tablets couldn't be read.

func (Server) GetTabletMapForShard 

func (ts Server) GetTabletMapForShard(ctx context.Context, keyspace, shard string) (map[topodatapb.TabletAlias]*TabletInfo, error)

GetTabletMapForShard returns the tablets for a shard. It can return ErrPartialResult if it couldn't read all the cells, or all the individual tablets, in which case the map is valid, but partial.

func (Server) GetTabletMapForShardByCell 

func (ts Server) GetTabletMapForShardByCell(ctx context.Context, keyspace, shard string, cells []string) (map[topodatapb.TabletAlias]*TabletInfo, error)

GetTabletMapForShardByCell returns the tablets for a shard. It can return ErrPartialResult if it couldn't read all the cells, or all the individual tablets, in which case the map is valid, but partial.

func (Server) GetWorkflow 

func (ts Server) GetWorkflow(ctx context.Context, uuid string) (*WorkflowInfo, error)

GetWorkflow reads a workflow from the Backend.

func (Server) GetWorkflowNames 

func (ts Server) GetWorkflowNames(ctx context.Context) ([]string, error)

GetWorkflowNames returns the names of the existing workflows. They are sorted by uuid.

func (Server) LockKeyspace 

func (ts Server) LockKeyspace(ctx context.Context, keyspace, action string) (context.Context, func(*error), error)

LockKeyspace will lock the keyspace, and return: - a context with a locksInfo structure for future reference. - an unlock method - an error if anything failed.

We lock a keyspace for the following operations to be guaranteed exclusive operation: * changing a keyspace sharding info fields (is this one necessary?) * changing a keyspace 'ServedFrom' field (is this one necessary?) * resharding operations:

  • horizontal resharding: includes changing the shard's 'ServedType', as well as the associated horizontal resharding operations.
  • vertical resharding: includes changing the keyspace 'ServedFrom' field, as well as the associated vertical resharding operations.
  • 'vtctl SetShardServedTypes' emergency operations
  • 'vtctl SetShardTabletControl' emergency operations
  • 'vtctl SourceShardAdd' and 'vtctl SourceShardDelete' emergency operations

* keyspace-wide schema changes

func (Server) LockShard 

func (ts Server) LockShard(ctx context.Context, keyspace, shard, action string) (context.Context, func(*error), error)

LockShard will lock the shard, and return: - a context with a locksInfo structure for future reference. - an unlock method - an error if anything failed.

We are currently only using this method to lock actions that would impact each-other. Most changes of the Shard object are done by UpdateShardFields, which is not locking the shard object. The current list of actions that lock a shard are: * all Vitess-controlled re-parenting operations:

  • InitShardMaster
  • PlannedReparentShard
  • EmergencyReparentShard

* operations that we don't want to conflict with re-parenting:

  • DeleteTablet when it's the shard's current master

func (Server) SaveVSchema 

func (ts Server) SaveVSchema(ctx context.Context, keyspace string, vschema *vschemapb.Keyspace) error

SaveVSchema first validates the VSchema, then sends it to the underlying Impl.

func (Server) SaveWorkflow 

func (ts Server) SaveWorkflow(ctx context.Context, wi *WorkflowInfo) error

SaveWorkflow saves the WorkflowInfo object. If the version is not good any more, ErrBadVersion is returned.

func (Server) UpdateCellInfoFields 

func (ts Server) UpdateCellInfoFields(ctx context.Context, cell string, update func(*topodatapb.CellInfo) error) error

UpdateCellInfoFields is a high level helper method to read a CellInfo object, update its fields, and then write it back. If the write fails due to a version mismatch, it will re-read the record and retry the update. If the update method returns ErrNoUpdateNeeded, nothing is written, and nil is returned.

func (Server) UpdateKeyspace 

func (ts Server) UpdateKeyspace(ctx context.Context, ki *KeyspaceInfo) error

UpdateKeyspace updates the keyspace data. It checks the keyspace is locked.

func (Server) UpdateShard 

func (ts Server) UpdateShard() error

UpdateShard masks ts.Impl.UpdateShard so nobody is tempted to use it.

func (Server) UpdateShardFields 

func (ts Server) UpdateShardFields(ctx context.Context, keyspace, shard string, update func(*ShardInfo) error) (*ShardInfo, error)

UpdateShardFields is a high level helper to read a shard record, call an update function on it, and then write it back. If the write fails due to a version mismatch, it will re-read the record and retry the update. If the update succeeds, it returns the updated ShardInfo. If the update method returns ErrNoUpdateNeeded, nothing is written, and nil,nil is returned.

Note the callback method takes a ShardInfo, so it can get the keyspace and shard from it, or use all the ShardInfo methods.

func (Server) UpdateTablet 

func (ts Server) UpdateTablet(ctx context.Context, tablet *TabletInfo) error

UpdateTablet updates the tablet data only - not associated replication paths. It also uses a span, and sends the event.

func (Server) UpdateTabletFields 

func (ts Server) UpdateTabletFields(ctx context.Context, alias *topodatapb.TabletAlias, update func(*topodatapb.Tablet) error) (*topodatapb.Tablet, error)

UpdateTabletFields is a high level helper to read a tablet record, call an update function on it, and then write it back. If the write fails due to a version mismatch, it will re-read the record and retry the update. If the update succeeds, it returns the updated tablet. If the update method returns ErrNoUpdateNeeded, nothing is written, and nil,nil is returned.

func (Server) WatchSrvKeyspace 

func (ts Server) WatchSrvKeyspace(ctx context.Context, cell, keyspace string) (*WatchSrvKeyspaceData, <-chan *WatchSrvKeyspaceData, CancelFunc)

WatchSrvKeyspace will set a watch on the SrvKeyspace object. It has the same contract as Backend.Watch, but it also unpacks the contents into a SrvKeyspace object.

func (Server) WatchSrvVSchema 

func (ts Server) WatchSrvVSchema(ctx context.Context, cell string) (*WatchSrvVSchemaData, <-chan *WatchSrvVSchemaData, CancelFunc)

WatchSrvVSchema will set a watch on the SrvVSchema object. It has the same contract as Backend.Watch, but it also unpacks the contents into a SrvVSchema object.

type ShardInfo 

type ShardInfo struct {
	*topodatapb.Shard
	// contains filtered or unexported fields
}

ShardInfo is a meta struct that contains metadata to give the data more context and convenience. This is the main way we interact with a shard.

func NewShardInfo 

func NewShardInfo(keyspace, shard string, value *topodatapb.Shard, version int64) *ShardInfo

NewShardInfo returns a ShardInfo basing on shard with the keyspace / shard. This function should be only used by Server implementations.

func (*ShardInfo) CheckServedTypesMigration 

func (si *ShardInfo) CheckServedTypesMigration(tabletType topodatapb.TabletType, cells []string, remove bool) error

CheckServedTypesMigration makes sure the provided migration is possible

func (*ShardInfo) GetServedType 

func (si *ShardInfo) GetServedType(tabletType topodatapb.TabletType) *topodatapb.Shard_ServedType

GetServedType returns the Shard_ServedType for a TabletType, or nil

func (*ShardInfo) GetServedTypesPerCell 

func (si *ShardInfo) GetServedTypesPerCell(cell string) []topodatapb.TabletType

GetServedTypesPerCell returns the list of types this shard is serving in the provided cell.

func (*ShardInfo) GetTabletControl 

func (si *ShardInfo) GetTabletControl(tabletType topodatapb.TabletType) *topodatapb.Shard_TabletControl

GetTabletControl returns the Shard_TabletControl for the given tablet type, or nil if it is not in the map.

func (*ShardInfo) HasCell 

func (si *ShardInfo) HasCell(cell string) bool

HasCell returns true if the cell is listed in the Cells for the shard.

func (*ShardInfo) HasMaster 

func (si *ShardInfo) HasMaster() bool

HasMaster returns true if the Shard has an assigned Master.

func (*ShardInfo) Keyspace 

func (si *ShardInfo) Keyspace() string

Keyspace returns the keyspace a shard belongs to.

func (*ShardInfo) ShardName 

func (si *ShardInfo) ShardName() string

ShardName returns the shard name for a shard.

func (*ShardInfo) UpdateDisableQueryService 

func (si *ShardInfo) UpdateDisableQueryService(ctx context.Context, tabletType topodatapb.TabletType, cells []string, disableQueryService bool) error

UpdateDisableQueryService will make sure the disableQueryService is set appropriately in the shard record. Note we don't support a lot of the corner cases:

  • we don't support DisableQueryService at the same time as BlacklistedTables, because it's not used in the same context (vertical vs horizontal sharding)

This function should be called while holding the keyspace lock.

func (*ShardInfo) UpdateServedTypesMap 

func (si *ShardInfo) UpdateServedTypesMap(tabletType topodatapb.TabletType, cells []string, remove bool) error

UpdateServedTypesMap handles ServedTypesMap. It can add or remove records, cells, ...

func (*ShardInfo) UpdateSourceBlacklistedTables 

func (si *ShardInfo) UpdateSourceBlacklistedTables(ctx context.Context, tabletType topodatapb.TabletType, cells []string, remove bool, tables []string) error

UpdateSourceBlacklistedTables will add or remove the listed tables in the shard record's TabletControl structures. Note we don't support a lot of the corner cases:

  • only support one table list per shard. If we encounter a different table list that the provided one, we error out.
  • we don't support DisableQueryService at the same time as BlacklistedTables, because it's not used in the same context (vertical vs horizontal sharding)

This function should be called while holding the keyspace lock.

func (*ShardInfo) Version 

func (si *ShardInfo) Version() int64

Version returns the shard version from last time it was read or updated.

type ShardReplicationInfo 

type ShardReplicationInfo struct {
	*topodatapb.ShardReplication
	// contains filtered or unexported fields
}

ShardReplicationInfo is the companion structure for ShardReplication.

func NewShardReplicationInfo 

func NewShardReplicationInfo(sr *topodatapb.ShardReplication, cell, keyspace, shard string) *ShardReplicationInfo

NewShardReplicationInfo is for topo.Server implementations to create the structure

func (*ShardReplicationInfo) Cell 

func (sri *ShardReplicationInfo) Cell() string

Cell returns the cell for a ShardReplicationInfo

func (*ShardReplicationInfo) GetShardReplicationNode 

func (sri *ShardReplicationInfo) GetShardReplicationNode(tabletAlias *topodatapb.TabletAlias) (*topodatapb.ShardReplication_Node, error)

GetShardReplicationNode finds a node for a given tablet.

func (*ShardReplicationInfo) Keyspace 

func (sri *ShardReplicationInfo) Keyspace() string

Keyspace returns the keyspace for a ShardReplicationInfo

func (*ShardReplicationInfo) Shard 

func (sri *ShardReplicationInfo) Shard() string

Shard returns the shard for a ShardReplicationInfo

type SrvTopoServer 

type SrvTopoServer interface {
	GetSrvKeyspaceNames(ctx context.Context, cell string) ([]string, error)
	GetSrvKeyspace(ctx context.Context, cell, keyspace string) (*topodatapb.SrvKeyspace, error)
	WatchSrvVSchema(ctx context.Context, cell string) (*WatchSrvVSchemaData, <-chan *WatchSrvVSchemaData, CancelFunc)
}

SrvTopoServer is a subset of the Server API that only contains the serving graph read-only calls used by clients to resolve serving addresses, and how to get VSchema. It is mostly used by our discovery modules, and by vtgate.

type TabletInfo 

type TabletInfo struct {
	*topodatapb.Tablet
	// contains filtered or unexported fields
}

TabletInfo is the container for a Tablet, read from the topology server.

func NewTabletInfo 

func NewTabletInfo(tablet *topodatapb.Tablet, version int64) *TabletInfo

NewTabletInfo returns a TabletInfo basing on tablet with the version set. This function should be only used by Server implementations.

func (*TabletInfo) Addr 

func (ti *TabletInfo) Addr() string

Addr returns hostname:vt port.

func (*TabletInfo) AliasString 

func (ti *TabletInfo) AliasString() string

AliasString returns the string representation of the tablet alias

func (*TabletInfo) DbName 

func (ti *TabletInfo) DbName() string

DbName is usually implied by keyspace. Having the shard information in the database name complicates mysql replication.

func (*TabletInfo) IsInServingGraph 

func (ti *TabletInfo) IsInServingGraph() bool

IsInServingGraph returns if this tablet is in the serving graph

func (*TabletInfo) IsSlaveType 

func (ti *TabletInfo) IsSlaveType() bool

IsSlaveType returns if this tablet's type is a slave

func (*TabletInfo) MysqlAddr 

func (ti *TabletInfo) MysqlAddr() string

MysqlAddr returns hostname:mysql port.

func (*TabletInfo) String 

func (ti *TabletInfo) String() string

String returns a string describing the tablet.

func (*TabletInfo) Version 

func (ti *TabletInfo) Version() int64

Version returns the version of this tablet from last time it was read or updated.

type Version 

type Version interface {
	// String returns a text representation of the version.
	String() string
}

Version is an interface that describes a file version.

type WatchData 

type WatchData struct {
	// Contents has the bytes that were stored by Create
	// or Update.
	Contents []byte

	// Version contains an opaque representation of the Version
	// of that file.
	Version Version

	// Err is set the same way for both the 'current' value
	// returned by Watch, or the values read on the 'changes'
	// channel. It can be:
	// - nil, then Contents and Version are set.
	// - ErrNoNode if the file doesn't exist.
	// - ErrInterrupted if 'cancel' was called.
	// - any other platform-specific error.
	Err error
}

WatchData is the structure returned by the Watch() API. It can contain: a) an error in Err if the call failed (or if the watch was terminated). b) the current or new version of the data.

type WatchSrvKeyspaceData 

type WatchSrvKeyspaceData struct {
	Value *topodatapb.SrvKeyspace
	Err   error
}

WatchSrvKeyspaceData is returned / streamed by WatchSrvKeyspace. The WatchSrvKeyspace API guarantees exactly one of Value or Err will be set.

type WatchSrvVSchemaData 

type WatchSrvVSchemaData struct {
	Value *vschemapb.SrvVSchema
	Err   error
}

WatchSrvVSchemaData is returned / streamed by WatchSrvVSchema. The WatchSrvVSchema API guarantees exactly one of Value or Err will be set.

type WildcardBackend 

type WildcardBackend interface {
	// GetKeyspaces returns the known keyspaces. They shall be sorted.
	GetKeyspaces(ctx context.Context) ([]string, error)

	// GetShard reads a shard and returns it.
	// Can return ErrNoNode
	GetShard(ctx context.Context, keyspace, shard string) (*ShardInfo, error)

	// GetShardNames returns the known shards in a keyspace.
	// Can return ErrNoNode
	GetShardNames(ctx context.Context, keyspace string) ([]string, error)
}

WildcardBackend is a subset of Server for the methods used by the wildcard code. This lets us test with a very simple fake topo server.

type WorkflowInfo 

type WorkflowInfo struct {
	*workflowpb.Workflow
	// contains filtered or unexported fields
}

WorkflowInfo is a meta struct that contains the version of a Workflow.

Source Files

View all Source files

Directories

Path Synopsis
Package consultopo implements topo.Server with consul as the backend.
 Package consultopo implements topo.Server with consul as the backend.
Package etcd2topo implements topo.Server with etcd as the backend.
 Package etcd2topo implements topo.Server with etcd as the backend.
Package events defines the structures used for events relating directly to individual topology elements, like keyspaces, shards, and tablets.
 Package events defines the structures used for events relating directly to individual topology elements, like keyspaces, shards, and tablets.
Package helpers contains a few utility classes to handle topo.Server objects, and transitions from one topo implementation to another.
 Package helpers contains a few utility classes to handle topo.Server objects, and transitions from one topo implementation to another.
Package memorytopo contains an implementation of the topo.Backend API based on an in-process memory map.
 Package memorytopo contains an implementation of the topo.Backend API based on an in-process memory map.
Package test contains utilities to test topo.Impl implementations.
 Package test contains utilities to test topo.Impl implementations.
faketopo
Package faketopo contains utitlities for tests that have to interact with a Vitess topology.
 Package faketopo contains utitlities for tests that have to interact with a Vitess topology.
Package topoproto contains utility functions to deal with the proto3 structures defined in proto/topodata.
 Package topoproto contains utility functions to deal with the proto3 structures defined in proto/topodata.
Package topotests contains all the unit tests for the topo.Server code that is based on topo.Backend.
 Package topotests contains all the unit tests for the topo.Server code that is based on topo.Backend.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.