topo

package
v0.21.0-rc2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 23, 2024 License: Apache-2.0 Imports: 41 Imported by: 92

Documentation

Overview

Package topo is the module responsible for interacting with the topology service. It uses one Conn connection to the global topo service (with possibly another one to a read-only version of the global topo service), and one to each cell topo service.

It contains the plug-in interfaces Conn, Factory and Version that topo implementations will use. We support Zookeeper, etcd, consul as real topo servers, and in-memory, tee as test and utility topo servers. Implementations are in sub-directories here.

In tests, we do not mock this package. Instead, we just use a memorytopo.

We also support copying data across topo servers (using helpers/copy.go and the topo2topo cmd binary), and writing to two topo servers at the same time (using helpers/tee.go). This is to facilitate migrations between topo servers.

There are two test sub-packages associated with this code:

  • test/ contains a test suite that is run against all of our implementations. It just performs a bunch of common topo server activities (create, list, delete various objects, ...). If a topo implementation passes all these tests, it most likely will work as expected in a real deployment.
  • topotests/ contains tests that use a memorytopo to test the code in this package.

Index

Constants

View Source 
const (
	NodeExists = ErrorCode(iota)
	NoNode
	NodeNotEmpty
	Timeout
	Interrupted
	BadVersion
	PartialResult
	NoUpdateNeeded
	NoImplementation
	NoReadOnlyImplementation
	ResourceExhausted
)

The following is the list of error codes.

View Source 
const (
	// GlobalCell is the name of the global cell.  It is special
	// as it contains the global topology, and references the other cells.
	GlobalCell = "global"

	// GlobalReadOnlyCell is the name of the global read-only cell
	// connection cell name.
	GlobalReadOnlyCell = "global-read-only"
)
View Source 
const (
	CellInfoFile           = "CellInfo"
	CellsAliasFile         = "CellsAlias"
	KeyspaceFile           = "Keyspace"
	ShardFile              = "Shard"
	VSchemaFile            = "VSchema"
	ShardReplicationFile   = "ShardReplication"
	TabletFile             = "Tablet"
	SrvVSchemaFile         = "SrvVSchema"
	SrvKeyspaceFile        = "SrvKeyspace"
	RoutingRulesFile       = "RoutingRules"
	ExternalClustersFile   = "ExternalClusters"
	ShardRoutingRulesFile  = "ShardRoutingRules"
	CommonRoutingRulesFile = "Rules"
	MirrorRulesFile        = "MirrorRules"
)

Filenames for all object types.

View Source 
const (
	CellsPath                = "cells"
	CellsAliasesPath         = "cells_aliases"
	KeyspacesPath            = "keyspaces"
	ShardsPath               = "shards"
	TabletsPath              = "tablets"
	MetadataPath             = "metadata"
	ExternalClusterVitess    = "vitess"
	RoutingRulesPath         = "routing_rules"
	KeyspaceRoutingRulesPath = "keyspace"
	NamedLocksPath           = "internal/named_locks"
)

Path for all object types.

View Source 
const NamedLockTTL = 24 * time.Hour

How long named locks are kept in the topo server. This ensures that orphaned named locks are not kept around forever. This should never happen, but it provides a final safety net.

Variables

View Source 
var (
	// LockTimeout is the maximum duration for which a
	// shard / keyspace lock can be acquired for.
	LockTimeout = 45 * time.Second

	// RemoteOperationTimeout is used for operations where we have to
	// call out to another process.
	// Used for RPC calls (including topo server calls)
	RemoteOperationTimeout = 15 * time.Second
)
View Source 
var DefaultConcurrency = 32

Default concurrency to use in order to avoid overhwelming the topo server.

View Source 
var (
	FlagBinaries = []string{"vttablet", "vtctl", "vtctld", "vtcombo", "vtgate",
		"vtorc", "vtbackup"}
)

Functions

func CheckKeyspaceLocked 

func CheckKeyspaceLocked(ctx context.Context, keyspace string) error

CheckKeyspaceLocked can be called on a context to make sure we have the lock for a given keyspace.

func CheckNameLocked added in v0.21.0 

func CheckNameLocked(ctx context.Context, name string) error

CheckNameLocked can be called on a context to make sure we have the lock for a given opaque identifier.

func CheckRoutingRulesLocked added in v0.21.0 

func CheckRoutingRulesLocked(ctx context.Context) error

CheckRoutingRulesLocked checks if a lock for routing rules is still possessed.

func CheckShardLocked 

func CheckShardLocked(ctx context.Context, keyspace, shard string) error

CheckShardLocked can be called on a context to make sure we have the lock for a given shard.

func DecodeContent added in v0.15.0 

func DecodeContent(filename string, data []byte, json bool) (string, error)

DecodeContent uses the filename to imply a type, and proto-decodes the right object, then echoes it as a string.

func DeleteTabletReplicationData 

func DeleteTabletReplicationData(ctx context.Context, ts *Server, tablet *topodatapb.Tablet) error

DeleteTabletReplicationData deletes replication data.

func DirEntriesSortByName 

func DirEntriesSortByName(entries []DirEntry)

DirEntriesSortByName sorts a slice of DirEntry objects by Name.

func DirEntriesToStringArray 

func DirEntriesToStringArray(entries []DirEntry) []string

DirEntriesToStringArray is a helper method to extract the names from an []DirEntry

func FixShardReplication 

func FixShardReplication(ctx context.Context, ts *Server, logger logutil.Logger, cell, keyspace, shard string) (*topodatapb.ShardReplicationError, error)

FixShardReplication will fix the first problem it encounters within a ShardReplication object. It returns info about the error being fixed, if an error was found.

A return value of (nil, nil) indicates no issues in the replication graph.

func GetAliasByCell 

func GetAliasByCell(ctx context.Context, ts *Server, cell string) string

GetAliasByCell returns the alias group this `cell` belongs to, if there's none, it returns the `cell` as alias.

func GetExternalVitessClusterDir added in v0.10.0 

func GetExternalVitessClusterDir() string

GetExternalVitessClusterDir returns node path containing external vitess clusters

func GetExternalVitessClusterPath added in v0.10.0 

func GetExternalVitessClusterPath(clusterName string) string

GetExternalVitessClusterPath returns node path containing external clusters

func InCellList 

func InCellList(cell string, cells []string) bool

InCellList returns true if the cell list is empty, or if the passed cell is in the cell list.

func IsErrType 

func IsErrType(err error, code ErrorCode) bool

IsErrType returns true if the error has the specified ErrorCode.

func IsInServingGraph 

func IsInServingGraph(tt topodatapb.TabletType) bool

IsInServingGraph returns if a tablet appears in the serving graph

func IsReplicaType 

func IsReplicaType(tt topodatapb.TabletType) bool

IsReplicaType returns if this type should be connected to a primary db and actively replicating? PRIMARY is not obviously (only support one level replication graph) BACKUP, RESTORE, DRAINED may or may not be, but we don't know for sure

func IsRunningQueryService 

func IsRunningQueryService(tt topodatapb.TabletType) bool

IsRunningQueryService returns if a tablet is running the query service

func IsRunningUpdateStream 

func IsRunningUpdateStream(tt topodatapb.TabletType) bool

IsRunningUpdateStream returns if a tablet is running the update stream RPC service.

func IsShardUsingRangeBasedSharding 

func IsShardUsingRangeBasedSharding(shard string) bool

IsShardUsingRangeBasedSharding returns true if the shard name implies it is using range based sharding.

func IsTrivialTypeChange 

func IsTrivialTypeChange(oldTabletType, newTabletType topodatapb.TabletType) bool

IsTrivialTypeChange returns if this db type be trivially reassigned without changes to the replication graph

func NewError 

func NewError(code ErrorCode, node string) error

NewError creates a new topo error.

func NewTablet 

func NewTablet(uid uint32, cell, host string) *topodatapb.Tablet

NewTablet create a new Tablet record with the given id, cell, and hostname.

func OrderAndCheckPartitions 

func OrderAndCheckPartitions(cell string, srvKeyspace *topodatapb.SrvKeyspace) error

OrderAndCheckPartitions will re-order the partition list, and check it's correct.

func ParseServingTabletType added in v0.10.0 

func ParseServingTabletType(param string) (topodatapb.TabletType, error)

ParseServingTabletType parses the tablet type into the enum, and makes sure that the enum is of serving type (PRIMARY, REPLICA, RDONLY/BATCH).

Note: This function more closely belongs in topoproto, but that would create a circular import between packages topo and topoproto.

func RegisterFactory 

func RegisterFactory(name string, factory Factory)

RegisterFactory registers a Factory for an implementation for a Server. If an implementation with that name already exists, it log.Fatals out. Call this in the 'init' function in your topology implementation module.

func RemoveShardReplicationRecord 

func RemoveShardReplicationRecord(ctx context.Context, ts *Server, cell, keyspace, shard string, tabletAlias *topodatapb.TabletAlias) error

RemoveShardReplicationRecord is a low level function to remove an entry from the ShardReplication object.

func UpdateShardReplicationRecord 

func UpdateShardReplicationRecord(ctx context.Context, ts *Server, keyspace, shard string, tabletAlias *topodatapb.TabletAlias) error

UpdateShardReplicationRecord is a low level function to add / update an entry to the ShardReplication object.

func UpdateTabletReplicationData 

func UpdateTabletReplicationData(ctx context.Context, ts *Server, tablet *topodatapb.Tablet) error

UpdateTabletReplicationData creates or updates the replication graph data for a tablet

func Validate 

func Validate(ctx context.Context, ts *Server, tabletAlias *topodatapb.TabletAlias) error

Validate makes sure a tablet is represented correctly in the topology server.

func ValidateKeyspaceName added in v0.16.1 

func ValidateKeyspaceName(name string) error

ValidateKeyspaceName checks if the provided name is a valid name for a keyspace.

func ValidateShardName 

func ValidateShardName(shard string) (string, *topodatapb.KeyRange, error)

ValidateShardName takes a shard name and sanitizes it, and also returns the KeyRange.

Types

type CancelFunc 

type CancelFunc func()

CancelFunc is returned by the Watch method.

type Conn 

type Conn interface {

	// ListDir returns the entries in a directory.  The returned
	// list should be sorted by entry.Name.
	// If there are no files under the provided path, returns ErrNoNode.
	// dirPath is a path relative to the root directory of the cell.
	// If full is set, we want all the fields in DirEntry to be filled in.
	// If full is not set, only Name will be used. This is intended for
	// implementations where getting more than the names is more expensive,
	// as in most cases only the names are needed.
	ListDir(ctx context.Context, dirPath string, full bool) ([]DirEntry, error)

	// Create creates the initial version of a file.
	// Returns ErrNodeExists if the file exists.
	// filePath is a path relative to the root directory of the cell.
	Create(ctx context.Context, filePath string, contents []byte) (Version, error)

	// Update updates the file with the provided filename with the
	// new content.
	// If version is nil, it is an unconditional update
	// (which is then the same as a Create is the file doesn't exist).
	// filePath is a path relative to the root directory of the cell.
	// It returns the new Version of the file after update.
	// Returns ErrBadVersion if the provided version is not current.
	Update(ctx context.Context, filePath string, contents []byte, version Version) (Version, error)

	// Get returns the content and version of a file.
	// filePath is a path relative to the root directory of the cell.
	// Can return ErrNoNode if the file doesn't exist.
	Get(ctx context.Context, filePath string) ([]byte, Version, error)

	// GetVersion returns the content of a file at the given version.
	// filePath is a path relative to the root directory of the cell.
	// Can return ErrNoNode if the file doesn't exist at the given
	// version or ErrNoImplementation if the topo server does not
	// support storing multiple versions and retrieving a specific one.
	GetVersion(ctx context.Context, filePath string, version int64) ([]byte, error)

	// List returns KV pairs, along with metadata like the version, for
	// entries where the key contains the specified prefix.
	// filePathPrefix is a path relative to the root directory of the cell.
	// Can return ErrNoNode if there are no matches.
	List(ctx context.Context, filePathPrefix string) ([]KVInfo, error)

	// Delete deletes the provided file.
	// If version is nil, it is an unconditional delete.
	// If the last entry of a directory is deleted, using ListDir
	// on its parent directory should not return the directory.
	// For instance, when deleting /keyspaces/aaa/Keyspace, and if
	// there is no other file in /keyspaces/aaa, then aaa should not
	// appear any more when listing /keyspaces.
	// filePath is a path relative to the root directory of the cell.
	//
	// Delete will never be called on a directory.
	// Returns ErrNodeExists if the file doesn't exist.
	// Returns ErrBadVersion if the provided version is not current.
	Delete(ctx context.Context, filePath string, version Version) error

	// Lock takes a lock on the given directory.
	// It does not prevent any modification to any file in the topology.
	// It just prevents two concurrent processes (wherever they are)
	// to run concurrently. It is used for instance to make sure only
	// one reparent operation is running on a Shard at a given time.
	// dirPath is the directory associated with a resource, for instance
	// a Keyspace or a Shard. It is not a file location.
	// (this means the implementation can for instance create a
	// file in this directory to materialize the lock).
	// contents describes the lock holder and purpose, but has no other
	// meaning, so it can be used as a lock file contents, for instance.
	// Returns ErrNoNode if the directory doesn't exist (meaning
	//   there is no existing file under that directory).
	// Returns ErrTimeout if ctx expires.
	// Returns ErrInterrupted if ctx is canceled.
	Lock(ctx context.Context, dirPath, contents string) (LockDescriptor, error)

	// LockWithTTL is similar to `Lock` but the difference is that it allows
	// you to override the global default TTL that is configured for the
	// implementation (--topo_etcd_lease_ttl and --topo_consul_lock_session_ttl).
	// Note: this is no different than `Lock` for ZooKeeper as it does not
	// support lock TTLs and they exist until released or the session ends.
	LockWithTTL(ctx context.Context, dirPath, contents string, ttl time.Duration) (LockDescriptor, error)

	// LockName is similar to `Lock` but the difference is that it does not require
	// the path to exist and have children in order to lock it. This is because with
	// named locks you are NOT locking an actual topo entity such as a Keyspace record.
	// Because this lock is not blocking any Vitess operations OTHER than another
	// caller that is trying to get the same named lock, there is a static 24 hour
	// TTL on them to ensure that they are eventually cleaned up.
	LockName(ctx context.Context, dirPath, contents string) (LockDescriptor, error)

	// TryLock takes lock on the given directory with a fail-fast approach.
	// It is similar to `Lock` but the difference is it attempts to acquire the lock
	// if it is likely to succeed. If there is already a lock on given path, then unlike `Lock`
	// instead of waiting and blocking the client it returns with `Lock already exists` error.
	// With current implementation it may not be able to fail-fast for some scenarios.
	// For example there is a possibility that a thread checks for lock for a given path
	// but by the time it acquires the lock, some other thread has already acquired it,
	// in this case the client will block until the other caller releases the lock or the
	// client call times out (just like standard `Lock' implementation). In short the lock checking
	// and acquiring is not under the same mutex in current implementation of `TryLock`.
	TryLock(ctx context.Context, dirPath, contents string) (LockDescriptor, error)

	// Watch starts watching a file in the provided cell.  It
	// returns the current value, a 'changes' channel to read the
	// changes from, and an error.
	// If the initial read fails, or the file doesn't
	// exist, an error is returned.
	//
	// To stop the watch, cancel the provided context.
	// This will eventually result in a final WatchData result with Err =
	// ErrInterrupted. It should be safe to cancel the context
	// multiple times, or after the Watch already errored out.
	//
	// The 'changes' channel may return a record with Err != nil.
	// In that case, the channel will also be closed right after
	// that record.  In any case, 'changes' has to be drained of
	// all events, even when 'stop' is closed.
	//
	// Note the 'changes' channel can return twice the same
	// Version/Contents (for instance, if the watch is interrupted
	// and restarted within the Conn implementation).
	// Similarly, the 'changes' channel may skip versions / changes
	// (that is, if value goes [A, B, C, D, E, F], the watch may only
	// receive [A, B, F]). This should only happen for rapidly
	// changing values though. Usually, the initial value will come
	// back right away. And a stable value (that hasn't changed for
	// a while) should be seen shortly.
	//
	// The Watch call is not guaranteed to return exactly up to
	// date data right away. For instance, if a file is created
	// and saved, and then a watch is set on that file, it may
	// return ErrNoNode (as the underlying configuration service
	// may use asynchronous caches that are not up to date
	// yet). The only guarantee is that the watch data will
	// eventually converge. Vitess doesn't explicitly depend on the data
	// being correct quickly, as long as it eventually gets there.
	//
	// filePath is a path relative to the root directory of the cell.
	Watch(ctx context.Context, filePath string) (current *WatchData, changes <-chan *WatchData, err error)

	// WatchRecursive starts watching a file prefix in the provided cell. It
	// returns all the current values for existing files with the given
	// prefix, a 'changes' channel  to read the changes from and an error.
	//
	// The provided context should be canceled when stopping WatchRecursive().
	// This API is different from Watch() and Watch() will be changed
	// to match this API as well in the future.
	//
	// Canceling will eventually result in a final WatchDataRecursive result with Err =
	// ErrInterrupted.
	//
	// The 'changes' channel may return a record with Err != nil.
	// In that case, the channel will also be closed right after
	// that record.  In any case, 'changes' has to be drained of
	// all events, even when 'stop' is closed.
	//
	// Note the 'changes' channel can return twice the same
	// Version/Contents (for instance, if the watch is interrupted
	// and restarted within the Conn implementation).
	// Similarly, the 'changes' channel may skip versions / changes
	// (that is, if value goes [A, B, C, D, E, F], the watch may only
	// receive [A, B, F]). This should only happen for rapidly
	// changing values though. Usually, the initial value will come
	// back right away. And a stable value (that hasn't changed for
	// a while) should be seen shortly.
	//
	// The WatchRecursive call is not guaranteed to return exactly up to
	// date data right away. For instance, if a file is created
	// and saved, and then a watch is set on that file, it may
	// return ErrNoNode (as the underlying configuration service
	// may use asynchronous caches that are not up to date
	// yet). The only guarantee is that the watch data will
	// eventually converge. Vitess doesn't explicitly depend on the data
	// being correct quickly, as long as it eventually gets there.
	//
	// path is a path relative to the root directory of the cell.
	WatchRecursive(ctx context.Context, path string) ([]*WatchDataRecursive, <-chan *WatchDataRecursive, error)

	// NewLeaderParticipation creates a LeaderParticipation
	// object, used to become the Leader in an election for the
	// provided group name.  Id is the name of the local process,
	// passing in the hostname:port of the current process as id
	// is the common usage. Id must be unique for each process
	// calling this, for a given name. Calling this function does
	// not make the current process a candidate for the election.
	NewLeaderParticipation(name, id string) (LeaderParticipation, error)

	// Close closes the connection to the server.
	Close()
}

Conn defines the interface that must be implemented by topology plug-ins to be used with Vitess.

Zookeeper is a good example of an implementation, as defined in go/vt/topo/zk2topo.

This API is very generic, and key/value store oriented. We use regular paths for object names, and we can list all immediate children of a path. All paths sent through this API are relative paths, from the root directory of the cell.

The Conn objects are created by the Factory implementations.

type DirEntry 

type DirEntry struct {
	// Name is the name of the entry.
	// Always filled in.
	Name string

	// Type is the DirEntryType of the entry.
	// Only filled in if full is true.
	Type DirEntryType

	// Ephemeral is set if the directory / file only contains
	// data that was not set by the file API, like lock files
	// or primary-election related files.
	// Only filled in if full is true.
	Ephemeral bool
}

DirEntry is an entry in a directory, as returned by ListDir.

type DirEntryType 

type DirEntryType int

DirEntryType is the type of an entry in a directory. 

const (
	// TypeDirectory describes a directory.
	TypeDirectory DirEntryType = iota

	// TypeFile describes a file.
	TypeFile
)

type Error 

type Error struct {
	// contains filtered or unexported fields
}

Error represents a topo error.

func (Error) Error 

func (e Error) Error() string

Error satisfies error.

type ErrorCode 

type ErrorCode int

ErrorCode is the error code for topo errors.

type ExternalVitessClusterInfo added in v0.10.0 

type ExternalVitessClusterInfo struct {
	ClusterName string

	*topodatapb.ExternalVitessCluster
	// contains filtered or unexported fields
}

ExternalVitessClusterInfo is a meta struct that contains metadata to give the data more context and convenience. This is the main way we interact with a vitess cluster stored in the topo.

type Factory 

type Factory interface {
	// HasGlobalReadOnlyCell returns true if the global cell
	// has read-only replicas of the topology data. The global topology
	// is usually more expensive to read from / write to, as it is
	// replicated over many cells. Some topology services provide
	// more efficient way to read the data, like Observer servers
	// for Zookeeper. If this returns true, we will maintain
	// two connections for the global topology: the 'global' cell
	// for consistent reads and writes, and the 'global-read-only'
	// cell for reads only.
	HasGlobalReadOnlyCell(serverAddr, root string) bool

	// Create creates a topo.Conn object.
	Create(cell, serverAddr, root string) (Conn, error)
}

Factory is a factory interface to create Conn objects. Topo implementations will provide an implementation for this.

type FindAllShardsInKeyspaceOptions added in v0.19.0 

type FindAllShardsInKeyspaceOptions struct {
	// Concurrency controls the maximum number of concurrent calls to GetShard.
	// If <= 0, Concurrency is set to 1.
	Concurrency int
}

FindAllShardsInKeyspaceOptions controls the behavior of Server.FindAllShardsInKeyspace.

type GetTabletsByCellOptions added in v0.19.0 

type GetTabletsByCellOptions struct {
	// Concurrency controls the maximum number of concurrent calls to GetTablet.
	Concurrency int
}

GetTabletsByCellOptions controls the behavior of Server.FindAllShardsInKeyspace.

type KVInfo added in v0.13.0 

type KVInfo struct {
	Key     []byte
	Value   []byte
	Version Version // version - used to prevent stomping concurrent writes
}

KVInfo is a structure that contains a generic key/value pair from the topo server, along with important metadata about it. This should be used to provide multiple entries in List like calls that return N KVs based on a key prefix, so that you don't lose information or context you would otherwise have when using Get for a single key.

type KeyspaceInfo 

type KeyspaceInfo struct {
	*topodatapb.Keyspace
	// contains filtered or unexported fields
}

KeyspaceInfo is a meta struct that contains metadata to give the data more context and convenience. This is the main way we interact with a keyspace.

func (*KeyspaceInfo) KeyspaceName 

func (ki *KeyspaceInfo) KeyspaceName() string

KeyspaceName returns the keyspace name

func (*KeyspaceInfo) SetKeyspaceName added in v0.14.0 

func (ki *KeyspaceInfo) SetKeyspaceName(name string)

SetKeyspaceName sets the keyspace name

type KeyspaceShard 

type KeyspaceShard struct {
	Keyspace string
	Shard    string
}

KeyspaceShard is a type used by ResolveShardWildcard

type LeaderParticipation added in v0.13.0 

type LeaderParticipation interface {
	// WaitForLeadership makes the current process a candidate
	// for election, and waits until this process is the primary.
	// After we become the primary, we may lose primaryship. In that case,
	// the returned context will be canceled. If Stop was called,
	// WaitForLeadership will return nil, ErrInterrupted.
	WaitForLeadership() (context.Context, error)

	// Stop is called when we don't want to participate in the
	// primary election any more. Typically, that is when the
	// hosting process is terminating.  We will relinquish
	// primaryship at that point, if we had it. Stop should
	// not return until everything has been done.
	// The LeaderParticipation object should be discarded
	// after Stop has been called. Any call to WaitForLeadership
	// after Stop() will return nil, ErrInterrupted.
	// If WaitForLeadership() was running, it will return
	// nil, ErrInterrupted as soon as possible.
	Stop()

	// GetCurrentLeaderID returns the current primary id.
	// This may not work after Stop has been called.
	GetCurrentLeaderID(ctx context.Context) (string, error)

	// WaitForNewLeader allows for nodes to wait until a leadership
	// election cycle completes and to get subsequent updates of
	// leadership changes. This way logic that needs to know if leadership
	// changes also if we're not the leader ourselves doesn't need to
	// poll for leadership status.
	//
	// For topo implementation that have this, it can be used more
	// efficiently than needing a busy wait loop.
	WaitForNewLeader(ctx context.Context) (<-chan string, error)
}

LeaderParticipation is the object returned by NewLeaderParticipation. Sample usage:

mp := server.NewLeaderParticipation("vtctld", "hostname:8080") job := NewJob() 

go func() {
  for {
    ctx, err := mp.WaitForLeadership()
    switch err {
    case nil:
      job.RunUntilContextDone(ctx)
    case topo.ErrInterrupted:
      return
    default:
      log.Errorf("Got error while waiting for primary, will retry in 5s: %v", err)
      time.Sleep(5 * time.Second)
    }
  }
}()

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
  if job.Running() {
    job.WriteStatus(w, r)
  } else {
    http.Redirect(w, r, mp.GetCurrentLeaderID(context.Background()), http.StatusFound)
  }
})

servenv.OnTermSync(func() {
  mp.Stop()
})

type Lock 

type Lock struct {
	// Action and the following fields are set at construction time.
	Action   string
	HostName string
	UserName string
	Time     string
	Options  lockOptions

	// Status is the current status of the Lock.
	Status string
}

Lock describes a long-running lock on a keyspace or a shard. It needs to be public as we JSON-serialize it.

func (*Lock) ToJSON 

func (l *Lock) ToJSON() (string, error)

ToJSON returns a JSON representation of the object.

type LockDescriptor 

type LockDescriptor interface {
	// Check returns an error if the lock was lost.
	// Some topology implementations use a keep-alive mechanism, and
	// sometimes it fails. The users of the lock are responsible for
	// checking on it when convenient.
	Check(ctx context.Context) error

	// Unlock releases the lock.
	Unlock(ctx context.Context) error
}

LockDescriptor is an interface that describes a lock. It will be returned by Lock().

type LockOption added in v0.21.0 

type LockOption interface {
	// contains filtered or unexported methods
}

LockOption configures how we perform the locking operation.

func WithTTL added in v0.21.0 

func WithTTL(ttl time.Duration) LockOption

WithTTL allows you to specify how long the underlying topo server implementation should hold the lock before releasing it — even if the caller has not explicitly released it. This provides a way to override the global ttl values that are set via --topo_consul_lock_session_ttl and --topo_etcd_lease_ttl. Note: This option is ignored by the ZooKeeper implementation as it does not support TTLs.

func WithType added in v0.21.0 

func WithType(lt LockType) LockOption

WithType determines the type of lock we take. The options are defined by the LockType type.

type LockType added in v0.21.0 

type LockType int

Support different lock types. 

const (
	// Blocking is the default lock type when no other valid type
	// is specified.
	Blocking    LockType = iota
	NonBlocking          // Uses TryLock
	Named                // Uses LockName
)

func (LockType) String added in v0.21.0 

func (lt LockType) String() string

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server is the main topo.Server object. We support two ways of creating one:

  1. From an implementation, server address, and root path. This uses a plugin mechanism, and we have implementations for etcd, zookeeper and consul.
  2. Specific implementations may have higher level creation methods (in which case they may provide a more complex Factory). We support memorytopo (for tests and processes that only need an in-memory server), and tee (a helper implementation to transition between one server implementation and another).

func NewWithFactory 

func NewWithFactory(factory Factory, serverAddress, root string) (*Server, error)

NewWithFactory creates a new Server based on the given Factory. It also opens the global cell connection.

func Open 

func Open() *Server

Open returns a Server using the command line parameter flags for implementation, address and root. It log.Exits out if an error occurs.

func OpenServer 

func OpenServer(implementation, serverAddress, root string) (*Server, error)

OpenServer returns a Server using the provided implementation, address and root for the global server.

func (*Server) AddSrvKeyspacePartitions 

func (ts *Server) AddSrvKeyspacePartitions(ctx context.Context, keyspace string, shards []*ShardInfo, tabletType topodatapb.TabletType, cells []string) (err error)

AddSrvKeyspacePartitions adds partitions to srvKeyspace

func (*Server) Close 

func (ts *Server) Close()

Close will close all connections to underlying topo Server. It will nil all member variables, so any further access will panic.

func (*Server) ConnForCell 

func (ts *Server) ConnForCell(ctx context.Context, cell string) (Conn, error)

ConnForCell returns a Conn object for the given cell. It caches Conn objects from previously requested cells.

func (*Server) CreateCellInfo 

func (ts *Server) CreateCellInfo(ctx context.Context, cell string, ci *topodatapb.CellInfo) error

CreateCellInfo creates a new CellInfo with the provided content.

func (*Server) CreateCellsAlias 

func (ts *Server) CreateCellsAlias(ctx context.Context, alias string, cellsAlias *topodatapb.CellsAlias) error

CreateCellsAlias creates a new CellInfo with the provided content.

func (*Server) CreateExternalVitessCluster added in v0.10.0 

func (ts *Server) CreateExternalVitessCluster(ctx context.Context, clusterName string, value *topodatapb.ExternalVitessCluster) error

CreateExternalVitessCluster creates a topo record for the passed vitess cluster

func (*Server) CreateKeyspace 

func (ts *Server) CreateKeyspace(ctx context.Context, keyspace string, value *topodatapb.Keyspace) error

CreateKeyspace wraps the underlying Conn.Create and dispatches the event.

func (*Server) CreateKeyspaceRoutingRules added in v0.20.0 

func (ts *Server) CreateKeyspaceRoutingRules(ctx context.Context, value *vschemapb.KeyspaceRoutingRules) error

CreateKeyspaceRoutingRules wraps the underlying Conn.Create.

func (*Server) CreateShard 

func (ts *Server) CreateShard(ctx context.Context, keyspace, shard string) (err error)

CreateShard creates a new shard and tries to fill in the right information. This will lock the Keyspace, as we may be looking at other shard servedTypes. Using GetOrCreateShard is probably a better idea for most use cases.

func (*Server) CreateTablet 

func (ts *Server) CreateTablet(ctx context.Context, tablet *topodatapb.Tablet) error

CreateTablet creates a new tablet and all associated paths for the replication graph.

func (*Server) CreateWorkflow 

func (ts *Server) CreateWorkflow(ctx context.Context, w *workflowpb.Workflow) (*WorkflowInfo, error)

CreateWorkflow creates the given workflow, and returns the initial WorkflowInfo.

func (*Server) DeleteCellInfo 

func (ts *Server) DeleteCellInfo(ctx context.Context, cell string, force bool) error

DeleteCellInfo deletes the specified CellInfo. We first try to make sure no Shard record points to the cell, but we'll continue regardless if 'force' is true.

func (*Server) DeleteCellsAlias 

func (ts *Server) DeleteCellsAlias(ctx context.Context, alias string) error

DeleteCellsAlias deletes the specified CellsAlias

func (*Server) DeleteExternalVitessCluster added in v0.10.0 

func (ts *Server) DeleteExternalVitessCluster(ctx context.Context, clusterName string) error

DeleteExternalVitessCluster deletes the topo record for the named vitess cluster

func (*Server) DeleteKeyspace 

func (ts *Server) DeleteKeyspace(ctx context.Context, keyspace string) error

DeleteKeyspace wraps the underlying Conn.Delete and dispatches the event.

func (*Server) DeleteKeyspaceReplication 

func (ts *Server) DeleteKeyspaceReplication(ctx context.Context, cell, keyspace string) error

DeleteKeyspaceReplication deletes all the ShardReplication objects for a cell/keyspace.

func (*Server) DeleteMetadata 

func (ts *Server) DeleteMetadata(ctx context.Context, key string) error

DeleteMetadata deletes the key in the metadata

func (*Server) DeleteShard 

func (ts *Server) DeleteShard(ctx context.Context, keyspace, shard string) error

DeleteShard wraps the underlying conn.Delete and dispatches the event.

func (*Server) DeleteShardReplication 

func (ts *Server) DeleteShardReplication(ctx context.Context, cell, keyspace, shard string) error

DeleteShardReplication deletes a ShardReplication object.

func (*Server) DeleteSrvKeyspace 

func (ts *Server) DeleteSrvKeyspace(ctx context.Context, cell, keyspace string) error

DeleteSrvKeyspace deletes a SrvKeyspace.

func (*Server) DeleteSrvKeyspacePartitions 

func (ts *Server) DeleteSrvKeyspacePartitions(ctx context.Context, keyspace string, shards []*ShardInfo, tabletType topodatapb.TabletType, cells []string) (err error)

DeleteSrvKeyspacePartitions deletes shards from srvKeyspace partitions

func (*Server) DeleteSrvVSchema 

func (ts *Server) DeleteSrvVSchema(ctx context.Context, cell string) error

DeleteSrvVSchema deletes the SrvVSchema file for a cell.

func (*Server) DeleteTablet 

func (ts *Server) DeleteTablet(ctx context.Context, tabletAlias *topodatapb.TabletAlias) error

DeleteTablet wraps the underlying conn.Delete and dispatches the event.

func (*Server) DeleteVSchema 

func (ts *Server) DeleteVSchema(ctx context.Context, keyspace string) error

DeleteVSchema delete the keyspace if it exists

func (*Server) DeleteWorkflow 

func (ts *Server) DeleteWorkflow(ctx context.Context, wi *WorkflowInfo) error

DeleteWorkflow deletes the specified workflow. After this, the WorkflowInfo object should not be used any more.

func (*Server) EnsureVSchema 

func (ts *Server) EnsureVSchema(ctx context.Context, keyspace string) error

EnsureVSchema makes sure that a vschema is present for this keyspace or creates a blank one if it is missing

func (*Server) ExpandCells added in v0.10.0 

func (ts *Server) ExpandCells(ctx context.Context, cells string) ([]string, error)

ExpandCells takes a comma-separated list of cells and returns an array of cell names Aliases are expanded and an empty string returns all cells

func (*Server) FindAllShardsInKeyspace 

func (ts *Server) FindAllShardsInKeyspace(ctx context.Context, keyspace string, opt *FindAllShardsInKeyspaceOptions) (map[string]*ShardInfo, error)

FindAllShardsInKeyspace reads and returns all the existing shards in a keyspace. It doesn't take any lock.

If opt is non-nil, it is used to configure the method's behavior. Otherwise, the default options are used.

func (*Server) FindAllTabletAliasesInShard 

func (ts *Server) FindAllTabletAliasesInShard(ctx context.Context, keyspace, shard string) ([]*topodatapb.TabletAlias, error)

FindAllTabletAliasesInShard uses the replication graph to find all the tablet aliases in the given shard.

It can return ErrPartialResult if some cells were not fetched, in which case the result only contains the cells that were fetched.

The tablet aliases are sorted by cell, then by UID.

func (*Server) FindAllTabletAliasesInShardByCell 

func (ts *Server) FindAllTabletAliasesInShardByCell(ctx context.Context, keyspace, shard string, cells []string) ([]*topodatapb.TabletAlias, error)

FindAllTabletAliasesInShardByCell uses the replication graph to find all the tablet aliases in the given shard.

It can return ErrPartialResult if some cells were not fetched, in which case the result only contains the cells that were fetched.

The tablet aliases are sorted by cell, then by UID.

func (*Server) GetCellInfo 

func (ts *Server) GetCellInfo(ctx context.Context, cell string, strongRead bool) (*topodatapb.CellInfo, error)

GetCellInfo reads a CellInfo from the global Conn.

func (*Server) GetCellInfoNames 

func (ts *Server) GetCellInfoNames(ctx context.Context) ([]string, error)

GetCellInfoNames returns the names of the existing cells. They are sorted by name.

func (*Server) GetCellsAlias 

func (ts *Server) GetCellsAlias(ctx context.Context, name string, strongRead bool) (*topodatapb.CellsAlias, error)

GetCellsAlias returns the CellsAlias that matches the given name.

func (*Server) GetCellsAliases 

func (ts *Server) GetCellsAliases(ctx context.Context, strongRead bool) (ret map[string]*topodatapb.CellsAlias, err error)

GetCellsAliases returns the names of the existing cells. They are sorted by name.

func (*Server) GetExternalVitessCluster added in v0.10.0 

func (ts *Server) GetExternalVitessCluster(ctx context.Context, clusterName string) (*ExternalVitessClusterInfo, error)

GetExternalVitessCluster returns a topo record for the named vitess cluster

func (*Server) GetExternalVitessClusters added in v0.10.0 

func (ts *Server) GetExternalVitessClusters(ctx context.Context) ([]string, error)

GetExternalVitessClusters returns the list of external vitess clusters in the topology.

func (*Server) GetKeyspace 

func (ts *Server) GetKeyspace(ctx context.Context, keyspace string) (*KeyspaceInfo, error)

GetKeyspace reads the given keyspace and returns it

func (*Server) GetKeyspaceDurability added in v0.14.0 

func (ts *Server) GetKeyspaceDurability(ctx context.Context, keyspace string) (string, error)

GetKeyspaceDurability reads the given keyspace and returns its durabilty policy

func (*Server) GetKeyspaceRoutingRules added in v0.20.0 

func (ts *Server) GetKeyspaceRoutingRules(ctx context.Context) (*vschemapb.KeyspaceRoutingRules, error)

func (*Server) GetKeyspaceRoutingRulesPath added in v0.20.0 

func (ts *Server) GetKeyspaceRoutingRulesPath() string

GetKeyspaceRoutingRulesPath returns the path to the keyspace routing rules file in the topo.

func (*Server) GetKeyspaces 

func (ts *Server) GetKeyspaces(ctx context.Context) ([]string, error)

GetKeyspaces returns the list of keyspaces in the topology.

func (*Server) GetKnownCells 

func (ts *Server) GetKnownCells(ctx context.Context) ([]string, error)

GetKnownCells returns the list of known cells. For now, it just lists the 'cells' directory in the global topology server. TODO(alainjobart) once the cell map is migrated to this generic package, we can do better than this.

func (*Server) GetMetadata 

func (ts *Server) GetMetadata(ctx context.Context, keyFilter string) (map[string]string, error)

GetMetadata retrieves all metadata value that matches the given key regular expression. If empty all values are returned.

func (*Server) GetMirrorRules added in v0.21.0 

func (ts *Server) GetMirrorRules(ctx context.Context) (*vschemapb.MirrorRules, error)

GetMirrorRules fetches the mirror rules from the topo.

func (*Server) GetOnlyShard 

func (ts *Server) GetOnlyShard(ctx context.Context, keyspace string) (*ShardInfo, error)

GetOnlyShard returns the single ShardInfo of an unsharded keyspace.

func (*Server) GetOrCreateShard 

func (ts *Server) GetOrCreateShard(ctx context.Context, keyspace, shard string) (si *ShardInfo, err error)

GetOrCreateShard will return the shard object, or create one if it doesn't already exist. Note the shard creation is protected by a keyspace Lock.

func (*Server) GetRoutingRules 

func (ts *Server) GetRoutingRules(ctx context.Context) (*vschemapb.RoutingRules, error)

GetRoutingRules fetches the routing rules from the topo.

func (*Server) GetServingShards 

func (ts *Server) GetServingShards(ctx context.Context, keyspace string) ([]*ShardInfo, error)

GetServingShards returns all shards where the primary is serving.

func (*Server) GetShard 

func (ts *Server) GetShard(ctx context.Context, keyspace, shard string) (*ShardInfo, error)

GetShard is a high level function to read shard data. It generates trace spans.

func (*Server) GetShardNames 

func (ts *Server) GetShardNames(ctx context.Context, keyspace string) ([]string, error)

GetShardNames returns the list of shards in a keyspace.

func (*Server) GetShardReplication 

func (ts *Server) GetShardReplication(ctx context.Context, cell, keyspace, shard string) (*ShardReplicationInfo, error)

GetShardReplication returns the ShardReplicationInfo object.

func (*Server) GetShardRoutingRules added in v0.15.0 

func (ts *Server) GetShardRoutingRules(ctx context.Context) (*vschemapb.ShardRoutingRules, error)

GetShardRoutingRules fetches the shard routing rules from the topo.

func (*Server) GetShardServingCells 

func (ts *Server) GetShardServingCells(ctx context.Context, si *ShardInfo) (servingCells []string, err error)

GetShardServingCells returns cells where this shard is serving

func (*Server) GetShardServingTypes 

func (ts *Server) GetShardServingTypes(ctx context.Context, si *ShardInfo) (servingTypes []topodatapb.TabletType, err error)

GetShardServingTypes returns served types for given shard across all cells

func (*Server) GetSidecarDBName added in v0.17.0 

func (ts *Server) GetSidecarDBName(ctx context.Context, keyspace string) (string, error)

func (*Server) GetSrvKeyspace 

func (ts *Server) GetSrvKeyspace(ctx context.Context, cell, keyspace string) (*topodatapb.SrvKeyspace, error)

GetSrvKeyspace returns the SrvKeyspace for a cell/keyspace.

func (*Server) GetSrvKeyspaceAllCells 

func (ts *Server) GetSrvKeyspaceAllCells(ctx context.Context, keyspace string) ([]*topodatapb.SrvKeyspace, error)

GetSrvKeyspaceAllCells returns the SrvKeyspace for all cells

func (*Server) GetSrvKeyspaceNames 

func (ts *Server) GetSrvKeyspaceNames(ctx context.Context, cell string) ([]string, error)

GetSrvKeyspaceNames returns the SrvKeyspace objects for a cell.

func (*Server) GetSrvVSchema 

func (ts *Server) GetSrvVSchema(ctx context.Context, cell string) (*vschemapb.SrvVSchema, error)

GetSrvVSchema returns the SrvVSchema for a cell.

func (*Server) GetTablet 

func (ts *Server) GetTablet(ctx context.Context, alias *topodatapb.TabletAlias) (*TabletInfo, error)

GetTablet is a high level function to read tablet data. It generates trace spans.

func (*Server) GetTabletAliasesByCell added in v0.13.0 

func (ts *Server) GetTabletAliasesByCell(ctx context.Context, cell string) ([]*topodatapb.TabletAlias, error)

GetTabletAliasesByCell returns all the tablet aliases in a cell. It returns ErrNoNode if the cell doesn't exist. It returns (nil, nil) if the cell exists, but there are no tablets in it.

func (*Server) GetTabletMap 

func (ts *Server) GetTabletMap(ctx context.Context, tabletAliases []*topodatapb.TabletAlias, opt *GetTabletsByCellOptions) (map[string]*TabletInfo, error)

GetTabletMap tries to read all the tablets in the provided list, and returns them in a map. If error is ErrPartialResult, the results in the map are incomplete, meaning some tablets couldn't be read. The map is indexed by topoproto.TabletAliasString(tablet alias).

func (*Server) GetTabletMapForShard 

func (ts *Server) GetTabletMapForShard(ctx context.Context, keyspace, shard string) (map[string]*TabletInfo, error)

GetTabletMapForShard returns the tablets for a shard. It can return ErrPartialResult if it couldn't read all the cells, or all the individual tablets, in which case the map is valid, but partial. The map is indexed by topoproto.TabletAliasString(tablet alias).

func (*Server) GetTabletMapForShardByCell 

func (ts *Server) GetTabletMapForShardByCell(ctx context.Context, keyspace, shard string, cells []string) (map[string]*TabletInfo, error)

GetTabletMapForShardByCell returns the tablets for a shard. It can return ErrPartialResult if it couldn't read all the cells, or all the individual tablets, in which case the map is valid, but partial. The map is indexed by topoproto.TabletAliasString(tablet alias).

func (*Server) GetTabletsByCell 

func (ts *Server) GetTabletsByCell(ctx context.Context, cellAlias string, opt *GetTabletsByCellOptions) ([]*TabletInfo, error)

GetTabletsByCell returns all the tablets in the cell. It returns ErrNoNode if the cell doesn't exist. It returns ErrPartialResult if some tablets couldn't be read. The results in the slice are incomplete. It returns (nil, nil) if the cell exists, but there are no tablets in it.

func (*Server) GetTabletsIndividuallyByCell added in v0.13.0 

func (ts *Server) GetTabletsIndividuallyByCell(ctx context.Context, cell string, opt *GetTabletsByCellOptions) ([]*TabletInfo, error)

GetTabletsIndividuallyByCell returns a sorted list of tablets for topo servers that do not directly support the topoConn.List() functionality. It returns ErrNoNode if the cell doesn't exist. It returns ErrPartialResult if some tablets couldn't be read. The results in the slice are incomplete. It returns (nil, nil) if the cell exists, but there are no tablets in it.

func (*Server) GetThrottlerConfig added in v0.16.1 

func (ts *Server) GetThrottlerConfig(ctx context.Context, keyspace string) (*topodatapb.ThrottlerConfig, error)

func (*Server) GetVSchema 

func (ts *Server) GetVSchema(ctx context.Context, keyspace string) (*vschemapb.Keyspace, error)

GetVSchema fetches the vschema from the topo.

func (*Server) GetWorkflow 

func (ts *Server) GetWorkflow(ctx context.Context, uuid string) (*WorkflowInfo, error)

GetWorkflow reads a workflow from the global cell.

func (*Server) GetWorkflowNames 

func (ts *Server) GetWorkflowNames(ctx context.Context) ([]string, error)

GetWorkflowNames returns the names of the existing workflows. They are sorted by uuid.

func (*Server) InitTablet added in v0.13.0 

func (ts *Server) InitTablet(ctx context.Context, tablet *topodatapb.Tablet, allowPrimaryOverride, createShardAndKeyspace, allowUpdate bool) error

InitTablet creates or updates a tablet. If no parent is specified in the tablet, and the tablet has a replica type, we will find the appropriate parent. If createShardAndKeyspace is true and the parent keyspace or shard don't exist, they will be created. If allowUpdate is true, and a tablet with the same ID exists, just update it. If a tablet is created as primary, and there is already a different primary in the shard, allowPrimaryOverride must be set.

func (*Server) IsReadOnly added in v0.13.0 

func (ts *Server) IsReadOnly() (bool, error)

IsReadOnly is initially ONLY implemented by StatsConn and used in ReadOnlyServer

func (*Server) LockKeyspace 

func (ts *Server) LockKeyspace(ctx context.Context, keyspace, action string, opts ...LockOption) (context.Context, func(*error), error)

LockKeyspace will lock the keyspace, and return: - a context with a locksInfo structure for future reference. - an unlock method - an error if anything failed.

func (*Server) LockName added in v0.21.0 

func (ts *Server) LockName(ctx context.Context, name, action string) (context.Context, func(*error), error)

LockName will lock the opaque identifier, and return: - a context with a locksInfo structure for future reference. - an unlock method - an error if anything failed.

func (*Server) LockRoutingRules added in v0.21.0 

func (ts *Server) LockRoutingRules(ctx context.Context, action string, opts ...LockOption) (context.Context, func(*error), error)

LockRoutingRules acquires a lock for routing rules.

func (*Server) LockShard 

func (ts *Server) LockShard(ctx context.Context, keyspace, shard, action string, opts ...LockOption) (context.Context, func(*error), error)

LockShard will lock the shard, and return: - a context with a locksInfo structure for future reference. - an unlock method - an error if anything failed.

We are currently only using this method to lock actions that would impact each-other. Most changes of the Shard object are done by UpdateShardFields, which is not locking the shard object. The current list of actions that lock a shard are: * all Vitess-controlled re-parenting operations:

  • PlannedReparentShard
  • EmergencyReparentShard

* any vtorc recovery e.g

  • RecoverDeadPrimary
  • ElectNewPrimary
  • FixPrimary

* operations that we don't want to conflict with re-parenting:

  • DeleteTablet when it's the shard's current primary

func (*Server) MigrateServedType 

func (ts *Server) MigrateServedType(ctx context.Context, keyspace string, shardsToAdd, shardsToRemove []*ShardInfo, tabletType topodatapb.TabletType, cells []string) (err error)

MigrateServedType removes/adds shards from srvKeyspace when migrating a served type.

func (*Server) OpenExternalVitessClusterServer added in v0.10.0 

func (ts *Server) OpenExternalVitessClusterServer(ctx context.Context, clusterName string) (*Server, error)

OpenExternalVitessClusterServer returns the topo server of the external cluster

func (*Server) RebuildSrvVSchema 

func (ts *Server) RebuildSrvVSchema(ctx context.Context, cells []string) error

RebuildSrvVSchema rebuilds the SrvVSchema for the provided cell list (or all cells if cell list is empty).

func (*Server) ResolveKeyspaceWildcard 

func (ts *Server) ResolveKeyspaceWildcard(ctx context.Context, param string) ([]string, error)

ResolveKeyspaceWildcard will resolve keyspace wildcards.

  • If the param is not a wildcard, it will just be returned (if the keyspace doesn't exist, it is still returned).
  • If the param is a wildcard, it will get all keyspaces and returns the ones which match the wildcard (which may be an empty list).

func (*Server) ResolveShardWildcard 

func (ts *Server) ResolveShardWildcard(ctx context.Context, param string) ([]KeyspaceShard, error)

ResolveShardWildcard will resolve shard wildcards. Both keyspace and shard names can use wildcard. Errors talking to the topology server are returned. ErrNoNode is ignored if it's the result of resolving a wildcard. Examples:

  • */* returns all keyspace/shard pairs, or empty list if none.
  • user/* returns all shards in user keyspace (or error if user keyspace doesn't exist)
  • us*/* returns all shards in all keyspaces that start with 'us'. If no such keyspace exists, list is empty (it is not an error).

func (*Server) ResolveWildcards 

func (ts *Server) ResolveWildcards(ctx context.Context, cell string, paths []string) ([]string, error)

ResolveWildcards resolves paths like: /keyspaces/*/Keyspace into real existing paths

If you send paths that don't contain any wildcard and don't exist, this function will return an empty array.

func (*Server) SaveKeyspaceRoutingRules added in v0.20.0 

func (ts *Server) SaveKeyspaceRoutingRules(ctx context.Context, rules *vschemapb.KeyspaceRoutingRules) error

SaveKeyspaceRoutingRules saves the given routing rules proto in the topo at the defined path. It does NOT delete the file if you have requested to save empty routing rules (effectively deleting all routing rules in the file). This makes it different from the other routing rules (table and shard) save functions today. This is done as it simplifies the interactions with this key/file so that the typical access pattern is:

  • If the file exists, we can lock it, read it, modify it, and save it back.
  • If the file does not exist, we can create it and save the new rules.
  • If multiple callers are racing to create the file, only one will succeed and all other callers can simply retry once as the file will now exist.

We can revisit this in the future and align things as we add locking and other topo server features to the other types of routing rules. We may then apply this new model used for keyspace routing rules to the other routing rules, or we may come up with a better model and apply it to the keyspace routing rules as well.

func (*Server) SaveMirrorRules added in v0.21.0 

func (ts *Server) SaveMirrorRules(ctx context.Context, mirrorRules *vschemapb.MirrorRules) error

SaveMirrorRules saves the mirror rules into the topo.

func (*Server) SaveRoutingRules 

func (ts *Server) SaveRoutingRules(ctx context.Context, routingRules *vschemapb.RoutingRules) error

SaveRoutingRules saves the routing rules into the topo.

func (*Server) SaveShardRoutingRules added in v0.15.0 

func (ts *Server) SaveShardRoutingRules(ctx context.Context, shardRoutingRules *vschemapb.ShardRoutingRules) error

SaveShardRoutingRules saves the shard routing rules into the topo.

func (*Server) SaveVSchema 

func (ts *Server) SaveVSchema(ctx context.Context, keyspace string, vschema *vschemapb.Keyspace) error

SaveVSchema saves a Vschema. A valid Vschema should be passed in. It does not verify its correctness. If the VSchema is empty, just remove it.

func (*Server) SaveWorkflow 

func (ts *Server) SaveWorkflow(ctx context.Context, wi *WorkflowInfo) error

SaveWorkflow saves the WorkflowInfo object. If the version is not good any more, ErrBadVersion is returned.

func (*Server) SetReadOnly added in v0.13.0 

func (ts *Server) SetReadOnly(readOnly bool) error

SetReadOnly is initially ONLY implemented by StatsConn and used in ReadOnlyServer

func (*Server) TryLockShard added in v0.16.0 

func (ts *Server) TryLockShard(ctx context.Context, keyspace, shard, action string) (context.Context, func(*error), error)

TryLockShard will lock the shard, and return: - a context with a locksInfo structure for future reference. - an unlock method - an error if anything failed.

`TryLockShard` is different from `LockShard`. If there is already a lock on given shard, then unlike `LockShard` instead of waiting and blocking the client it returns with `Lock already exists` error. With current implementation it may not be able to fail-fast for some scenarios. For example there is a possibility that a thread checks for lock for a given shard but by the time it acquires the lock, some other thread has already acquired it, in this case the client will block until the other caller releases the lock or the client call times out (just like standard `LockShard' implementation). In short the lock checking and acquiring is not under the same mutex in current implementation of `TryLockShard`.

We are currently using `TryLockShard` during tablet discovery in Vtorc recovery

func (*Server) UpdateCellInfoFields 

func (ts *Server) UpdateCellInfoFields(ctx context.Context, cell string, update func(*topodatapb.CellInfo) error) error

UpdateCellInfoFields is a high level helper method to read a CellInfo object, update its fields, and then write it back. If the write fails due to a version mismatch, it will re-read the record and retry the update. If the update method returns ErrNoUpdateNeeded, nothing is written, and nil is returned.

func (*Server) UpdateCellsAlias 

func (ts *Server) UpdateCellsAlias(ctx context.Context, alias string, update func(*topodatapb.CellsAlias) error) error

UpdateCellsAlias updates cells for a given alias

func (*Server) UpdateDisableQueryService 

func (ts *Server) UpdateDisableQueryService(ctx context.Context, keyspace string, shards []*ShardInfo, tabletType topodatapb.TabletType, cells []string, disableQueryService bool) (err error)

UpdateDisableQueryService will make sure the disableQueryService is set appropriately in tablet controls in srvKeyspace.

func (*Server) UpdateExternalVitessCluster added in v0.10.0 

func (ts *Server) UpdateExternalVitessCluster(ctx context.Context, vc *ExternalVitessClusterInfo) error

UpdateExternalVitessCluster updates the topo record for the named vitess cluster

func (*Server) UpdateKeyspace 

func (ts *Server) UpdateKeyspace(ctx context.Context, ki *KeyspaceInfo) error

UpdateKeyspace updates the keyspace data. It checks the keyspace is locked.

func (*Server) UpdateShardFields 

func (ts *Server) UpdateShardFields(ctx context.Context, keyspace, shard string, update func(*ShardInfo) error) (*ShardInfo, error)

UpdateShardFields is a high level helper to read a shard record, call an update function on it, and then write it back. If the write fails due to a version mismatch, it will re-read the record and retry the update. If the update succeeds, it returns the updated ShardInfo. If the update method returns ErrNoUpdateNeeded, nothing is written, and nil,nil is returned.

Note the callback method takes a ShardInfo, so it can get the keyspace and shard from it, or use all the ShardInfo methods.

func (*Server) UpdateShardReplicationFields 

func (ts *Server) UpdateShardReplicationFields(ctx context.Context, cell, keyspace, shard string, update func(*topodatapb.ShardReplication) error) error

UpdateShardReplicationFields updates the fields inside a topo.ShardReplication object.

func (*Server) UpdateSrvKeyspace 

func (ts *Server) UpdateSrvKeyspace(ctx context.Context, cell, keyspace string, srvKeyspace *topodatapb.SrvKeyspace) error

UpdateSrvKeyspace saves a new SrvKeyspace. It is a blind write.

func (*Server) UpdateSrvKeyspaceThrottlerConfig added in v0.16.0 

func (ts *Server) UpdateSrvKeyspaceThrottlerConfig(ctx context.Context, keyspace string, cells []string, update func(throttlerConfig *topodatapb.ThrottlerConfig) *topodatapb.ThrottlerConfig) (updatedCells []string, err error)

UpdateSrvKeyspaceThrottlerConfig updates existing throttler configuration

func (*Server) UpdateSrvVSchema 

func (ts *Server) UpdateSrvVSchema(ctx context.Context, cell string, srvVSchema *vschemapb.SrvVSchema) error

UpdateSrvVSchema updates the SrvVSchema file for a cell.

func (*Server) UpdateTablet 

func (ts *Server) UpdateTablet(ctx context.Context, ti *TabletInfo) error

UpdateTablet updates the tablet data only - not associated replication paths. It also uses a span, and sends the event.

func (*Server) UpdateTabletFields 

func (ts *Server) UpdateTabletFields(ctx context.Context, alias *topodatapb.TabletAlias, update func(*topodatapb.Tablet) error) (*topodatapb.Tablet, error)

UpdateTabletFields is a high level helper to read a tablet record, call an update function on it, and then write it back. If the write fails due to a version mismatch, it will re-read the record and retry the update. If the update succeeds, it returns the updated tablet. If the update method returns ErrNoUpdateNeeded, nothing is written, and nil,nil is returned.

func (*Server) UpsertMetadata 

func (ts *Server) UpsertMetadata(ctx context.Context, key string, val string) error

UpsertMetadata sets the key/value in the metadata if it doesn't exist, otherwise it updates the content

func (*Server) ValidateSrvKeyspace added in v0.10.0 

func (ts *Server) ValidateSrvKeyspace(ctx context.Context, keyspace, cells string) error

ValidateSrvKeyspace validates that the SrvKeyspace for given keyspace in the provided cells is not corrupted

func (*Server) WatchShard 

func (ts *Server) WatchShard(ctx context.Context, keyspace, shard string) (*WatchShardData, <-chan *WatchShardData, error)

WatchShard will set a watch on the Shard object. It has the same contract as conn.Watch, but it also unpacks the contents into a Shard object

func (*Server) WatchSrvKeyspace 

func (ts *Server) WatchSrvKeyspace(ctx context.Context, cell, keyspace string) (*WatchSrvKeyspaceData, <-chan *WatchSrvKeyspaceData, error)

WatchSrvKeyspace will set a watch on the SrvKeyspace object. It has the same contract as Conn.Watch, but it also unpacks the contents into a SrvKeyspace object.

func (*Server) WatchSrvVSchema 

func (ts *Server) WatchSrvVSchema(ctx context.Context, cell string) (*WatchSrvVSchemaData, <-chan *WatchSrvVSchemaData, error)

WatchSrvVSchema will set a watch on the SrvVSchema object. It has the same contract as Conn.Watch, but it also unpacks the contents into a SrvVSchema object.

type ShardInfo 

type ShardInfo struct {
	*topodatapb.Shard
	// contains filtered or unexported fields
}

ShardInfo is a meta struct that contains metadata to give the data more context and convenience. This is the main way we interact with a shard.

func NewShardInfo 

func NewShardInfo(keyspace, shard string, value *topodatapb.Shard, version Version) *ShardInfo

NewShardInfo returns a ShardInfo basing on shard with the keyspace / shard. This function should be only used by Server implementations.

func (*ShardInfo) GetPrimaryTermStartTime added in v0.12.0 

func (si *ShardInfo) GetPrimaryTermStartTime() time.Time

GetPrimaryTermStartTime returns the shard's primary term start time as a Time value.

func (*ShardInfo) GetTabletControl 

func (si *ShardInfo) GetTabletControl(tabletType topodatapb.TabletType) *topodatapb.Shard_TabletControl

GetTabletControl returns the Shard_TabletControl for the given tablet type, or nil if it is not in the map.

func (*ShardInfo) HasPrimary added in v0.12.0 

func (si *ShardInfo) HasPrimary() bool

HasPrimary returns true if the Shard has an assigned primary.

func (*ShardInfo) Keyspace 

func (si *ShardInfo) Keyspace() string

Keyspace returns the keyspace a shard belongs to.

func (*ShardInfo) SetPrimaryTermStartTime added in v0.12.0 

func (si *ShardInfo) SetPrimaryTermStartTime(t time.Time)

SetPrimaryTermStartTime sets the shard's primary term start time as a Time value.

func (*ShardInfo) ShardName 

func (si *ShardInfo) ShardName() string

ShardName returns the shard name for a shard.

func (*ShardInfo) UpdateDeniedTables added in v0.18.0 

func (si *ShardInfo) UpdateDeniedTables(ctx context.Context, tabletType topodatapb.TabletType, cells []string, remove bool, tables []string) error

UpdateDeniedTables will add or remove the listed tables in the shard record's TabletControl structures. Note we don't support a lot of the corner cases:

  • only support one table list per shard. If we encounter a different table list that the provided one, we error out.
  • we don't support DisableQueryService at the same time as DeniedTables, because it's not used in the same context (vertical vs horizontal sharding)

This function should be called while holding the keyspace lock.

func (*ShardInfo) Version 

func (si *ShardInfo) Version() Version

Version returns the shard version from last time it was read or updated.

type ShardReplicationInfo 

type ShardReplicationInfo struct {
	*topodatapb.ShardReplication
	// contains filtered or unexported fields
}

ShardReplicationInfo is the companion structure for ShardReplication.

func NewShardReplicationInfo 

func NewShardReplicationInfo(sr *topodatapb.ShardReplication, cell, keyspace, shard string) *ShardReplicationInfo

NewShardReplicationInfo is for topo.Server implementations to create the structure

func (*ShardReplicationInfo) Cell 

func (sri *ShardReplicationInfo) Cell() string

Cell returns the cell for a ShardReplicationInfo

func (*ShardReplicationInfo) GetShardReplicationNode 

func (sri *ShardReplicationInfo) GetShardReplicationNode(tabletAlias *topodatapb.TabletAlias) (*topodatapb.ShardReplication_Node, error)

GetShardReplicationNode finds a node for a given tablet.

func (*ShardReplicationInfo) Keyspace 

func (sri *ShardReplicationInfo) Keyspace() string

Keyspace returns the keyspace for a ShardReplicationInfo

func (*ShardReplicationInfo) Shard 

func (sri *ShardReplicationInfo) Shard() string

Shard returns the shard for a ShardReplicationInfo

type StatsConn 

type StatsConn struct {
	// contains filtered or unexported fields
}

The StatsConn is a wrapper for a Conn that emits stats for every operation

func NewStatsConn 

func NewStatsConn(cell string, conn Conn) *StatsConn

NewStatsConn returns a StatsConn

func (*StatsConn) Close 

func (st *StatsConn) Close()

Close is part of the Conn interface

func (*StatsConn) Create 

func (st *StatsConn) Create(ctx context.Context, filePath string, contents []byte) (Version, error)

Create is part of the Conn interface

func (*StatsConn) Delete 

func (st *StatsConn) Delete(ctx context.Context, filePath string, version Version) error

Delete is part of the Conn interface

func (*StatsConn) Get 

func (st *StatsConn) Get(ctx context.Context, filePath string) ([]byte, Version, error)

Get is part of the Conn interface

func (*StatsConn) GetVersion added in v0.20.0 

func (st *StatsConn) GetVersion(ctx context.Context, filePath string, version int64) ([]byte, error)

GetVersion is part of the Conn interface.

func (*StatsConn) IsReadOnly added in v0.13.0 

func (st *StatsConn) IsReadOnly() bool

IsReadOnly allows you to check the access type for the topo connection

func (*StatsConn) List added in v0.13.0 

func (st *StatsConn) List(ctx context.Context, filePathPrefix string) ([]KVInfo, error)

List is part of the Conn interface

func (*StatsConn) ListDir 

func (st *StatsConn) ListDir(ctx context.Context, dirPath string, full bool) ([]DirEntry, error)

ListDir is part of the Conn interface

func (*StatsConn) Lock 

func (st *StatsConn) Lock(ctx context.Context, dirPath, contents string) (LockDescriptor, error)

Lock is part of the Conn interface

func (*StatsConn) LockName added in v0.21.0 

func (st *StatsConn) LockName(ctx context.Context, dirPath, contents string) (LockDescriptor, error)

LockName is part of the Conn interface

func (*StatsConn) LockWithTTL added in v0.21.0 

func (st *StatsConn) LockWithTTL(ctx context.Context, dirPath, contents string, ttl time.Duration) (LockDescriptor, error)

LockWithTTL is part of the Conn interface

func (*StatsConn) NewLeaderParticipation added in v0.13.0 

func (st *StatsConn) NewLeaderParticipation(name, id string) (LeaderParticipation, error)

NewLeaderParticipation is part of the Conn interface

func (*StatsConn) SetReadOnly added in v0.13.0 

func (st *StatsConn) SetReadOnly(readOnly bool)

SetReadOnly with true prevents any write operations from being made on the topo connection

func (*StatsConn) TryLock added in v0.16.0 

func (st *StatsConn) TryLock(ctx context.Context, dirPath, contents string) (LockDescriptor, error)

TryLock is part of the topo.Conn interface. Its implementation is same as Lock

func (*StatsConn) Update 

func (st *StatsConn) Update(ctx context.Context, filePath string, contents []byte, version Version) (Version, error)

Update is part of the Conn interface

func (*StatsConn) Watch 

func (st *StatsConn) Watch(ctx context.Context, filePath string) (current *WatchData, changes <-chan *WatchData, err error)

Watch is part of the Conn interface

func (*StatsConn) WatchRecursive added in v0.15.0 

func (st *StatsConn) WatchRecursive(ctx context.Context, path string) ([]*WatchDataRecursive, <-chan *WatchDataRecursive, error)

type TabletInfo 

type TabletInfo struct {
	*topodatapb.Tablet
	// contains filtered or unexported fields
}

TabletInfo is the container for a Tablet, read from the topology server.

func NewTabletInfo 

func NewTabletInfo(tablet *topodatapb.Tablet, version Version) *TabletInfo

NewTabletInfo returns a TabletInfo basing on tablet with the version set. This function should be only used by Server implementations.

func (*TabletInfo) Addr 

func (ti *TabletInfo) Addr() string

Addr returns hostname:vt port.

func (*TabletInfo) AliasString 

func (ti *TabletInfo) AliasString() string

AliasString returns the string representation of the tablet alias

func (*TabletInfo) DbName 

func (ti *TabletInfo) DbName() string

DbName is usually implied by keyspace. Having the shard information in the database name complicates mysql replication.

func (*TabletInfo) GetPrimaryTermStartTime added in v0.12.0 

func (ti *TabletInfo) GetPrimaryTermStartTime() time.Time

GetPrimaryTermStartTime returns the tablet's primary term start time as a Time value.

func (*TabletInfo) IsInServingGraph 

func (ti *TabletInfo) IsInServingGraph() bool

IsInServingGraph returns if this tablet is in the serving graph

func (*TabletInfo) IsReplicaType 

func (ti *TabletInfo) IsReplicaType() bool

IsReplicaType returns if this tablet's type is a replica

func (*TabletInfo) MysqlAddr 

func (ti *TabletInfo) MysqlAddr() string

MysqlAddr returns hostname:mysql port.

func (*TabletInfo) String 

func (ti *TabletInfo) String() string

String returns a string describing the tablet.

func (*TabletInfo) Version 

func (ti *TabletInfo) Version() Version

Version returns the version of this tablet from last time it was read or updated.

type Version 

type Version interface {
	// String returns a text representation of the version.
	String() string
}

Version is an interface that describes a file version.

type WatchData 

type WatchData struct {
	// Contents has the bytes that were stored by Create
	// or Update.
	Contents []byte

	// Version contains an opaque representation of the Version
	// of that file.
	Version Version

	// Err is set the same way for both the 'current' value
	// returned by Watch, or the values read on the 'changes'
	// channel. It can be:
	// - nil, then Contents and Version are set.
	// - ErrNoNode if the file doesn't exist.
	// - ErrInterrupted if 'cancel' was called.
	// - any other platform-specific error.
	Err error
}

WatchData is the structure returned by the Watch() API. It can contain: a) an error in Err if the call failed (or if the watch was terminated). b) the current or new version of the data.

type WatchDataRecursive added in v0.15.0 

type WatchDataRecursive struct {
	// Path is the path that has changed
	Path string

	WatchData
}

WatchDataRecursive is the structure returned by the WatchRecursive() API. It contains the same data as WatchData, but additionally also the specific path of the entry that the recursive watch applies to, since an entire file prefix can be watched.

type WatchShardData 

type WatchShardData struct {
	Value *topodatapb.Shard
	Err   error
}

WatchShardData wraps the data we receive on the watch channel The WatchShard API guarantees exactly one of Value or Err will be set.

type WatchSrvKeyspaceData 

type WatchSrvKeyspaceData struct {
	Value *topodatapb.SrvKeyspace
	Err   error
}

WatchSrvKeyspaceData is returned / streamed by WatchSrvKeyspace. The WatchSrvKeyspace API guarantees exactly one of Value or Err will be set.

type WatchSrvVSchemaData 

type WatchSrvVSchemaData struct {
	Value *vschemapb.SrvVSchema
	Err   error
}

WatchSrvVSchemaData is returned / streamed by WatchSrvVSchema. The WatchSrvVSchema API guarantees exactly one of Value or Err will be set.

type WorkflowInfo 

type WorkflowInfo struct {
	*workflowpb.Workflow
	// contains filtered or unexported fields
}

WorkflowInfo is a meta struct that contains the version of a Workflow.

Source Files

View all Source files

Directories

Path Synopsis
Package consultopo implements topo.Server with consul as the backend.
 Package consultopo implements topo.Server with consul as the backend.
Package etcd2topo implements topo.Server with etcd as the backend.
 Package etcd2topo implements topo.Server with etcd as the backend.
Package events defines the structures used for events relating directly to individual topology elements, like keyspaces, shards, and tablets.
 Package events defines the structures used for events relating directly to individual topology elements, like keyspaces, shards, and tablets.
Package helpers contains a few utility classes to handle topo.Server objects, and transitions from one topo implementation to another.
 Package helpers contains a few utility classes to handle topo.Server objects, and transitions from one topo implementation to another.
Package memorytopo contains an implementation of the topo.Factory / topo.Conn interfaces based on an in-memory tree of data.
 Package memorytopo contains an implementation of the topo.Factory / topo.Conn interfaces based on an in-memory tree of data.
Package test contains utilities to test topo.Conn implementations.
 Package test contains utilities to test topo.Conn implementations.
Package topoproto contains utility functions to deal with the proto3 structures defined in proto/topodata.
 Package topoproto contains utility functions to deal with the proto3 structures defined in proto/topodata.
Package topotests contains all the unit tests for the topo.Server code that is based on topo.Conn.
 Package topotests contains all the unit tests for the topo.Server code that is based on topo.Conn.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.