cluster

package
v0.18.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 11, 2024 License: Apache-2.0 Imports: 34 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// DefaultRWPoolSize is the pool size used when creating read-write RPC
	// pools if a config has no size set.
	DefaultRWPoolSize = 50
	// DefaultReadPoolSize is the pool size used when creating read-only RPC
	// pools if a config has no size set.
	DefaultReadPoolSize = 500
	// DefaultRWPoolWaitTimeout is the pool wait timeout used when creating
	// read-write RPC pools if a config has no wait timeout set.
	DefaultRWPoolWaitTimeout = time.Millisecond * 100
	// DefaultReadPoolWaitTimeout is the pool wait timeout used when creating
	// read-only RPC pools if a config has no wait timeout set.
	DefaultReadPoolWaitTimeout = time.Millisecond * 100
)
View Source
var ErrNoConfigID = stderrors.New("loaded config has no id")

ErrNoConfigID is returned from LoadConfig when a cluster spec has a missing or empty id.

Functions

func AnnotateSpan added in v0.10.0

func AnnotateSpan(c *Cluster, span trace.Span)

AnnotateSpan adds the cluster_id and cluster_name to a span.

Types

type Cluster

type Cluster struct {
	ID        string
	Name      string
	Discovery discovery.Discovery

	DB     vtsql.DB
	Vtctld vtctldclient.Proxy

	// Fields for generating FQDNs for tablets
	TabletFQDNTmpl *template.Template
	// contains filtered or unexported fields
}

Cluster is the self-contained unit of services required for vtadmin to talk to a vitess cluster. This consists of a discovery service, a database connection, and a vtctl client.

func New

func New(ctx context.Context, cfg Config) (*Cluster, error)

New creates a new Cluster from a Config.

func (*Cluster) Close added in v0.14.0

func (c *Cluster) Close() error

Close closes a cluster, gracefully closing any open proxy connections to Vtctld(s) or VTGate(s) in the cluster, as well as gracefully shutting-down any background cache goroutines.

Its primary functions are to avoid leaking connections and other resources when dynamic clusters are evicted from an API using dynamic clusters, and to avoid data races in tests (the latter of these is caused by the cache goroutines).

Sub-components of the cluster are `Close`-d concurrently, caches first, then proxy connections.

func (*Cluster) CreateKeyspace added in v0.12.0

CreateKeyspace creates a keyspace in the given cluster, proxying a CreateKeyspaceRequest to a vtctld in that cluster.

func (*Cluster) CreateShard added in v0.13.0

CreateShard creates a shard in the given cluster, proxying a CreateShardRequest to a vtctld in that cluster.

func (*Cluster) Debug added in v0.12.0

func (c *Cluster) Debug() map[string]any

Debug returns a map of debug information for a cluster.

func (*Cluster) DeleteKeyspace added in v0.12.0

DeleteKeyspace deletes a keyspace in the given cluster, proxying a DeleteKeyspaceRequest to a vtctld in that cluster.

func (*Cluster) DeleteShards added in v0.13.0

DeleteShards deletes one or more shards in the given cluster, proxying a single DeleteShardsRequest to a vtctld in that cluster.

func (*Cluster) DeleteTablets added in v0.14.0

DeleteTablets deletes one or more tablets in the given cluster.

func (*Cluster) EmergencyFailoverShard added in v0.14.0

EmergencyFailoverShard fails over a shard to a new primary. It assumes the old primary is dead or otherwise not responding.

func (*Cluster) Equal added in v0.14.0

func (c *Cluster) Equal(otherCluster *Cluster) (bool, error)

Equal compares the vtctld and vtgate addresses of the clusters for equality

func (*Cluster) FindAllShardsInKeyspace added in v0.10.0

func (c *Cluster) FindAllShardsInKeyspace(ctx context.Context, keyspace string, opts FindAllShardsInKeyspaceOptions) (map[string]*vtctldatapb.Shard, error)

FindAllShardsInKeyspace proxies a FindAllShardsInKeyspace RPC to a cluster's vtctld, unpacking the response struct.

It can also optionally ensure the vtctldclient has a valid connection before making the RPC call.

func (*Cluster) FindTablet added in v0.10.0

func (c *Cluster) FindTablet(ctx context.Context, filter func(*vtadminpb.Tablet) bool) (*vtadminpb.Tablet, error)

FindTablet returns the first tablet in a given cluster that satisfies the filter function.

func (*Cluster) FindTablets added in v0.10.0

func (c *Cluster) FindTablets(ctx context.Context, filter func(*vtadminpb.Tablet) bool, n int) ([]*vtadminpb.Tablet, error)

FindTablets returns the first N tablets in the given cluster that satisfy the filter function. If N = -1, then all matching tablets are returned. Ordering is not guaranteed, and callers should write their filter functions accordingly.

func (*Cluster) FindWorkflows added in v0.10.0

func (c *Cluster) FindWorkflows(ctx context.Context, keyspaces []string, opts FindWorkflowsOptions) (*vtadminpb.ClusterWorkflows, error)

FindWorkflows returns a list of Workflows in this cluster, across the given keyspaces and filtering according to the options passed in.

If the list of keyspaces to check is empty, then FindWorkflows will use the result of GetKeyspaces to search all keyspaces in the cluster. In this case, opts.IgnoreKeyspaces is respected.

Callers should use this function when they want more fine-grained filtering, and GetWorkflows when they just want to filter on keyspace name.

Note that if only a subset of keyspaces error on their vtctld GetWorkflows rpc, this is treated as a partial success, and the ClusterWorkflows response will include any errors in the Warnings slice. If all keyspaces fail, or if non-(Vtctld.GetWorkflows) calls fail, this is treated as an error by this function.

func (*Cluster) GetBackups added in v0.11.0

GetBackups returns a ClusterBackups object for all backups in the cluster.

func (*Cluster) GetCellInfos added in v0.14.0

GetCellInfos returns a list of ClusterCellInfo objects for cells in the given cluster.

If req.Cells is set, cells are restricted only to cells with those names. Note: specifying a cell name that does not exist in the cluster fails the overall request.

If req.NamesOnly is set, each ClusterCellInfo will only contain the Cluster and Name fields. req.Cells takes precedence over this option.

func (*Cluster) GetCellsAliases added in v0.14.0

func (c *Cluster) GetCellsAliases(ctx context.Context) (*vtadminpb.ClusterCellsAliases, error)

GetCellsAliases returns all CellsAliases in the cluster.

func (*Cluster) GetGates added in v0.11.0

func (c *Cluster) GetGates(ctx context.Context) ([]*vtadminpb.VTGate, error)

GetGates returns the list of all VTGates in the cluster.

func (*Cluster) GetKeyspace added in v0.11.0

func (c *Cluster) GetKeyspace(ctx context.Context, name string) (*vtadminpb.Keyspace, error)

GetKeyspace returns a single keyspace in the cluster.

func (*Cluster) GetKeyspaces added in v0.11.0

func (c *Cluster) GetKeyspaces(ctx context.Context) ([]*vtadminpb.Keyspace, error)

GetKeyspaces returns all keyspaces, with their shard maps, in the cluster.

func (*Cluster) GetSchema added in v0.10.0

func (c *Cluster) GetSchema(ctx context.Context, keyspace string, opts GetSchemaOptions) (*vtadminpb.Schema, error)

GetSchema returns the schema for a given keyspace. GetSchema has a few different behaviors depending on the GetSchemaOptions provided, as follows:

(1) If opts.SizeOpts.AggregateSizes is true, we will also make a call to FindAllShardsInKeyspace, in order to fan out GetSchema RPCs to a tablet in each shard. If this option is false, we make exactly one GetSchema request to a single, randomly-chosen, tablet in the keyspace.

(1.1) If, in size aggregation mode, opts.SizeOpts.IncludeNonServingShards is false (the default), then we will filter out any shards for which IsPrimaryServing is false in the topo, and make GetSchema RPCs to one tablet in every _serving_ shard. Otherwise we will make a GetSchema RPC to one tablet in _every_ shard.

(2) Irrespective of whether we're including nonserving shards, or whether we're doing size aggregation at all, we will only make GetSchema RPCs to tablets that are in SERVING state; we don't want to use a tablet that might be in a bad state as the source of truth for a schema. Therefore if we can't find a SERVING tablet for the keyspace (in non-aggregation mode) or for a shard in that keyspace (in aggregation mode), then we will return an error back to the caller.

func (*Cluster) GetSchemas added in v0.14.0

func (c *Cluster) GetSchemas(ctx context.Context, opts GetSchemaOptions) ([]*vtadminpb.Schema, error)

GetSchemas returns all of the schemas across all keyspaces in the cluster.

func (*Cluster) GetShardReplicationPositions added in v0.12.0

GetShardReplicationPositions returns a ClusterShardReplicationPosition object for each keyspace/shard in the cluster.

func (*Cluster) GetSrvKeyspaces added in v0.17.0

func (c *Cluster) GetSrvKeyspaces(ctx context.Context, cells []string) (map[string]*vtctldatapb.GetSrvKeyspacesResponse, error)

GetSrvKeyspaces returns all SrvKeyspaces for all keyspaces in a cluster.

func (*Cluster) GetSrvVSchema added in v0.11.0

func (c *Cluster) GetSrvVSchema(ctx context.Context, cell string) (*vtadminpb.SrvVSchema, error)

GetSrvVSchema returns the SrvVSchema for a given cell in the cluster.

func (*Cluster) GetSrvVSchemas added in v0.11.0

func (c *Cluster) GetSrvVSchemas(ctx context.Context, cells []string) ([]*vtadminpb.SrvVSchema, error)

GetSrvVSchemas returns the SrvVSchema for all cells in the cluster, optionally filtered by cell.

func (*Cluster) GetTablets added in v0.10.0

func (c *Cluster) GetTablets(ctx context.Context) ([]*vtadminpb.Tablet, error)

GetTablets returns all tablets in the cluster.

func (*Cluster) GetVSchema added in v0.10.0

func (c *Cluster) GetVSchema(ctx context.Context, keyspace string) (*vtadminpb.VSchema, error)

GetVSchema returns the vschema for a given keyspace in this cluster. The caller is responsible for making at least one call to c.Vtctld.Dial prior to calling this function.

func (*Cluster) GetVtctlds added in v0.12.0

func (c *Cluster) GetVtctlds(ctx context.Context) ([]*vtadminpb.Vtctld, error)

GetVtctlds returns a list of all Vtctlds in the cluster.

func (*Cluster) GetWorkflow added in v0.10.0

func (c *Cluster) GetWorkflow(ctx context.Context, keyspace string, name string, opts GetWorkflowOptions) (*vtadminpb.Workflow, error)

GetWorkflow returns the single Workflow in this cluster for the given keyspace and workflow name. It returns an error if either no workflows or multiple workflows are found.

func (*Cluster) GetWorkflows added in v0.10.0

func (c *Cluster) GetWorkflows(ctx context.Context, keyspaces []string, opts GetWorkflowsOptions) (*vtadminpb.ClusterWorkflows, error)

GetWorkflows returns a list of Workflows in this cluster, across the given keyspaces and filtering according to the options passed in.

If the list of keyspaces to check is empty, then GetWorkflows will use the result of GetKeyspaces to search all keyspaces in the cluster. In this case, opts.IgnoreKeyspaces is respected.

func (*Cluster) PlannedFailoverShard added in v0.14.0

PlannedFailoverShard fails over the shard either to a new primary or away from an old primary. Both the current and candidate primaries must be reachable and running.

func (*Cluster) RefreshState added in v0.14.0

func (c *Cluster) RefreshState(ctx context.Context, tablet *vtadminpb.Tablet) error

RefreshState reloads the tablet record from a cluster's topo on a tablet.

func (*Cluster) RefreshTabletReplicationSource added in v0.14.0

func (c *Cluster) RefreshTabletReplicationSource(ctx context.Context, tablet *vtadminpb.Tablet) (*vtadminpb.RefreshTabletReplicationSourceResponse, error)

RefreshTabletReplicationSource performs a `CHANGE REPLICATION SOURCE TO` on a tablet to replicate from the current primary in the shard.

func (*Cluster) ReloadSchemas added in v0.14.0

ReloadSchemas reloads schemas in one or more keyspaces, shards, or tablets in the cluster, depending on the request parameters.

func (*Cluster) SetWritable added in v0.14.0

func (c *Cluster) SetWritable(ctx context.Context, req *vtctldatapb.SetWritableRequest) error

SetWritable toggles the writability of a tablet, setting it to either read-write or read-only.

func (*Cluster) TabletExternallyPromoted added in v0.14.0

func (c *Cluster) TabletExternallyPromoted(ctx context.Context, tablet *vtadminpb.Tablet) (*vtadminpb.TabletExternallyPromotedResponse, error)

TabletExternallyPromoted updates the topo record for a shard to reflect a tablet that was promoted to primary external to Vitess (e.g. orchestrator).

func (Cluster) ToProto

func (c Cluster) ToProto() *vtadminpb.Cluster

ToProto returns a value-copy protobuf equivalent of the cluster.

func (*Cluster) ToggleTabletReplication added in v0.14.0

func (c *Cluster) ToggleTabletReplication(ctx context.Context, tablet *vtadminpb.Tablet, start bool) (err error)

ToggleTabletReplication either starts or stops replication on the specified tablet.

type ClustersFlag

type ClustersFlag map[string]Config

ClustersFlag implements flag.Value allowing multiple occurrences of a flag to be accumulated into a map.

func (*ClustersFlag) Set

func (cf *ClustersFlag) Set(value string) error

Set is part of the flag.Value interface. It merges the parsed config into the map, allowing ClustersFlag to power a repeated flag. See (*Config).Set for details on flag parsing.

func (*ClustersFlag) String

func (cf *ClustersFlag) String() string

String is part of the flag.Value interface.

func (*ClustersFlag) Type

func (cf *ClustersFlag) Type() string

Type is part of the pflag.Value interface.

type Config

type Config struct {
	ID                   string
	Name                 string
	DiscoveryImpl        string
	DiscoveryFlagsByImpl FlagsByImpl
	TabletFQDNTmplStr    string
	VtSQLFlags           map[string]string
	VtctldFlags          map[string]string

	BackupReadPoolConfig   *RPCPoolConfig
	SchemaReadPoolConfig   *RPCPoolConfig
	TopoRWPoolConfig       *RPCPoolConfig
	TopoReadPoolConfig     *RPCPoolConfig
	WorkflowReadPoolConfig *RPCPoolConfig

	// EmergencyFailoverPoolConfig specifies the config for a pool dedicated
	// solely to EmergencyFailoverShard operations. It has the semantics and
	// defaults of an RW RPCPool.
	EmergencyFailoverPoolConfig *RPCPoolConfig
	// FailoverPoolConfig specifies the config for a pool shared by
	// PlannedFailoverShard operations. It has the semantics and defaults of an
	// RW RPCPool.
	FailoverPoolConfig *RPCPoolConfig

	SchemaCacheConfig *cache.Config
	// contains filtered or unexported fields
}

Config represents the options to configure a vtadmin cluster.

func LoadConfig added in v0.14.0

func LoadConfig(r io.Reader, configType string) (cfg *Config, id string, err error)

LoadConfig reads an io.Reader into viper and tries unmarshal a Config.

The second parameter is used to instruct viper what config type it will read, so it knows what Unmarshaller to use. If no config type is given, LoadConfig defaults to "json". It is the callers responsibility to pass a configType value that viper can use, which, at the time of writing, include "json", "yaml", "toml", "ini", "hcl", "tfvars", "env", and "props".

Any error that occurs during viper's initial read results in a return value of (nil, "", <the error>), and a triple of this shape should indicate to the caller complete failure. If viper is able to read the config, and get a non-empty cluster ID, then the returned id will be non-empty, but the returned Config and error values may or may not be nil, depending on if the Config can be fully unmarshalled or not.

See dynamic.ClusterFromString for additional details on this three-tuple and how callers should expect to use it.

func (Config) Cluster

func (cfg Config) Cluster(ctx context.Context) (*Cluster, error)

Cluster returns a new cluster instance from the given config.

func (*Config) MarshalJSON added in v0.12.0

func (cfg *Config) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

func (Config) Merge

func (cfg Config) Merge(override Config) Config

Merge returns the result of merging the calling config into the passed config. Neither the caller or the argument are modified in any way.

func (*Config) Set

func (cfg *Config) Set(value string) error

Set is part of the flag.Value interface. Each flag is parsed according to the following DSN:

id= // ID or shortname of the cluster.
name= // Name of the cluster.
discovery= // Name of the discovery implementation
discovery-.*= // Per-discovery-implementation flags. These are passed to
              // a given discovery implementation's constructor.
vtsql-.*= // VtSQL-specific flags. Further parsing of these is delegated
          // to the vtsql package.

func (*Config) String

func (cfg *Config) String() string

String is part of the flag.Value interface.

func (*Config) Type

func (cfg *Config) Type() string

Type is part of the pflag.Value interface.

func (Config) WithVtSQLTestConfigOptions added in v0.14.0

func (cfg Config) WithVtSQLTestConfigOptions(opts ...vtsql.ConfigOption) Config

WithVtSQLTestConfigOptions returns a new Config with the given vtsql ConfigOptions appended to any existing ConfigOptions in the current Config.

It should be used in tests only, and is exported to for use in the vtadmin/testutil package.

func (Config) WithVtctldTestConfigOptions added in v0.14.0

func (cfg Config) WithVtctldTestConfigOptions(opts ...vtctldclient.ConfigOption) Config

WithVtctldTestConfigOptions returns a new Config with the given vtctldclient ConfigOptions appended to any existing ConfigOptions in the current Config.

It should be used in tests only, and is exported to for use in the vtadmin/testutil package.

type FileConfig

type FileConfig struct {
	Defaults Config
	Clusters map[string]Config
}

FileConfig represents the structure of a set of cluster configs on disk. It contains both a default config, and cluster-specific overrides. Viper is used internally to load the config file, so any file format supported by viper is permitted.

A valid YAML config looks like:

defaults:
	discovery: k8s
clusters:
	clusterID1:
		name: clusterName1
		discovery-k8s-some-flag: some-val
	clusterID2:
		name: clusterName2
		discovery: consul

func (*FileConfig) Combine

func (fc *FileConfig) Combine(defaults Config, clusters map[string]Config) []Config

Combine combines a FileConfig with a default Config and a ClustersFlag (each defined on the command-line) into a slice of final Configs that are suitable to use for cluster creation.

Combination uses the following precedence: 1. Command-line cluster-specific overrides. 2. File-based cluster-specific overrides. 3. Command-line cluster defaults. 4. File-based cluster defaults.

func (*FileConfig) Set

func (fc *FileConfig) Set(value string) error

Set is part of the flag.Value interface. It loads the file configuration found at the path passed to the flag. Any config file format supported by viper is supported by FileConfig.

func (*FileConfig) String

func (fc *FileConfig) String() string

String is part of the flag.Value interface.

func (*FileConfig) Type

func (fc *FileConfig) Type() string

Type is part of the pflag.Value interface.

type FindAllShardsInKeyspaceOptions added in v0.10.0

type FindAllShardsInKeyspaceOptions struct {
	// contains filtered or unexported fields
}

FindAllShardsInKeyspaceOptions modify the behavior of a cluster's FindAllShardsInKeyspace method.

type FindWorkflowsOptions added in v0.10.0

type FindWorkflowsOptions struct {
	ActiveOnly      bool
	IgnoreKeyspaces sets.Set[string]
	Filter          func(workflow *vtadminpb.Workflow) bool
}

FindWorkflowsOptions is the set of options for FindWorkflows requests.

type FlagsByImpl

type FlagsByImpl map[string]map[string]string

FlagsByImpl groups a set of flags by discovery implementation. Its mapping is impl_name=>flag=>value.

func (*FlagsByImpl) Merge

func (base *FlagsByImpl) Merge(override map[string]map[string]string)

Merge applies the flags in the parameter to the receiver, conflicts are resolved in favor of the parameter and not the receiver.

type GetSchemaOptions added in v0.10.0

type GetSchemaOptions struct {
	// BaseRequest is used to share some common parameters to use for the
	// individual tablet GetSchema RPCs made by (*Cluster).GetSchema, which
	// takes a copy of this request in order to makeb certain overrides as
	// needed, so these mutations are transparent to the caller.
	//
	// The TabletAlias field is ignored completely by (*Cluster).GetSchema, as
	// it is overwritten for each tablet RPC that method makes.
	//
	// The TableSizesOnly field is overwritten only in certain tablet RPCs when
	// SizeOpts.AggregateSizes is true. In order to move minimal bytes over the
	// wire, we assume that schema definitions match across all shards, so we
	// can get the full schema from just one tablet, and then just the table
	// size information from the other N-1 tablets.
	//
	// The TableNamesOnly field is untouched by (*Cluster).GetSchema when not
	// doing size aggregation. However, when doing size aggregation, if
	// TableNamesOnly is true, we log a warning and override it. This is because
	// TableNamesOnly is mutually exclusive with TableSizesOnly, and size
	// aggregation requires setting TableSizesOnly in the cases described above.
	BaseRequest *vtctldatapb.GetSchemaRequest
	// TableSizeOptions control whether the (*Cluster).GetSchema method performs
	// cross-shard table size aggregation (via the AggregateSizes field).
	//
	// If the AggregateSizes field is false, the rest of this struct is ignored,
	// no size aggregation is done, and (*Cluster).GetSchema will make exactly
	// one GetSchema RPC to a SERVING tablet in the keyspace.
	//
	// If the AggregateSizes field is true, (*Cluster).GetSchema will make a
	// FindAllShardsInKeyspace vtctld RPC, and then filter the given Tablets
	// (described above) to find one SERVING tablet for each shard in the
	// keyspace, skipping any non-serving shards in the keyspace.
	TableSizeOptions *vtadminpb.GetSchemaTableSizeOptions
	// contains filtered or unexported fields
}

GetSchemaOptions contains the options that modify the behavior of the (*Cluster).GetSchema method.

type GetWorkflowOptions added in v0.10.0

type GetWorkflowOptions struct {
	ActiveOnly bool
}

GetWorkflowOptions is the set of filtering options for GetWorkflow requests.

type GetWorkflowsOptions added in v0.10.0

type GetWorkflowsOptions struct {
	ActiveOnly      bool
	IgnoreKeyspaces sets.Set[string]
}

GetWorkflowsOptions is the set of filtering options for GetWorkflows requests.

type RPCPoolConfig added in v0.12.0

type RPCPoolConfig struct {
	Size        int           `json:"size"`
	WaitTimeout time.Duration `json:"wait_timeout"`
}

RPCPoolConfig holds configuration options for creating RPCPools.

func (*RPCPoolConfig) NewRWPool added in v0.12.0

func (cfg *RPCPoolConfig) NewRWPool() *pools.RPCPool

NewRWPool returns an RPCPool from the given config that should be used for performing read-write operations. If the config is nil, or has a non-positive size, DefaultRWPoolSize will be used. Similarly, if the config is nil or has a negative wait timeout, DefaultRWPoolWaitTimeout will be used.

func (*RPCPoolConfig) NewReadPool added in v0.12.0

func (cfg *RPCPoolConfig) NewReadPool() *pools.RPCPool

NewReadPool returns an RPCPool from the given config that should be used for performing read-only operations. If the config is nil, or has a non-positive size, DefaultReadPoolSize will be used. Similarly, if the config is nil or has a negative wait timeout, DefaultReadPoolWaitTimeout will be used.

Directories

Path Synopsis
fakediscovery
Package fakediscovery provides a fake, in-memory discovery implementation.
Package fakediscovery provides a fake, in-memory discovery implementation.
internal
caches/schemacache
Package schemacache provides wrapper functions for interacting with instances of the generic (vtadmin/cache).Cache that store schemas.
Package schemacache provides wrapper functions for interacting with instances of the generic (vtadmin/cache).Cache that store schemas.
Package resolver provides a discovery-based resolver for VTAdmin clusters.
Package resolver provides a discovery-based resolver for VTAdmin clusters.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL