README
¶
Vitess
Vitess is a storage platform for scaling MySQL. It is optimized to run as effectively in cloud architectures as it does on dedicated hardware. It combines many important features of MySQL with the scalability of a NoSQL database.
It's been actively developed since 2011, and is currently used as a fundamental component of YouTube's MySQL infrastructure, serving thousands of QPS per server. If you want to find out whether Vitess is a good fit for your project, please visit vitess.io.
There are a couple of videos from sougou that you can watch: a short intro prepared for Google I/O 2014 and a more detailed presentation from @Scale '14.
Documentation
Intro
- Helicopter overview: high level overview of Vitess that should tell you whether Vitess is for you.
- Sharding in Vitess
Using Vitess
- Getting Started
- On Kubernetes.
- From the ground up.
- Architecture: all Vitess tools and servers.
- Reparenting: performing master failover.
- Resharding: adding more shards to your cluster.
- Schema management: managing your database schema using Vitess.
Reference
Contact
Ask questions in the vitess@googlegroups.com discussion forum.
Subscribe to vitess-announce@googlegroups.com for low-frequency updates like new features and releases.
License
Unless otherwise noted, the vitess source files are distributed under the BSD-style license found in the LICENSE file.
Documentation
¶
Overview ¶
test.go is a "Go script" for running Vitess tests. It runs each test in its own Docker container for hermeticity and (potentially) parallelism. If a test fails, this script will save the output in _test/ and continue with other tests.
Before using it, you should have Docker 1.5+ installed, and have your user in the group that lets you run the docker command without sudo. The first time you run against a given flavor, it may take some time for the corresponding bootstrap image (vitess/bootstrap:<flavor>) to be downloaded.
It is meant to be run from the Vitess root, like so:
~/src/github.com/youtube/vitess$ go run test.go [args]
For a list of options, run:
$ go run test.go --help
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
docker
|
|
|
etcd
getsrv is a tiny utility that just calls net.LookupSRV().
|
getsrv is a tiny utility that just calls net.LookupSRV(). |
|
examples
|
|
|
local
client.go is a sample for using the Vitess Go SQL driver with an unsharded keyspace.
|
client.go is a sample for using the Vitess Go SQL driver with an unsharded keyspace. |
|
go
|
|
|
acl
Package acl contains functions to enforce access control lists.
|
Package acl contains functions to enforce access control lists. |
|
bufio2
Package bufio2 implements buffered I/O. It wraps an io.Reader or io.Writer object, creating another object (Reader or AsyncWriter) that also implements the interface but provides buffering and some help for textual I/O.
|
Package bufio2 implements buffered I/O. It wraps an io.Reader or io.Writer object, creating another object (Reader or AsyncWriter) that also implements the interface but provides buffering and some help for textual I/O. |
|
bytes2
Package bytes2 provides alternate implementations of functionality similar to go's bytes package.
|
Package bytes2 provides alternate implementations of functionality similar to go's bytes package. |
|
cache
Package cache implements a LRU cache.
|
Package cache implements a LRU cache. |
|
cgzip
Package cgzip wraps the C library for gzip.
|
Package cgzip wraps the C library for gzip. |
|
cmd/mysqlctl
mysqlctl initializes and controls mysqld with Vitess-specific configuration.
|
mysqlctl initializes and controls mysqld with Vitess-specific configuration. |
|
cmd/mysqlctld
mysqlctld is a daemon that starts or initializes mysqld and provides an RPC interface for vttablet to stop and start mysqld from a different container without having to restart the container running mysqlctld.
|
mysqlctld is a daemon that starts or initializes mysqld and provides an RPC interface for vttablet to stop and start mysqld from a different container without having to restart the container running mysqlctld. |
|
cmd/vtcombo
vtcombo: a single binary that contains: - a ZK topology server based on an in-memory map.
|
vtcombo: a single binary that contains: - a ZK topology server based on an in-memory map. |
|
cmd/vtgateclienttest
Package main is the implementation of vtgateclienttest.
|
Package main is the implementation of vtgateclienttest. |
|
cmd/vtgateclienttest/services
Package services exposes all the services for the vtgateclienttest binary.
|
Package services exposes all the services for the vtgateclienttest binary. |
|
cmd/vtworker
vtworker is the main program to run a worker job.
|
vtworker is the main program to run a worker job. |
|
cmd/zkctl
zkctl initializes and controls ZooKeeper with Vitess-specific configuration.
|
zkctl initializes and controls ZooKeeper with Vitess-specific configuration. |
|
cmd/zkctld
zkctld is a daemon that starts or initializes ZooKeeper with Vitess-specific configuration.
|
zkctld is a daemon that starts or initializes ZooKeeper with Vitess-specific configuration. |
|
event
Package event provides a reflect-based framework for low-frequency global dispatching of events, which are values of any arbitrary type, to a set of listener functions, which are usually registered by plugin packages during init().
|
Package event provides a reflect-based framework for low-frequency global dispatching of events, which are values of any arbitrary type, to a set of listener functions, which are usually registered by plugin packages during init(). |
|
event/syslogger
Package syslogger uses the event package to listen for any event that implements the Syslogger interface.
|
Package syslogger uses the event package to listen for any event that implements the Syslogger interface. |
|
ewma
Package ewma implements exponentially weighted moving averages(EWMA).
|
Package ewma implements exponentially weighted moving averages(EWMA). |
|
exit
Package exit provides an alternative to os.Exit(int).
|
Package exit provides an alternative to os.Exit(int). |
|
fileutil
Package fileutil contains utility functions related to files and paths.
|
Package fileutil contains utility functions related to files and paths. |
|
flagutil
Package flagutil contains flags that parse string lists and string maps.
|
Package flagutil contains flags that parse string lists and string maps. |
|
hack
Package hack gives you some efficient functionality at the cost of breaking some Go rules.
|
Package hack gives you some efficient functionality at the cost of breaking some Go rules. |
|
history
Package history implements a circular buffer with adjacent-item deduplication.
|
Package history implements a circular buffer with adjacent-item deduplication. |
|
ioutil2
Package ioutil2 provides extra functionality along similar lines to io/ioutil.
|
Package ioutil2 provides extra functionality along similar lines to io/ioutil. |
|
memcache
Package memcache is a client for memcached.
|
Package memcache is a client for memcached. |
|
mysql
Package mysql wraps the C client library for MySQL.
|
Package mysql wraps the C client library for MySQL. |
|
netutil
Package netutil contains network-related utility functions.
|
Package netutil contains network-related utility functions. |
|
pools
Package pools provides functionality to manage and reuse resources like connections.
|
Package pools provides functionality to manage and reuse resources like connections. |
|
proc
Package proc allows you to configure servers to be restarted with negligible downtime.
|
Package proc allows you to configure servers to be restarted with negligible downtime. |
|
ratelimiter
Package ratelimiter implements rate limiting functionality.
|
Package ratelimiter implements rate limiting functionality. |
|
sqldb
Package sqldb defines an interface for low level db connection.
|
Package sqldb defines an interface for low level db connection. |
|
sqltypes
Package sqltypes implements interfaces and types that represent SQL values.
|
Package sqltypes implements interfaces and types that represent SQL values. |
|
stats
Package stats is a wrapper for expvar.
|
Package stats is a wrapper for expvar. |
|
stats/influxdbbackend
Package influxdbbackend is useful for publishing metrics to an InfluxDB backend (tested on v0.88).
|
Package influxdbbackend is useful for publishing metrics to an InfluxDB backend (tested on v0.88). |
|
streamlog
Package streamlog provides a non-blocking message broadcaster.
|
Package streamlog provides a non-blocking message broadcaster. |
|
sync2
Package sync2 provides extra functionality along the same lines as sync.
|
Package sync2 provides extra functionality along the same lines as sync. |
|
tb
Package tb exposes some handy traceback functionality buried in the runtime.
|
Package tb exposes some handy traceback functionality buried in the runtime. |
|
testfiles
Package testfiles locates test files within the Vitess directory tree.
|
Package testfiles locates test files within the Vitess directory tree. |
|
timer
Package timer provides various enhanced timer functions.
|
Package timer provides various enhanced timer functions. |
|
trace
Package trace contains a helper interface that allows various tracing tools to be plugged in to components using this interface.
|
Package trace contains a helper interface that allows various tracing tools to be plugged in to components using this interface. |
|
vt/automation
Package automation contains code to execute high-level cluster operations (e.g.
|
Package automation contains code to execute high-level cluster operations (e.g. |
|
vt/automation/resolver
Package resolver provides a mechanism to resolve a vtworker or vtctld hostname to multiple addresses.
|
Package resolver provides a mechanism to resolve a vtworker or vtctld hostname to multiple addresses. |
|
vt/binlog/binlogplayer
Package binlogplayer contains the code that plays a filtered replication stream on a client database.
|
Package binlogplayer contains the code that plays a filtered replication stream on a client database. |
|
vt/binlog/grpcbinlogstreamer
Package grpcbinlogstreamer contains the gRPC implementation of the binlog streamer server component.
|
Package grpcbinlogstreamer contains the gRPC implementation of the binlog streamer server component. |
|
vt/callerid
Package callerid stores/retrives CallerIDs (immediate CallerID and effective CallerID) to/from the Context
|
Package callerid stores/retrives CallerIDs (immediate CallerID and effective CallerID) to/from the Context |
|
vt/callinfo
Package callinfo stores custom values into the Context (related to the RPC source)
|
Package callinfo stores custom values into the Context (related to the RPC source) |
|
vt/dbconfigs
Package dbconfigs is reusable by vt tools to load the db configs file.
|
Package dbconfigs is reusable by vt tools to load the db configs file. |
|
vt/dbconnpool
Package dbconnpool exposes a single DBConnection object with wrapped access to a single DB connection, and a ConnectionPool object to pool these DBConnections.
|
Package dbconnpool exposes a single DBConnection object with wrapped access to a single DB connection, and a ConnectionPool object to pool these DBConnections. |
|
vt/estimator
Package estimator implements future value estimation by EWMA algorithm.
|
Package estimator implements future value estimation by EWMA algorithm. |
|
vt/etcdtopo
Package etcdtopo implements topo.Server with etcd as the backend.
|
Package etcdtopo implements topo.Server with etcd as the backend. |
|
vt/events
Package events defines common structures used for events dispatched from various other package.
|
Package events defines common structures used for events dispatched from various other package. |
|
vt/mysqlctl/backupstorage
Package backupstorage contains the interface and file system implementation of the backup system.
|
Package backupstorage contains the interface and file system implementation of the backup system. |
|
vt/mysqlctl/cephbackupstorage
Package Cephbackupstorage implements the BackupStorage interface for Ceph Cloud Storage.
|
Package Cephbackupstorage implements the BackupStorage interface for Ceph Cloud Storage. |
|
vt/mysqlctl/filebackupstorage
Package filebackupstorage implements the BackupStorage interface for a local filesystem (which can be an NFS mount).
|
Package filebackupstorage implements the BackupStorage interface for a local filesystem (which can be an NFS mount). |
|
vt/mysqlctl/gcsbackupstorage
Package gcsbackupstorage implements the BackupStorage interface for Google Cloud Storage.
|
Package gcsbackupstorage implements the BackupStorage interface for Google Cloud Storage. |
|
vt/mysqlctl/grpcmysqlctlclient
Package grpcmysqlctlclient contains the gRPC1 version of the mysqlctl client protocol.
|
Package grpcmysqlctlclient contains the gRPC1 version of the mysqlctl client protocol. |
|
vt/mysqlctl/grpcmysqlctlserver
Package grpcmysqlctlserver contains the gRPC implementation of the server side of the remote execution of mysqlctl commands.
|
Package grpcmysqlctlserver contains the gRPC implementation of the server side of the remote execution of mysqlctl commands. |
|
vt/mysqlctl/mysqlctlclient
Package mysqlctlclient contains the generic client side of the remote mysqlctl protocol.
|
Package mysqlctlclient contains the generic client side of the remote mysqlctl protocol. |
|
vt/mysqlctl/replication
Package replication contains data structures to deal with MySQL replication.
|
Package replication contains data structures to deal with MySQL replication. |
|
vt/mysqlctl/tmutils
Package tmutils contains helper methods to deal with the tabletmanagerdata proto3 structures.
|
Package tmutils contains helper methods to deal with the tabletmanagerdata proto3 structures. |
|
vt/proto/automation
Package automation is a generated protocol buffer package.
|
Package automation is a generated protocol buffer package. |
|
vt/proto/automationservice
Package automationservice is a generated protocol buffer package.
|
Package automationservice is a generated protocol buffer package. |
|
vt/proto/binlogdata
Package binlogdata is a generated protocol buffer package.
|
Package binlogdata is a generated protocol buffer package. |
|
vt/proto/binlogservice
Package binlogservice is a generated protocol buffer package.
|
Package binlogservice is a generated protocol buffer package. |
|
vt/proto/logutil
Package logutil is a generated protocol buffer package.
|
Package logutil is a generated protocol buffer package. |
|
vt/proto/mysqlctl
Package mysqlctl is a generated protocol buffer package.
|
Package mysqlctl is a generated protocol buffer package. |
|
vt/proto/query
Package query is a generated protocol buffer package.
|
Package query is a generated protocol buffer package. |
|
vt/proto/queryservice
Package queryservice is a generated protocol buffer package.
|
Package queryservice is a generated protocol buffer package. |
|
vt/proto/replicationdata
Package replicationdata is a generated protocol buffer package.
|
Package replicationdata is a generated protocol buffer package. |
|
vt/proto/tableacl
Package tableacl is a generated protocol buffer package.
|
Package tableacl is a generated protocol buffer package. |
|
vt/proto/tabletmanagerdata
Package tabletmanagerdata is a generated protocol buffer package.
|
Package tabletmanagerdata is a generated protocol buffer package. |
|
vt/proto/tabletmanagerservice
Package tabletmanagerservice is a generated protocol buffer package.
|
Package tabletmanagerservice is a generated protocol buffer package. |
|
vt/proto/topodata
Package topodata is a generated protocol buffer package.
|
Package topodata is a generated protocol buffer package. |
|
vt/proto/vtctldata
Package vtctldata is a generated protocol buffer package.
|
Package vtctldata is a generated protocol buffer package. |
|
vt/proto/vtctlservice
Package vtctlservice is a generated protocol buffer package.
|
Package vtctlservice is a generated protocol buffer package. |
|
vt/proto/vtgate
Package vtgate is a generated protocol buffer package.
|
Package vtgate is a generated protocol buffer package. |
|
vt/proto/vtgateservice
Package vtgateservice is a generated protocol buffer package.
|
Package vtgateservice is a generated protocol buffer package. |
|
vt/proto/vtrpc
Package vtrpc is a generated protocol buffer package.
|
Package vtrpc is a generated protocol buffer package. |
|
vt/proto/vtworkerdata
Package vtworkerdata is a generated protocol buffer package.
|
Package vtworkerdata is a generated protocol buffer package. |
|
vt/proto/vtworkerservice
Package vtworkerservice is a generated protocol buffer package.
|
Package vtworkerservice is a generated protocol buffer package. |
|
vt/sqlannotation
Package sqlannotation provides functions for annotating DML statements with keyspace-id comments and parsing them.
|
Package sqlannotation provides functions for annotating DML statements with keyspace-id comments and parsing them. |
|
vt/status
Package status defines a few useful functions for our binaries, mainly to link the status page with a vtctld instance.
|
Package status defines a few useful functions for our binaries, mainly to link the status page with a vtctld instance. |
|
vt/tabletmanager
Package tabletmanager exports the ActionAgent object.
|
Package tabletmanager exports the ActionAgent object. |
|
vt/tabletmanager/events
Package events contains event structs used by the tabletmanager package.
|
Package events contains event structs used by the tabletmanager package. |
|
vt/tabletserver/customrule/filecustomrule
Package filecustomrule implements static custom rule from a config file
|
Package filecustomrule implements static custom rule from a config file |
|
vt/tabletserver/endtoend
Package endtoend is a test-only package.
|
Package endtoend is a test-only package. |
|
vt/tabletserver/fakecacheservice
Package fakecacheservice provides a fake implementation of cacheservice.CacheService
|
Package fakecacheservice provides a fake implementation of cacheservice.CacheService |
|
vt/tabletserver/queryservice
Package queryservice contains the interface for the service definition of the Query Service.
|
Package queryservice contains the interface for the service definition of the Query Service. |
|
vt/tabletserver/querytypes
Package querytypes defines internal types used in the APIs to deal with queries.
|
Package querytypes defines internal types used in the APIs to deal with queries. |
|
vt/tabletserver/splitquery
Package splitquery contains the logic needed for implementing the tabletserver's SplitQuery RPC.
|
Package splitquery contains the logic needed for implementing the tabletserver's SplitQuery RPC. |
|
vt/tabletserver/tabletconntest
Package tabletconntest provides the test methods to make sure a tabletconn/queryservice pair over RPC works correctly.
|
Package tabletconntest provides the test methods to make sure a tabletconn/queryservice pair over RPC works correctly. |
|
vt/tabletserver/tabletservermock
Package tabletservermock provides mock interfaces for tabletserver.
|
Package tabletservermock provides mock interfaces for tabletserver. |
|
vt/topo/events
Package events defines the structures used for events relating directly to individual topology elements, like keyspaces, shards, and tablets.
|
Package events defines the structures used for events relating directly to individual topology elements, like keyspaces, shards, and tablets. |
|
vt/topo/helpers
Package helpers contains a few utility classes to handle topo.Server objects, and transitions from one topo implementation to another.
|
Package helpers contains a few utility classes to handle topo.Server objects, and transitions from one topo implementation to another. |
|
vt/topo/test
Package test contains utilities to test topo.Impl implementations.
|
Package test contains utilities to test topo.Impl implementations. |
|
vt/topo/test/faketopo
Package faketopo contains utitlities for tests that have to interact with a Vitess topology.
|
Package faketopo contains utitlities for tests that have to interact with a Vitess topology. |
|
vt/topo/topoproto
Package topoproto contains utility functions to deal with the proto3 structures defined in proto/topodata.
|
Package topoproto contains utility functions to deal with the proto3 structures defined in proto/topodata. |
|
vt/topotools
Package topotools contains high level functions based on vt/topo and vt/actionnode.
|
Package topotools contains high level functions based on vt/topo and vt/actionnode. |
|
vt/topotools/events
Package events defines the structures used for events dispatched from the wrangler package.
|
Package events defines the structures used for events dispatched from the wrangler package. |
|
vt/vitessdriver
Package vitessdriver contains the Vitess Go SQL driver.
|
Package vitessdriver contains the Vitess Go SQL driver. |
|
vt/vtctl/fakevtctlclient
Package fakevtctlclient contains a fake for the vtctlclient interface.
|
Package fakevtctlclient contains a fake for the vtctlclient interface. |
|
vt/vtctl/grpcvtctlclient
Package grpcvtctlclient contains the gRPC version of the vtctl client protocol
|
Package grpcvtctlclient contains the gRPC version of the vtctl client protocol |
|
vt/vtctl/grpcvtctlserver
Package grpcvtctlserver contains the gRPC implementation of the server side of the remote execution of vtctl commands.
|
Package grpcvtctlserver contains the gRPC implementation of the server side of the remote execution of vtctl commands. |
|
vt/vtctl/vtctlclient
Package vtctlclient contains the generic client side of the remote vtctl protocol.
|
Package vtctlclient contains the generic client side of the remote vtctl protocol. |
|
vt/vtctl/vtctlclienttest
Package vtctlclienttest contains the testsuite against which each RPC implementation of the vtctlclient interface must be tested.
|
Package vtctlclienttest contains the testsuite against which each RPC implementation of the vtctlclient interface must be tested. |
|
vt/vtctld
Package vtctld contains all the code to expose a vtctld server based on the provided topo.Server.
|
Package vtctld contains all the code to expose a vtctld server based on the provided topo.Server. |
|
vt/vterrors
Package vterrors provides helpers for propagating internal errors through the Vitess system (including across RPC boundaries) in a structured way.
|
Package vterrors provides helpers for propagating internal errors through the Vitess system (including across RPC boundaries) in a structured way. |
|
vt/vtgate
Package vtgate provides query routing rpc services for vttablets.
|
Package vtgate provides query routing rpc services for vttablets. |
|
vt/vtgate/fakerpcvtgateconn
Package fakerpcvtgateconn provides a fake implementation of vtgateconn.Impl that doesn't do any RPC, but uses a local map to return results.
|
Package fakerpcvtgateconn provides a fake implementation of vtgateconn.Impl that doesn't do any RPC, but uses a local map to return results. |
|
vt/vtgate/grpcvtgateconn
Package grpcvtgateconn provides gRPC connectivity for VTGate.
|
Package grpcvtgateconn provides gRPC connectivity for VTGate. |
|
vt/vtgate/grpcvtgateservice
Package grpcvtgateservice provides the gRPC glue for vtgate
|
Package grpcvtgateservice provides the gRPC glue for vtgate |
|
vt/vtgate/masterbuffer
Package masterbuffer contains experimental logic to buffer master requests in VTGate.
|
Package masterbuffer contains experimental logic to buffer master requests in VTGate. |
|
vt/vtgate/planbuilder
Package planbuilder allows you to build execution plans that describe how to fulfill a query that may span multiple keyspaces or shards.
|
Package planbuilder allows you to build execution plans that describe how to fulfill a query that may span multiple keyspaces or shards. |
|
vt/vtgate/vtgateconntest
Package vtgateconntest provides the test methods to make sure a vtgateconn/vtgateservice pair over RPC works correctly.
|
Package vtgateconntest provides the test methods to make sure a vtgateconn/vtgateservice pair over RPC works correctly. |
|
vt/vtgate/vtgateservice
Package vtgateservice provides to interface definition for the vtgate service
|
Package vtgateservice provides to interface definition for the vtgate service |
|
vt/vttest
Package vttest provides the functionality to bring up a test cluster.
|
Package vttest provides the functionality to bring up a test cluster. |
|
vt/vttest/fakesqldb
Package fakesqldb provides a fake implementation of sqldb.Conn
|
Package fakesqldb provides a fake implementation of sqldb.Conn |
|
vt/worker
Package worker contains the framework, utility methods and core functions for long running actions.
|
Package worker contains the framework, utility methods and core functions for long running actions. |
|
vt/worker/fakevtworkerclient
Package fakevtworkerclient contains a fake for the vtworkerclient interface.
|
Package fakevtworkerclient contains a fake for the vtworkerclient interface. |
|
vt/worker/grpcvtworkerclient
Package grpcvtworkerclient contains the gRPC version of the vtworker client protocol.
|
Package grpcvtworkerclient contains the gRPC version of the vtworker client protocol. |
|
vt/worker/grpcvtworkerserver
Package grpcvtworkerserver contains the gRPC implementation of the server side of the remote execution of vtworker commands.
|
Package grpcvtworkerserver contains the gRPC implementation of the server side of the remote execution of vtworker commands. |
|
vt/worker/vtworkerclient
Package vtworkerclient contains the generic client side of the remote vtworker protocol.
|
Package vtworkerclient contains the generic client side of the remote vtworker protocol. |
|
vt/worker/vtworkerclienttest
Package vtworkerclienttest contains the testsuite against which each RPC implementation of the vtworkerclient interface must be tested.
|
Package vtworkerclienttest contains the testsuite against which each RPC implementation of the vtworkerclient interface must be tested. |
|
vt/wrangler
Package wrangler contains the Wrangler object to manage complex topology actions.
|
Package wrangler contains the Wrangler object to manage complex topology actions. |
|
vt/wrangler/testlib
Package testlib contains utility methods to include in unit tests to deal with topology common tasks, like fake tablets and action loops.
|
Package testlib contains utility methods to include in unit tests to deal with topology common tasks, like fake tablets and action loops. |
|
zk
Package zk emulates a "global" namespace across n zk quorums.
|
Package zk emulates a "global" namespace across n zk quorums. |
|
zk/fakezk
Package fakezk is a pretty complete mock implementation of a Zookeper connection (see go/zk/zk.Conn).
|
Package fakezk is a pretty complete mock implementation of a Zookeper connection (see go/zk/zk.Conn). |
|
statsd is a simple server for hosting test.go remote stats.
|
statsd is a simple server for hosting test.go remote stats. |
|
The sole purpose of this file is to print the configured value of GOMAXPROCS.
|
The sole purpose of this file is to print the configured value of GOMAXPROCS. |
Click to show internal directories.
Click to hide internal directories.