tidb

README

What is TiDB?

TiDB (The pronunciation is: /'taɪdiːbi:/ tai-D-B, etymology: titanium) is a Hybrid Transactional/Analytical Processing (HTAP) database. Inspired by the design of Google F1 and Google Spanner, TiDB features infinite horizontal scalability, strong consistency, and high availability. The goal of TiDB is to serve as a one-stop solution for online transactions and analyses.

• Horizontal scalability

Grow TiDB as your business grows. You can increase the capacity for storage and computation simply by adding more machines.

• Compatible with MySQL protocol

Use TiDB as MySQL. You can replace MySQL with TiDB to power your application without changing a single line of code in most cases.

• Automatic Failover and high availability

Your data and applications are always-on. TiDB automatically handles malfunctions and protects your applications from machine failures or even downtime of an entire data-center.

• Consistent distributed transactions

Think of TiDB as a single-machine RDBMS. You can start a transaction that crosses multiple machines without worrying about consistency. TiDB makes your application code simple and robust.

• Online DDL

Evolve TiDB schemas as your requirement changes. You can add new columns and indexes without stopping or affecting the on-going operations.

• Multiple storage engine support

Power TiDB with your most favorite engines. TiDB supports local storage engines such as GolevelDB, as well as TiKV, a distributed storage engine.

For more details, see How we build TiDB.

Adopters

You can view the list of TiDB adopters here.

Roadmap

Read the Roadmap.

Quick start

Read the Quick Start.

Documentation

• English
• 简体中文

Architecture

Contributing

Contributions are welcomed and greatly appreciated. See CONTRIBUTING.md for details on submitting patches and the contribution workflow.

Connect with us

• Twitter: @PingCAP
• Reddit: https://www.reddit.com/r/TiDB/
• Stack Overflow: https://stackoverflow.com/questions/tagged/tidb
• Mailing list: Google Group

License

TiDB is under the Apache 2.0 license. See the LICENSE file for details.

Acknowledgments

• Thanks cznic for providing some great open source tools.
• Thanks GolevelDB, BoltDB, and RocksDB for their powerful storage engines.

Documentation

Index

• Constants
• Variables
• func BootstrapSession(store kv.Storage) (*domain.Domain, error)
• func Compile(ctx context.Context, sctx sessionctx.Context, stmtNode ast.StmtNode) (ast.Statement, error)
• func DialPumpClientWithRetry(binlogSocket string, maxRetries int, dialerOpt grpc.DialOption) (*grpc.ClientConn, error)
• func GetDomain(store kv.Storage) (*domain.Domain, error)
• func GetRows4Test(ctx context.Context, sctx sessionctx.Context, rs ast.RecordSet) ([]types.Row, error)
• func IsQuery(sql string) bool
• func NewStore(path string) (kv.Storage, error)
• func Parse(ctx sessionctx.Context, src string) ([]ast.StmtNode, error)
• func RegisterStore(name string, driver kv.Driver) error
• func SetCommitRetryLimit(limit uint)
• func SetSchemaLease(lease time.Duration)
• func SetStatsLease(lease time.Duration)
• type Session
  • func CreateSession(store kv.Storage) (Session, error)
  • func CreateSession4Test(store kv.Storage) (Session, error)
• type StmtHistory
  • func GetHistory(ctx sessionctx.Context) *StmtHistory
  • func (h *StmtHistory) Add(stmtID uint32, st ast.Statement, stmtCtx *stmtctx.StatementContext, ...)
  • func (h *StmtHistory) Count() int
• type TxnState
  • func (st *TxnState) Commit(ctx context.Context) error
  • func (st *TxnState) Delete(k kv.Key) error
  • func (st *TxnState) Get(k kv.Key) ([]byte, error)
  • func (st *TxnState) Rollback() error
  • func (st *TxnState) Seek(k kv.Key) (kv.Iterator, error)
  • func (st *TxnState) SeekReverse(k kv.Key) (kv.Iterator, error)
  • func (st *TxnState) Set(k kv.Key, v []byte) error
  • func (st *TxnState) String() string
  • func (st *TxnState) Valid() bool

Constants

const (
	// CreateUserTable is the SQL statement creates User table in system db.
	CreateUserTable = `` /* 1365-byte string literal not displayed */

	// CreateDBPrivTable is the SQL statement creates DB scope privilege table in system db.
	CreateDBPrivTable = `` /* 1127-byte string literal not displayed */

	// CreateTablePrivTable is the SQL statement creates table scope privilege table in system db.
	CreateTablePrivTable = `` /* 380-byte string literal not displayed */

	// CreateColumnPrivTable is the SQL statement creates column scope privilege table in system db.
	CreateColumnPrivTable = `` /* 300-byte string literal not displayed */

	// CreateGloablVariablesTable is the SQL statement creates global variable table in system db.
	// TODO: MySQL puts GLOBAL_VARIABLES table in INFORMATION_SCHEMA db.
	// INFORMATION_SCHEMA is a virtual db in TiDB. So we put this table in system db.
	// Maybe we will put it back to INFORMATION_SCHEMA.
	CreateGloablVariablesTable = `` /* 147-byte string literal not displayed */

	// CreateTiDBTable is the SQL statement creates a table in system db.
	// This table is a key-value struct contains some information used by TiDB.
	// Currently we only put bootstrapped in it which indicates if the system is already bootstrapped.
	CreateTiDBTable = `` /* 160-byte string literal not displayed */

	// CreateHelpTopic is the SQL statement creates help_topic table in system db.
	// See: https://dev.mysql.com/doc/refman/5.5/en/system-database.html#system-database-help-tables
	CreateHelpTopic = `` /* 392-byte string literal not displayed */

	// CreateStatsMetaTable stores the meta of table statistics.
	CreateStatsMetaTable = `` /* 270-byte string literal not displayed */

	// CreateStatsColsTable stores the statistics of table columns.
	CreateStatsColsTable = `` /* 394-byte string literal not displayed */

	// CreateStatsBucketsTable stores the histogram info for every table columns.
	CreateStatsBucketsTable = `` /* 349-byte string literal not displayed */

	// CreateGCDeleteRangeTable stores schemas which can be deleted by DeleteRange.
	CreateGCDeleteRangeTable = `` /* 390-byte string literal not displayed */

)

Variables

var (
	// SchemaOutOfDateRetryInterval is the sleeping time when we fail to try.
	SchemaOutOfDateRetryInterval = int64(500 * time.Millisecond)
	// SchemaOutOfDateRetryTimes is upper bound of retry times when the schema is out of date.
	SchemaOutOfDateRetryTimes = int32(10)
)

var SchemaChangedWithoutRetry bool

SchemaChangedWithoutRetry is used for testing.

Functions

func BootstrapSession

func BootstrapSession(store kv.Storage) (*domain.Domain, error)

BootstrapSession runs the first time when the TiDB server start.

func Compile

func Compile(ctx context.Context, sctx sessionctx.Context, stmtNode ast.StmtNode) (ast.Statement, error)

Compile is safe for concurrent use by multiple goroutines.

func DialPumpClientWithRetry

func DialPumpClientWithRetry(binlogSocket string, maxRetries int, dialerOpt grpc.DialOption) (*grpc.ClientConn, error)

DialPumpClientWithRetry tries to dial to binlogSocket, if any error happens, it will try to re-dial, or return this error when timeout.

func GetDomain

func GetDomain(store kv.Storage) (*domain.Domain, error)

GetDomain gets the associated domain for store.

func GetRows4Test

func GetRows4Test(ctx context.Context, sctx sessionctx.Context, rs ast.RecordSet) ([]types.Row, error)

GetRows4Test gets all the rows from a RecordSet, only used for test.

func IsQuery

func IsQuery(sql string) bool

IsQuery checks if a sql statement is a query statement.

func NewStore

func NewStore(path string) (kv.Storage, error)

NewStore creates a kv Storage with path.

The path must be a URL format 'engine://path?params' like the one for tidb.Open() but with the dbname cut off. Examples:

goleveldb://relative/path
boltdb:///absolute/path

The engine should be registered before creating storage.

func Parse

func Parse(ctx sessionctx.Context, src string) ([]ast.StmtNode, error)

Parse parses a query string to raw ast.StmtNode.

func RegisterStore

func RegisterStore(name string, driver kv.Driver) error

RegisterStore registers a kv storage with unique name and its associated Driver.

func SetCommitRetryLimit

func SetCommitRetryLimit(limit uint)

SetCommitRetryLimit setups the maximum number of retries when trying to recover from retryable errors. Retryable errors are generally refer to temporary errors that are expected to be reinstated by retry, including network interruption, transaction conflicts, and so on.

func SetSchemaLease

func SetSchemaLease(lease time.Duration)

SetSchemaLease changes the default schema lease time for DDL. This function is very dangerous, don't use it if you really know what you do. SetSchemaLease only affects not local storage after bootstrapped.

func SetStatsLease

func SetStatsLease(lease time.Duration)

SetStatsLease changes the default stats lease time for loading stats info.

Types

type Session

type Session interface {
	sessionctx.Context
	Status() uint16                                           // Flag of current status, such as autocommit.
	LastInsertID() uint64                                     // LastInsertID is the last inserted auto_increment ID.
	AffectedRows() uint64                                     // Affected rows by latest executed stmt.
	Execute(context.Context, string) ([]ast.RecordSet, error) // Execute a sql statement.
	String() string                                           // String is used to debug.
	CommitTxn(context.Context) error
	RollbackTxn(context.Context) error
	// PrepareStmt executes prepare statement in binary protocol.
	PrepareStmt(sql string) (stmtID uint32, paramCount int, fields []*ast.ResultField, err error)
	// ExecutePreparedStmt executes a prepared statement.
	ExecutePreparedStmt(ctx context.Context, stmtID uint32, param ...interface{}) (ast.RecordSet, error)
	DropPreparedStmt(stmtID uint32) error
	SetClientCapability(uint32) // Set client capability flags.
	SetConnectionID(uint64)
	SetTLSState(*tls.ConnectionState)
	SetCollation(coID int) error
	SetSessionManager(util.SessionManager)
	Close()
	Auth(user *auth.UserIdentity, auth []byte, salt []byte) bool
	ShowProcess() util.ProcessInfo
	// PrePareTxnCtx is exported for test.
	PrepareTxnCtx(context.Context)
	// FieldList returns fields list of a table.
	FieldList(tableName string) (fields []*ast.ResultField, err error)
}

Session context

func CreateSession

func CreateSession(store kv.Storage) (Session, error)

CreateSession creates a new session environment.

func CreateSession4Test

func CreateSession4Test(store kv.Storage) (Session, error)

CreateSession4Test creates a new session environment for test.

type StmtHistory

type StmtHistory struct {
	// contains filtered or unexported fields
}

StmtHistory holds all histories of statements in a txn.

func GetHistory

func GetHistory(ctx sessionctx.Context) *StmtHistory

GetHistory get all stmtHistory in current txn. Exported only for test.

func (*StmtHistory) Add

func (h *StmtHistory) Add(stmtID uint32, st ast.Statement, stmtCtx *stmtctx.StatementContext, params ...interface{})

Add appends a stmt to history list.

func (*StmtHistory) Count

func (h *StmtHistory) Count() int

Count returns the count of the history.

type TxnState

type TxnState struct {
	// States of a TxnState should be one of the followings:
	// Invalid: kv.Transaction == nil && txnFuture == nil
	// Pending: kv.Transaction == nil && txnFuture != nil
	// Valid:	kv.Transaction != nil && txnFuture == nil
	kv.Transaction
	// contains filtered or unexported fields
}

TxnState wraps kv.Transaction to provide a new kv.Transaction. 1. It holds all statement related modification in the buffer before flush to the txn, so if execute statement meets error, the txn won't be made dirty. 2. It's a lazy transaction, that means it's a txnFuture befort StartTS() is really need.

func (*TxnState) Commit

func (st *TxnState) Commit(ctx context.Context) error

Commit overrides the Transaction interface.

func (*TxnState) Delete

func (st *TxnState) Delete(k kv.Key) error

Delete overrides the Transaction interface.

func (*TxnState) Get

func (st *TxnState) Get(k kv.Key) ([]byte, error)

Get overrides the Transaction interface.

func (*TxnState) Rollback

func (st *TxnState) Rollback() error

Rollback overrides the Transaction interface.

func (*TxnState) Seek

func (st *TxnState) Seek(k kv.Key) (kv.Iterator, error)

Seek overrides the Transaction interface.

func (*TxnState) SeekReverse

func (st *TxnState) SeekReverse(k kv.Key) (kv.Iterator, error)

SeekReverse overrides the Transaction interface.

func (*TxnState) Set

func (st *TxnState) Set(k kv.Key, v []byte) error

Set overrides the Transaction interface.

func (*TxnState) String

func (st *TxnState) String() string

func (*TxnState) Valid

func (st *TxnState) Valid() bool

Valid overrides Transaction interface.