README ¶
Parser
TiDB SQL Parser
How to use it
See https://godoc.org/github.com/pingcap/parser
How to update parser for TiDB
Assuming that you want to file a PR (pull request) to TiDB, and your PR includes a change in the parser, follow these steps to update the parser in TiDB.
Step 1: Make changes in your parser repository
Fork this repository to your own account and commit the changes to your repository.
Note:
- Don't forget to run
make test
before you commit!- Make sure
parser.go
is updated.
Suppose the forked repository is https://github.com/your-repo/parser
.
Step 2: Make your parser changes take effect in TiDB and run CI
-
In your TiDB repository, execute the
replace
instruction to make your parser changes take effect:GO111MODULE=on go mod edit -replace github.com/pingcap/parser=github.com/your-repo/parser@your-branch
-
make dev
to run CI in TiDB. -
File a PR to TiDB.
Step 3: Merge the PR about the parser to this repository
File a PR to this repository. Link the related PR in TiDB in your PR description or comment.
This PR will be reviewed, and if everything goes well, it will be merged.
Step 4: Update TiDB to use the latest parser
In your TiDB pull request, modify the go.mod
file manually or use this command:
GO111MODULE=on go get -u github.com/pingcap/parser@master
Make sure the replace
instruction is changed back to the require
instruction and the version is the latest.
Documentation ¶
Overview ¶
Index ¶
- Variables
- func DigestHash(sql string) (result string)deprecated
- func DigestNormalized(normalized string) (result string)
- func Normalize(sql string) (result string)
- func NormalizeDigest(sql string) (normalized, digest string)
- func ParseErrorWith(errstr string, lineno int) error
- func ParseHint(input string, sqlMode mysql.SQLMode, initPos Pos) ([]*ast.TableOptimizerHint, []error)
- func TrimComment(txt string) string
- type Parser
- type Pos
- type Scanner
- func (s *Scanner) AppendError(err error)
- func (s *Scanner) EnableWindowFunc(val bool)
- func (s *Scanner) Errorf(format string, a ...interface{}) (err error)
- func (s *Scanner) Errors() (warns []error, errs []error)
- func (s *Scanner) GetSQLMode() mysql.SQLMode
- func (s *Scanner) InheritScanner(sql string) *Scanner
- func (s *Scanner) Lex(v *yySymType) int
- func (s *Scanner) SetSQLMode(mode mysql.SQLMode)
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrWarnOptimizerHintUnsupportedHint = terror.ClassParser.New(mysql.ErrWarnOptimizerHintUnsupportedHint, mysql.MySQLErrName[mysql.ErrWarnOptimizerHintUnsupportedHint]) ErrWarnOptimizerHintInvalidToken = terror.ClassParser.New(mysql.ErrWarnOptimizerHintInvalidToken, mysql.MySQLErrName[mysql.ErrWarnOptimizerHintInvalidToken]) ErrWarnMemoryQuotaOverflow = terror.ClassParser.New(mysql.ErrWarnMemoryQuotaOverflow, mysql.MySQLErrName[mysql.ErrWarnMemoryQuotaOverflow]) ErrWarnOptimizerHintParseError = terror.ClassParser.New(mysql.ErrWarnOptimizerHintParseError, mysql.MySQLErrName[mysql.ErrWarnOptimizerHintParseError]) ErrWarnOptimizerHintInvalidInteger = terror.ClassParser.New(mysql.ErrWarnOptimizerHintInvalidInteger, mysql.MySQLErrName[mysql.ErrWarnOptimizerHintInvalidInteger]) )
var ( // ErrSyntax returns for sql syntax error. ErrSyntax = terror.ClassParser.New(codeErrSyntax, mysql.MySQLErrName[mysql.ErrSyntax]) // ErrParse returns for sql parse error. ErrParse = terror.ClassParser.New(codeErrParse, mysql.MySQLErrName[mysql.ErrParse]) // ErrUnknownCharacterSet returns for no character set found error. ErrUnknownCharacterSet = terror.ClassParser.New(codeErrUnknownCharacterSet, mysql.MySQLErrName[mysql.ErrUnknownCharacterSet]) // ErrInvalidYearColumnLength returns for illegal column length for year type. ErrInvalidYearColumnLength = terror.ClassParser.New(codeErrInvalidYearColumnLength, mysql.MySQLErrName[mysql.ErrInvalidYearColumnLength]) // ErrWrongArguments returns for illegal argument. ErrWrongArguments = terror.ClassParser.New(codeWrongArgument, mysql.MySQLErrName[mysql.ErrWrongArguments]) // ErrWrongFieldTerminators returns for illegal field terminators. ErrWrongFieldTerminators = terror.ClassParser.New(codeWrongFieldTerminators, mysql.MySQLErrName[mysql.ErrWrongFieldTerminators]) // ErrTooBigDisplayWidth returns for data display width exceed limit . ErrTooBigDisplayWidth = terror.ClassParser.New(codeTooBigDisplayWidth, mysql.MySQLErrName[mysql.ErrTooBigDisplaywidth]) // ErrUnknownAlterLock returns for no alter lock type found error. ErrUnknownAlterLock = terror.ClassParser.New(codeErrUnknownAlterLock, mysql.MySQLErrName[mysql.ErrUnknownAlterLock]) // ErrUnknownAlterAlgorithm returns for no alter algorithm found error. ErrUnknownAlterAlgorithm = terror.ClassParser.New(codeErrUnknownAlterAlgorithm, mysql.MySQLErrName[mysql.ErrUnknownAlterAlgorithm]) // SpecFieldPattern special result field pattern SpecFieldPattern = regexp.MustCompile(`(\/\*!(M?[0-9]{5,6})?|\*\/)`) )
var SpecialCommentsController = specialCommentsCtrl{ // contains filtered or unexported fields }
SpecialCommentsController controls whether special comments like `/*T![xxx] yyy */` can be parsed as `yyy`. To add such rules, please use SpecialCommentsController.Register(). For example:
SpecialCommentsController.Register("30100");
Now the parser will treat
select a, /*T![30100] mysterious_keyword */ from t;
and
select a, mysterious_keyword from t;
equally. Similar special comments without registration are ignored by parser.
Functions ¶
func DigestHash
deprecated
DigestHash generates the digest of statements. it will generate a hash on normalized form of statement text which removes general property of a statement but keeps specific property.
for example: both DigestHash('select 1') and DigestHash('select 2') => e1c71d1661ae46e09b7aaec1c390957f0d6260410df4e4bc71b9c8d681021471
Deprecated: It is logically consistent with NormalizeDigest.
func DigestNormalized ¶
DigestNormalized generates the digest of a normalized sql. it will generate a hash on a normalized sql. Normalize + DigestNormalized equals to NormalizeDigest.
for example: DigestNormalized('select ?') DigestNormalized should be called with a normalized SQL string (like 'select ?') generated by function Normalize. do not call with SQL which is not normalized, DigestNormalized('select 1') and DigestNormalized('select 2') is not the same
func Normalize ¶
Normalize generates the normalized statements. it will get normalized form of statement text which removes general property of a statement but keeps specific property.
for example: Normalize('select 1 from b where a = 1') => 'select ? from b where a = ?'
func NormalizeDigest ¶
NormalizeDigest combines Normalize and DigestNormalized into one method.
func ParseErrorWith ¶
ParseErrorWith returns "You have a syntax error near..." error message compatible with mysql.
func ParseHint ¶
func ParseHint(input string, sqlMode mysql.SQLMode, initPos Pos) ([]*ast.TableOptimizerHint, []error)
ParseHint parses an optimizer hint (the interior of `/*+ ... */`).
func TrimComment ¶
TrimComment trim comment for special comment code of MySQL.
Types ¶
type Parser ¶
type Parser struct {
// contains filtered or unexported fields
}
Parser represents a parser instance. Some temporary objects are stored in it to reduce object allocation during Parse function.
func (*Parser) EnableWindowFunc ¶
EnableWindowFunc controls whether the parser to parse syntax related with window function.
func (*Parser) Parse ¶
func (parser *Parser) Parse(sql, charset, collation string) (stmt []ast.StmtNode, warns []error, err error)
Parse parses a query string to raw ast.StmtNode. If charset or collation is "", default charset and collation will be used.
func (*Parser) ParseOneStmt ¶
ParseOneStmt parses a query and returns an ast.StmtNode. The query must have one statement, otherwise ErrSyntax is returned.
func (*Parser) SetSQLMode ¶
SetSQLMode sets the SQL mode for parser.
type Scanner ¶
type Scanner struct {
// contains filtered or unexported fields
}
Scanner implements the yyLexer interface.
func (*Scanner) AppendError ¶
AppendError sets error into scanner. Scanner satisfies yyLexer interface which need this function.
func (*Scanner) EnableWindowFunc ¶
EnableWindowFunc controls whether the scanner recognize the keywords of window function.
func (*Scanner) Errorf ¶
Errorf tells scanner something is wrong. Scanner satisfies yyLexer interface which need this function.
func (*Scanner) GetSQLMode ¶
GetSQLMode return the SQL mode of scanner.
func (*Scanner) InheritScanner ¶
InheritScanner returns a new scanner object which inherits configurations from the parent scanner.
func (*Scanner) Lex ¶
Lex returns a token and store the token value in v. Scanner satisfies yyLexer interface. 0 and invalid are special token id this function would return: return 0 tells parser that scanner meets EOF, return invalid tells parser that scanner meets illegal character.
func (*Scanner) SetSQLMode ¶
SetSQLMode sets the SQL mode for scanner.
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package ast is the abstract syntax tree parsed from a SQL statement by parser.
|
Package ast is the abstract syntax tree parsed from a SQL statement by parser. |
Goyacc is a version of yacc generating Go parsers.
|
Goyacc is a version of yacc generating Go parsers. |