parser

README

Generated Documentation and Embedded Help Texts

Many parts of the parser package include special consideration for generation or other production of user-facing documentation. This includes interactive help messages, generated documentation of the set of available functions, or diagrams of the various expressions.

Generated documentation is produced and maintained at compile time, while the interactive, contextual help is returned at runtime.

We equip the generated parser with the ability to report contextual help in two circumstances:

• when the user explicitly requests help with the HELPTOKEN (current syntax: standalone "??")
• when the user makes a grammatical mistake (e.g. INSERT sometable INTO(x, y) ...)

We use the docgen tool to produce the generated documentation files that are then included in the broader (handwritten) published documentation.

Help texts embedded in the grammar

The help is embedded in the grammar using special markers in yacc comments, for example:

// %Help: HELPKEY - shortdescription
// %Category: SomeCat
// %Text: whatever until next %marker at start of line, or non-comment.
// %SeeAlso: whatever until next %marker at start of line, or non-comment.
// %End (optional)

The "HELPKEY" becomes the map key in the generated Go map.

These texts are extracted automatically by help.awk and converted into a Go data structure in help_messages.go.

Support in the parser

Primary mechanism - LALR error recovery

The primary mechanism is leveraging error recovery in LALR parsers using the special error token [1] [2]: when an unexpected token is encountered, the LALR parser will pop tokens on the stack until the prefix matches a grammar rule with the special "error" token (if any). If such a rule exists, its action is used to reduce and the erroneous tokens are discarded.

This mechanism is used both when the user makes a mistake, and when the user inserts the HELPTOKEN in the middle of a statement. When present in the middle of a statement, HELPTOKEN is considered an error and triggers the error recovery.

We use this for contextual help by providing error rules that generate a contextual help text during LALR error recovery.

For example:

backup_stmt:
  BACKUP targets TO string_or_placeholder opt_as_of_clause opt_incremental opt_with_options
  {
    $$.val = &Backup{Targets: $2.targetList(), To: $4.expr(), IncrementalFrom: $6.exprs(), AsOf: $5.asOfClause(), Options: $7.kvOptions()}
  }
| BACKUP error { return helpWith(sqllex, `BACKUP`) }

In this example, the grammar specifies that if the BACKUP keyword is followed by some input tokens such that the first (valid) grammar rule doesn't apply, the parser will "recover from the error" by backtracking up until the point it only sees BACKUP on the stack followed by non-parsable tokens, at which points it takes the error rule and executes its action.

The action is return helpWith(...). What this does is:

• halts parsing (the generated parser executes all actions in a big loop; a return interrupts this loop);
• makes the parser return with an error (the helpWith function returns non-zero);
• extends the parsing error message with a help text; this help text can subsequently be exploited in a client to display the help message in a friendly manner.

Code generation

Since the pattern "{ return helpWith(sqllex, ...) }" is common, we also implement a shorthand syntax based on comments, for example:

backup_stmt:
   ...
| BACKUP error // SHOW HELP: BACKUP

The special comment syntax "SHOW HELP: XXXX" is substituted by means of an auxiliary script (replace_help_rules.awk) into the form explained above.

Secondary mechanism - explicit help token

The mechanism described above works both when the user make a grammatical error and when they place the HELPTOKEN in the middle of a statement, rendering it invalid.

However for contextual help this is not sufficient: what happens if the user requests HELPTOKEN at a position in the grammar where everything before is a complete, valid SQL input?

For example: DELETE FROM foo ?

When encountering this input, the LALR parser will see DELETE FROM foo first, then reduce using the DELETE action because everything up to this point is a valid DELETE statement. When the HELPTOKEN is encountered, the statement has already been completed and the LALR parser doesn't 'know' any more that it was in the context of a DELETE statement.

If we try to place an error-based recovery rule at the top-level:

stmt:
  alter_stmt
| backup_stmt
| ...
| delete_stmt
| ...
| error { ??? }

This wouldn't work: the code inside the error action cannot "observe" the tokens observed so far and there would be no way to know whether the error should be about DELETE, or instead about ALTER, BACKUP, etc.

So in order to handle HELPTOKEN after a valid statement, we must place it in a rule where the context is still available, that is before the statement's grammar rule is reduced.

Where would that be? Suppose we had a simple statement rule:

somesimplestmt:
  SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE error { help ... }

We could extend with:

somesimplestmt:
  SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE DO SOMETHING HELPTOKEN { help ... }
| SIMPLE error { help ... }

(the alternative also works:

somesimplestmt:
  SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE DO SOMETHING error { help ... }
| SIMPLE error { help ... }

)

That is all fine and dandy, but in SQL we have statements with many alternate forms, for example:

alter_rename_table_stmt:
  ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }

To add complementary handling of the help token at the end of valid statements we could, but would hate to, duplicate all the rules:

alter_rename_table_stmt:
  ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME TO qualified_name HELPTOKEN { help ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name HELPTOKEN { help ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name HELPTOKEN { help ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name HELPTOKEN { help ... }

This duplication is horrendous (not to mention hard to maintain), so instead we should attempt to factor the help token in a context where it is still known that we are dealing just with that statement.

The following works:

alter_rename_table_stmt:
  real_alter_rename_table_stmt { $$ = $1 }
| real_alter_rename_table_stmt HELPTOKEN { help ... }

real_alter_rename_table_stmt:
  ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }

Or does it? Without anything else, yacc complains with a "shift/reduce conflict". The reason is coming from the ambiguity: when the parsing stack contains everything sufficient to match a real_alter_rename_table_stmt, there is a choice between reducing the simple form alter_rename_table_stmt: real_alter_rename_table_stmt, or shifting into the more complex form alter_rename_table_stmt: real_alter_rename_table_stmt HELPTOKEN.

This is another form of the textbook situation when yacc is used to parse if-else statements in a programming language: the rule stmt: IF cond THEN body | IF cond THEN body ELSE body is ambiguous (and yields a shift/reduce conflict) for exactly the same reason.

The solution here is also straight out of a textbook: one simply informs yacc of the relative priority between the two candidate rules. In this case, when faced with a neutral choice, we encourage yacc to shift. The particular mechanism is to tell yacc that one rule has a higher priority than another.

It just so happens however that the yacc language only allows us to set relative priorites of tokens, not rules. And here we have a problem, of the two rules that need to be prioritized, only one has a token to work with (the one with HELPTOKEN). Which token should we prioritze for the other?

Conveniently yacc knows about this trouble and offers us an awkward, but working solution: we can tell it "use for this rule the same priority level as an existing token, even though the token is not part of the rule". The syntax for this is rule %prec TOKEN.

We can then use this as follows:

alter_rename_table_stmt:
  real_alter_rename_table_stmt           %prec LOWTOKEN { $$ = $1 }
| real_alter_rename_table_stmt HELPTOKEN %prec HIGHTOKEN { help ... }

We could create two new pseudo-tokens for this (called LOWTOKEN and HIGHTOKEN) however conveniently we can also reuse otherwise valid tokens that have known relative priorities. We settled in our case on VALUES (low priority) and UMINUS (high priority).

Code generation

With the latter mechanism presented above the pattern

rule:
  somerule           %prec VALUES
| somerule HELPTOKEN %prec UMINUS { help ...}`

becomes super common, so we automate it with the following special syntax:

rule:
  somerule // EXTEND WITH HELP: XXX

And the code replacement in replace_help_rules.awk expands this to the form above automatically.

Generated Documentation

Documentation of the SQL functions and operators is generated by the docgen utility, using make generate PKG=./docs/.... The markdown-formatted files are kept in docs/generated/sql and should be re-generated whenever the functions/operators they document change, and indeed if regenerating produces a diff, a CI failure is expected.

References

1. https://www.gnu.org/software/bison/manual/html_node/Error-Recovery.html
2. http://stackoverflow.com/questions/9796608/error-handling-in-yacc

Documentation

Index

• Constants
• Variables
• func DocsURL(pageName string) string
• func EndsInSemicolon(sql string) bool
• func LastLexicalToken(sql string) (lastTok int, ok bool)
• func ParseExpr(sql string) (tree.Expr, error)
• func ParseExprs(sql []string) (tree.Exprs, error)
• func ParseQualifiedTableName(sql string) (*tree.TableName, error)
• func ParseTableName(sql string) (*tree.UnresolvedObjectName, error)
• func ParseType(sql string) (*types.T, error)
• func RunShowSyntax(ctx context.Context, stmt string, ...)
• func SplitFirstStatement(sql string) (pos int, ok bool)
• type HelpMessage
  • func (h *HelpMessage) Format(w io.Writer)
  • func (h *HelpMessage) String() string
• type HelpMessageBody
• type Parser
  • func (p *Parser) Parse(sql string) (Statements, error)
  • func (p *Parser) ParseWithInt(sql string, nakedIntType *types.T) (Statements, error)
• type Statement
  • func ParseOne(sql string) (Statement, error)
• type Statements
  • func Parse(sql string) (Statements, error)
  • func (stmts Statements) String() string
  • func (stmts Statements) StringWithFlags(flags tree.FmtFlags) string
• type TokenString
  • func Tokens(sql string) (tokens []TokenString, ok bool)

Constants

const ABORT = lex.ABORT

const ACTION = lex.ACTION

const ADD = lex.ADD

const ADMIN = lex.ADMIN

const AGGREGATE = lex.AGGREGATE

const ALL = lex.ALL

const ALTER = lex.ALTER

const ANALYSE = lex.ANALYSE

const ANALYZE = lex.ANALYZE

const AND = lex.AND

const AND_AND = lex.AND_AND

const ANNOTATE_TYPE = lex.ANNOTATE_TYPE

const ANY = lex.ANY

const ARRAY = lex.ARRAY

const AS = lex.AS

const ASC = lex.ASC

const ASYMMETRIC = lex.ASYMMETRIC

const AS_LA = lex.AS_LA

const AT = lex.AT

const AUTHORIZATION = lex.AUTHORIZATION

const AUTOMATIC = lex.AUTOMATIC

const BACKUP = lex.BACKUP

const BCONST = lex.BCONST

const BEGIN = lex.BEGIN

const BETWEEN = lex.BETWEEN

const BIGINT = lex.BIGINT

const BIGSERIAL = lex.BIGSERIAL

const BIT = lex.BIT

const BITCONST = lex.BITCONST

const BLOB = lex.BLOB

const BOOL = lex.BOOL

const BOOLEAN = lex.BOOLEAN

const BOTH = lex.BOTH

const BUCKET_COUNT = lex.BUCKET_COUNT

const BUNDLE = lex.BUNDLE

const BY = lex.BY

const BYTEA = lex.BYTEA

const BYTES = lex.BYTES

const CACHE = lex.CACHE

const CANCEL = lex.CANCEL

const CASCADE = lex.CASCADE

const CASE = lex.CASE

const CAST = lex.CAST

const CHANGEFEED = lex.CHANGEFEED

const CHAR = lex.CHAR

const CHARACTER = lex.CHARACTER

const CHARACTERISTICS = lex.CHARACTERISTICS

const CHECK = lex.CHECK

const CLUSTER = lex.CLUSTER

const COALESCE = lex.COALESCE

const COLLATE = lex.COLLATE

const COLLATION = lex.COLLATION

const COLUMN = lex.COLUMN

const COLUMNS = lex.COLUMNS

const COMMENT = lex.COMMENT

const COMMIT = lex.COMMIT

const COMMITTED = lex.COMMITTED

const COMPACT = lex.COMPACT

const COMPLETE = lex.COMPLETE

const CONCAT = lex.CONCAT

const CONCURRENTLY = lex.CONCURRENTLY

const CONFIGURATION = lex.CONFIGURATION

const CONFIGURATIONS = lex.CONFIGURATIONS

const CONFIGURE = lex.CONFIGURE

const CONFLICT = lex.CONFLICT

const CONSTRAINT = lex.CONSTRAINT

const CONSTRAINTS = lex.CONSTRAINTS

const CONTAINED_BY = lex.CONTAINED_BY

const CONTAINS = lex.CONTAINS

const CONVERSION = lex.CONVERSION

const COPY = lex.COPY

const COVERING = lex.COVERING

const CREATE = lex.CREATE

const CREATEROLE = lex.CREATEROLE

const CROSS = lex.CROSS

const CUBE = lex.CUBE

const CURRENT = lex.CURRENT

const CURRENT_CATALOG = lex.CURRENT_CATALOG

const CURRENT_DATE = lex.CURRENT_DATE

const CURRENT_ROLE = lex.CURRENT_ROLE

const CURRENT_SCHEMA = lex.CURRENT_SCHEMA

const CURRENT_TIME = lex.CURRENT_TIME

const CURRENT_TIMESTAMP = lex.CURRENT_TIMESTAMP

const CURRENT_USER = lex.CURRENT_USER

const CYCLE = lex.CYCLE

const DATA = lex.DATA

const DATABASE = lex.DATABASE

const DATABASES = lex.DATABASES

const DATE = lex.DATE

const DAY = lex.DAY

const DEALLOCATE = lex.DEALLOCATE

const DEC = lex.DEC

const DECIMAL = lex.DECIMAL

const DEFAULT = lex.DEFAULT

const DEFERRABLE = lex.DEFERRABLE

const DEFERRED = lex.DEFERRED

const DELETE = lex.DELETE

const DESC = lex.DESC

const DISCARD = lex.DISCARD

const DISTINCT = lex.DISTINCT

const DO = lex.DO

const DOMAIN = lex.DOMAIN

const DOT_DOT = lex.DOT_DOT

const DOUBLE = lex.DOUBLE

const DROP = lex.DROP

const ELSE = lex.ELSE

const ENCODING = lex.ENCODING

const END = lex.END

const ENUM = lex.ENUM

const ERROR = lex.ERROR

const ESCAPE = lex.ESCAPE

const EXCEPT = lex.EXCEPT

const EXCLUDE = lex.EXCLUDE

const EXECUTE = lex.EXECUTE

const EXISTS = lex.EXISTS

const EXPERIMENTAL = lex.EXPERIMENTAL

const EXPERIMENTAL_AUDIT = lex.EXPERIMENTAL_AUDIT

const EXPERIMENTAL_FINGERPRINTS = lex.EXPERIMENTAL_FINGERPRINTS

const EXPERIMENTAL_RELOCATE = lex.EXPERIMENTAL_RELOCATE

const EXPERIMENTAL_REPLICA = lex.EXPERIMENTAL_REPLICA

const EXPIRATION = lex.EXPIRATION

const EXPLAIN = lex.EXPLAIN

const EXPORT = lex.EXPORT

const EXTENSION = lex.EXTENSION

const EXTRACT = lex.EXTRACT

const EXTRACT_DURATION = lex.EXTRACT_DURATION

const FALSE = lex.FALSE

const FAMILY = lex.FAMILY

const FCONST = lex.FCONST

const FETCH = lex.FETCH

const FETCHTEXT = lex.FETCHTEXT

const FETCHTEXT_PATH = lex.FETCHTEXT_PATH

const FETCHVAL = lex.FETCHVAL

const FETCHVAL_PATH = lex.FETCHVAL_PATH

const FILES = lex.FILES

const FILTER = lex.FILTER

const FIRST = lex.FIRST

const FLOAT = lex.FLOAT

const FLOAT4 = lex.FLOAT4

const FLOAT8 = lex.FLOAT8

const FLOORDIV = lex.FLOORDIV

const FOLLOWING = lex.FOLLOWING

const FOR = lex.FOR

const FORCE_INDEX = lex.FORCE_INDEX

const FOREIGN = lex.FOREIGN

const FROM = lex.FROM

const FULL = lex.FULL

const FUNCTION = lex.FUNCTION

const GLOBAL = lex.GLOBAL

const GRANT = lex.GRANT

const GRANTS = lex.GRANTS

const GREATER_EQUALS = lex.GREATER_EQUALS

const GREATEST = lex.GREATEST

const GROUP = lex.GROUP

const GROUPING = lex.GROUPING

const GROUPS = lex.GROUPS

const HASH = lex.HASH

const HAVING = lex.HAVING

const HELPTOKEN = lex.HELPTOKEN

const HIGH = lex.HIGH

const HISTOGRAM = lex.HISTOGRAM

const HOUR = lex.HOUR

const ICONST = lex.ICONST

const IDENT = lex.IDENT

const IF = lex.IF

const IFERROR = lex.IFERROR

const IFNULL = lex.IFNULL

const IGNORE_FOREIGN_KEYS = lex.IGNORE_FOREIGN_KEYS

const ILIKE = lex.ILIKE

const IMMEDIATE = lex.IMMEDIATE

const IMPORT = lex.IMPORT

const IN = lex.IN

const INCLUDE = lex.INCLUDE

const INCREMENT = lex.INCREMENT

const INCREMENTAL = lex.INCREMENTAL

const INDEX = lex.INDEX

const INDEXES = lex.INDEXES

const INET = lex.INET

const INET_CONTAINED_BY_OR_EQUALS = lex.INET_CONTAINED_BY_OR_EQUALS

const INET_CONTAINS_OR_EQUALS = lex.INET_CONTAINS_OR_EQUALS

const INITIALLY = lex.INITIALLY

const INJECT = lex.INJECT

const INNER = lex.INNER

const INSERT = lex.INSERT

const INT = lex.INT

const INT2 = lex.INT2

const INT2VECTOR = lex.INT2VECTOR

const INT4 = lex.INT4

const INT64 = lex.INT64

const INT8 = lex.INT8

const INTEGER = lex.INTEGER

const INTERLEAVE = lex.INTERLEAVE

const INTERSECT = lex.INTERSECT

const INTERVAL = lex.INTERVAL

const INTO = lex.INTO

const INVERTED = lex.INVERTED

const IS = lex.IS

const ISERROR = lex.ISERROR

const ISNULL = lex.ISNULL

const ISOLATION = lex.ISOLATION

const JOB = lex.JOB

const JOBS = lex.JOBS

const JOIN = lex.JOIN

const JSON = lex.JSON

const JSONB = lex.JSONB

const JSON_ALL_EXISTS = lex.JSON_ALL_EXISTS

const JSON_SOME_EXISTS = lex.JSON_SOME_EXISTS

const KEY = lex.KEY

const KEYS = lex.KEYS

const KV = lex.KV

const LANGUAGE = lex.LANGUAGE

const LAST = lex.LAST

const LATERAL = lex.LATERAL

const LC_COLLATE = lex.LC_COLLATE

const LC_CTYPE = lex.LC_CTYPE

const LEADING = lex.LEADING

const LEASE = lex.LEASE

const LEAST = lex.LEAST

const LEFT = lex.LEFT

const LESS = lex.LESS

const LESS_EQUALS = lex.LESS_EQUALS

const LEVEL = lex.LEVEL

const LIKE = lex.LIKE

const LIMIT = lex.LIMIT

const LIST = lex.LIST

const LOCAL = lex.LOCAL

const LOCALTIME = lex.LOCALTIME

const LOCALTIMESTAMP = lex.LOCALTIMESTAMP

const LOCKED = lex.LOCKED

const LOGIN = lex.LOGIN

const LOOKUP = lex.LOOKUP

const LOW = lex.LOW

const LSHIFT = lex.LSHIFT

const MATCH = lex.MATCH

const MATERIALIZED = lex.MATERIALIZED

const MAXVALUE = lex.MAXVALUE

const MERGE = lex.MERGE

const MINUTE = lex.MINUTE

const MINVALUE = lex.MINVALUE

const MONTH = lex.MONTH

const MaxInt = int(MaxUint >> 1)

const MaxUint = ^uint(0)

const NAME = lex.NAME

const NAMES = lex.NAMES

const NAN = lex.NAN

const NATURAL = lex.NATURAL

const NEXT = lex.NEXT

const NO = lex.NO

const NOCREATEROLE = lex.NOCREATEROLE

const NOLOGIN = lex.NOLOGIN

const NONE = lex.NONE

const NORMAL = lex.NORMAL

const NOT = lex.NOT

const NOTHING = lex.NOTHING

const NOTNULL = lex.NOTNULL

const NOT_EQUALS = lex.NOT_EQUALS

const NOT_LA = lex.NOT_LA

const NOT_REGIMATCH = lex.NOT_REGIMATCH

const NOT_REGMATCH = lex.NOT_REGMATCH

const NOWAIT = lex.NOWAIT

const NO_INDEX_JOIN = lex.NO_INDEX_JOIN

const NULL = lex.NULL

const NULLIF = lex.NULLIF

const NULLS = lex.NULLS

const NUMERIC = lex.NUMERIC

const OF = lex.OF

const OFF = lex.OFF

const OFFSET = lex.OFFSET

const OID = lex.OID

const OIDS = lex.OIDS

const OIDVECTOR = lex.OIDVECTOR

const ON = lex.ON

const ONLY = lex.ONLY

const OPERATOR = lex.OPERATOR

const OPT = lex.OPT

const OPTION = lex.OPTION

const OPTIONS = lex.OPTIONS

const OR = lex.OR

const ORDER = lex.ORDER

const ORDINALITY = lex.ORDINALITY

const OTHERS = lex.OTHERS

const OUT = lex.OUT

const OUTER = lex.OUTER

const OVER = lex.OVER

const OVERLAPS = lex.OVERLAPS

const OVERLAY = lex.OVERLAY

const OWNED = lex.OWNED

const PARENT = lex.PARENT

const PARTIAL = lex.PARTIAL

const PARTITION = lex.PARTITION

const PARTITIONS = lex.PARTITIONS

const PASSWORD = lex.PASSWORD

const PAUSE = lex.PAUSE

const PHYSICAL = lex.PHYSICAL

const PLACEHOLDER = lex.PLACEHOLDER

const PLACING = lex.PLACING

const PLAN = lex.PLAN

const PLANS = lex.PLANS

const POSITION = lex.POSITION

const POSTFIXOP = lex.POSTFIXOP

const PRECEDING = lex.PRECEDING

const PRECISION = lex.PRECISION

const PREPARE = lex.PREPARE

const PRESERVE = lex.PRESERVE

const PRIMARY = lex.PRIMARY

const PRIORITY = lex.PRIORITY

const PROCEDURAL = lex.PROCEDURAL

const PUBLIC = lex.PUBLIC

const PUBLICATION = lex.PUBLICATION

const QUERIES = lex.QUERIES

const QUERY = lex.QUERY

const RANGE = lex.RANGE

const RANGES = lex.RANGES

const READ = lex.READ

const REAL = lex.REAL

const RECURSIVE = lex.RECURSIVE

const REF = lex.REF

const REFERENCES = lex.REFERENCES

const REGCLASS = lex.REGCLASS

const REGIMATCH = lex.REGIMATCH

const REGNAMESPACE = lex.REGNAMESPACE

const REGPROC = lex.REGPROC

const REGPROCEDURE = lex.REGPROCEDURE

const REGTYPE = lex.REGTYPE

const REINDEX = lex.REINDEX

const RELEASE = lex.RELEASE

const REMOVE_PATH = lex.REMOVE_PATH

const RENAME = lex.RENAME

const REPEATABLE = lex.REPEATABLE

const REPLACE = lex.REPLACE

const RESET = lex.RESET

const RESTORE = lex.RESTORE

const RESTRICT = lex.RESTRICT

const RESUME = lex.RESUME

const RETURNING = lex.RETURNING

const REVOKE = lex.REVOKE

const RIGHT = lex.RIGHT

const ROLE = lex.ROLE

const ROLES = lex.ROLES

const ROLLBACK = lex.ROLLBACK

const ROLLUP = lex.ROLLUP

const ROW = lex.ROW

const ROWS = lex.ROWS

const RSHIFT = lex.RSHIFT

const RULE = lex.RULE

const SAVEPOINT = lex.SAVEPOINT

const SCATTER = lex.SCATTER

const SCHEMA = lex.SCHEMA

const SCHEMAS = lex.SCHEMAS

const SCONST = lex.SCONST

const SCRUB = lex.SCRUB

const SEARCH = lex.SEARCH

const SECOND = lex.SECOND

const SELECT = lex.SELECT

const SEQUENCE = lex.SEQUENCE

const SEQUENCES = lex.SEQUENCES

const SERIAL = lex.SERIAL

const SERIAL2 = lex.SERIAL2

const SERIAL4 = lex.SERIAL4

const SERIAL8 = lex.SERIAL8

const SERIALIZABLE = lex.SERIALIZABLE

const SERVER = lex.SERVER

const SESSION = lex.SESSION

const SESSIONS = lex.SESSIONS

const SESSION_USER = lex.SESSION_USER

const SET = lex.SET

const SETTING = lex.SETTING

const SETTINGS = lex.SETTINGS

const SHARE = lex.SHARE

const SHOW = lex.SHOW

const SIMILAR = lex.SIMILAR

const SIMPLE = lex.SIMPLE

const SKIP = lex.SKIP

const SMALLINT = lex.SMALLINT

const SMALLSERIAL = lex.SMALLSERIAL

const SNAPSHOT = lex.SNAPSHOT

const SOME = lex.SOME

const SPLIT = lex.SPLIT

const SQL = lex.SQL

const START = lex.START

const STATISTICS = lex.STATISTICS

const STATUS = lex.STATUS

const STDIN = lex.STDIN

const STORE = lex.STORE

const STORED = lex.STORED

const STORING = lex.STORING

const STRICT = lex.STRICT

const STRING = lex.STRING

const SUBSCRIPTION = lex.SUBSCRIPTION

const SUBSTRING = lex.SUBSTRING

const SYMMETRIC = lex.SYMMETRIC

const SYNTAX = lex.SYNTAX

const SYSTEM = lex.SYSTEM

const TABLE = lex.TABLE

const TABLES = lex.TABLES

const TEMP = lex.TEMP

const TEMPLATE = lex.TEMPLATE

const TEMPORARY = lex.TEMPORARY

const TESTING_RELOCATE = lex.TESTING_RELOCATE

const TEXT = lex.TEXT

const THEN = lex.THEN

const THROTTLING = lex.THROTTLING

const TIES = lex.TIES

const TIME = lex.TIME

const TIMESTAMP = lex.TIMESTAMP

const TIMESTAMPTZ = lex.TIMESTAMPTZ

const TIMETZ = lex.TIMETZ

const TO = lex.TO

const TRACE = lex.TRACE

const TRACING = lex.TRACING

const TRAILING = lex.TRAILING

const TRANSACTION = lex.TRANSACTION

const TREAT = lex.TREAT

const TRIGGER = lex.TRIGGER

const TRIM = lex.TRIM

const TRUE = lex.TRUE

const TRUNCATE = lex.TRUNCATE

const TRUSTED = lex.TRUSTED

const TYPE = lex.TYPE

const TYPEANNOTATE = lex.TYPEANNOTATE

const TYPECAST = lex.TYPECAST

const UMINUS = lex.UMINUS

const UNBOUNDED = lex.UNBOUNDED

const UNCOMMITTED = lex.UNCOMMITTED

const UNION = lex.UNION

const UNIQUE = lex.UNIQUE

const UNKNOWN = lex.UNKNOWN

const UNLOGGED = lex.UNLOGGED

const UNSPLIT = lex.UNSPLIT

const UNTIL = lex.UNTIL

const UPDATE = lex.UPDATE

const UPSERT = lex.UPSERT

const USE = lex.USE

const USER = lex.USER

const USERS = lex.USERS

const USING = lex.USING

const UUID = lex.UUID

const VALID = lex.VALID

const VALIDATE = lex.VALIDATE

const VALUE = lex.VALUE

const VALUES = lex.VALUES

const VARBIT = lex.VARBIT

const VARCHAR = lex.VARCHAR

const VARIADIC = lex.VARIADIC

const VARYING = lex.VARYING

const VIEW = lex.VIEW

const VIRTUAL = lex.VIRTUAL

const WHEN = lex.WHEN

const WHERE = lex.WHERE

const WINDOW = lex.WINDOW

const WITH = lex.WITH

const WITHIN = lex.WITHIN

const WITHOUT = lex.WITHOUT

const WITH_LA = lex.WITH_LA

const WORK = lex.WORK

const WRITE = lex.WRITE

const YEAR = lex.YEAR

const ZONE = lex.ZONE

Variables

var AllHelp = func(h map[string]HelpMessageBody) string {

	cmds := make(map[string][]string)
	for c, details := range h {
		if details.Category == "" {
			continue
		}
		cmds[details.Category] = append(cmds[details.Category], c)
	}

	// Ensure the result is deterministic.
	var categories []string
	for c, l := range cmds {
		categories = append(categories, c)
		sort.Strings(l)
	}
	sort.Strings(categories)

	// Compile the final help index.
	var buf bytes.Buffer
	w := tabwriter.NewWriter(&buf, 0, 0, 1, ' ', 0)
	for _, cat := range categories {
		fmt.Fprintf(w, "%s:\n", strings.Title(cat))
		for _, item := range cmds[cat] {
			fmt.Fprintf(w, "\t\t%s\t%s\n", item, h[item].ShortDescription)
		}
		fmt.Fprintln(w)
	}
	_ = w.Flush()
	return buf.String()
}(helpMessages)

AllHelp contains an overview of all statements with help messages. For example, displayed in the CLI shell with \h without additional parameters.

var DocsURLBase = "https://www.cockroachlabs.com/docs/" + "v20.1.11"

DocsURLBase is the root URL for the version of the docs associated with this binary.

var HelpMessages = func(h map[string]HelpMessageBody) map[string]HelpMessageBody {
	appendSeeAlso := func(newItem, prevItems string) string {

		if prevItems != "" {
			return newItem + "\n  " + prevItems
		}
		return newItem
	}
	reformatSeeAlso := func(seeAlso string) string {
		return strings.Replace(
			strings.Replace(seeAlso, ", ", "\n  ", -1),
			"WEBDOCS", DocsURLBase, -1)
	}
	srcMsg := h["<SOURCE>"]
	srcMsg.SeeAlso = reformatSeeAlso(strings.TrimSpace(srcMsg.SeeAlso))
	selectMsg := h["<SELECTCLAUSE>"]
	selectMsg.SeeAlso = reformatSeeAlso(strings.TrimSpace(selectMsg.SeeAlso))
	for k, m := range h {
		m = h[k]
		m.ShortDescription = strings.TrimSpace(m.ShortDescription)
		m.Text = strings.TrimSpace(m.Text)
		m.SeeAlso = strings.TrimSpace(m.SeeAlso)

		if strings.Contains(m.Text, "<source>") && k != "<SOURCE>" {
			m.Text = strings.TrimSpace(m.Text) + "\n\n" + strings.TrimSpace(srcMsg.Text)
			m.SeeAlso = appendSeeAlso(srcMsg.SeeAlso, m.SeeAlso)
		}

		if strings.Contains(m.Text, "<selectclause>") && k != "<SELECTCLAUSE>" {
			m.Text = strings.TrimSpace(m.Text) + "\n\n" + strings.TrimSpace(selectMsg.Text)
			m.SeeAlso = appendSeeAlso(selectMsg.SeeAlso, m.SeeAlso)
		}

		if strings.Contains(m.Text, "<tablename>") {
			m.SeeAlso = appendSeeAlso("SHOW TABLES", m.SeeAlso)
		}
		m.SeeAlso = reformatSeeAlso(m.SeeAlso)
		h[k] = m
	}
	return h
}(helpMessages)

HelpMessages is the registry of all help messages, keyed by the top-level statement that they document. The key is intended for use via the \h client-side command.

Functions

func DocsURL

func DocsURL(pageName string) string

DocsURL generates the URL to pageName in the version of the docs associated with this binary.

func EndsInSemicolon

func EndsInSemicolon(sql string) bool

EndsInSemicolon returns true if the last lexical token is a semicolon.

func LastLexicalToken

func LastLexicalToken(sql string) (lastTok int, ok bool)

LastLexicalToken returns the last lexical token. If the string has no lexical tokens, returns 0 and ok=false.

func ParseExpr

func ParseExpr(sql string) (tree.Expr, error)

ParseExpr is a short-hand for parseExprs([]string{sql})

func ParseExprs

func ParseExprs(sql []string) (tree.Exprs, error)

ParseExprs is a short-hand for parseExprs(sql)

func ParseQualifiedTableName

func ParseQualifiedTableName(sql string) (*tree.TableName, error)

ParseQualifiedTableName parses a SQL string of the form `[ database_name . ] [ schema_name . ] table_name`.

func ParseTableName

func ParseTableName(sql string) (*tree.UnresolvedObjectName, error)

ParseTableName parses a table name.

func ParseType

func ParseType(sql string) (*types.T, error)

ParseType parses a column type.

func RunShowSyntax

func RunShowSyntax(
	ctx context.Context,
	stmt string,
	report func(ctx context.Context, field, msg string),
	reportErr func(ctx context.Context, err error),
)

RunShowSyntax analyzes the syntax and reports its structure as data for the client. Even an error is reported as data.

Since errors won't propagate to the client as an error, but as a result, the usual code path to capture and record errors will not be triggered. Instead, the caller can pass a reportErr closure to capture errors instead. May be nil.

func SplitFirstStatement

func SplitFirstStatement(sql string) (pos int, ok bool)

SplitFirstStatement returns the length of the prefix of the string up to and including the first semicolon that separates statements. If there is no semicolon, returns ok=false.

Types

type HelpMessage

type HelpMessage struct {
	// Command is set if the message is about a statement.
	Command string
	// Function is set if the message is about a built-in function.
	Function string

	// HelpMessageBody contains the details of the message.
	HelpMessageBody
}

HelpMessage describes a contextual help message.

func (*HelpMessage) Format

func (h *HelpMessage) Format(w io.Writer)

Format prints out details about the message onto the specified output stream.

func (*HelpMessage) String

func (h *HelpMessage) String() string

String implements the fmt.String interface.

type HelpMessageBody

type HelpMessageBody struct {
	Category         string
	ShortDescription string
	Text             string
	SeeAlso          string
}

HelpMessageBody defines the body of a help text. The messages are structured to facilitate future help navigation functionality.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser wraps a scanner, parser and other utilities present in the parser package.

func (*Parser) Parse

func (p *Parser) Parse(sql string) (Statements, error)

Parse parses the sql and returns a list of statements.

func (*Parser) ParseWithInt

func (p *Parser) ParseWithInt(sql string, nakedIntType *types.T) (Statements, error)

ParseWithInt parses a sql statement string and returns a list of Statements. The INT token will result in the specified TInt type.

type Statement

type Statement struct {
	// AST is the root of the AST tree for the parsed statement.
	AST tree.Statement

	// SQL is the original SQL from which the statement was parsed. Note that this
	// is not appropriate for use in logging, as it may contain passwords and
	// other sensitive data.
	SQL string

	// NumPlaceholders indicates the number of arguments to the statement (which
	// are referenced through placeholders). This corresponds to the highest
	// argument position (i.e. the x in "$x") that appears in the query.
	//
	// Note: where there are "gaps" in the placeholder positions, this number is
	// based on the highest position encountered. For example, for `SELECT $3`,
	// NumPlaceholders is 3. These cases are malformed and will result in a
	// type-check error.
	NumPlaceholders int

	// NumAnnotations indicates the number of annotations in the tree. It is equal
	// to the maximum annotation index.
	NumAnnotations tree.AnnotationIdx
}

Statement is the result of parsing a single statement. It contains the AST node along with other information.

func ParseOne

func ParseOne(sql string) (Statement, error)

ParseOne parses a sql statement string, ensuring that it contains only a single statement, and returns that Statement. ParseOne will always interpret the INT and SERIAL types as 64-bit types, since this is used in various internal-execution paths where we might receive bits of SQL from other nodes. In general, we expect that all user-generated SQL has been run through the ParseWithInt() function.

type Statements

type Statements []Statement

Statements is a list of parsed statements.

func Parse

func Parse(sql string) (Statements, error)

Parse parses a sql statement string and returns a list of Statements.

func (Statements) String

func (stmts Statements) String() string

String returns the AST formatted as a string.

func (Statements) StringWithFlags

func (stmts Statements) StringWithFlags(flags tree.FmtFlags) string

StringWithFlags returns the AST formatted as a string (with the given flags).

type TokenString

type TokenString struct {
	TokenID int32
	Str     string
}

TokenString is the unit value returned by Tokens.

func Tokens

func Tokens(sql string) (tokens []TokenString, ok bool)

Tokens decomposes the input into lexical tokens.