parser

README

Generated Documentation and Embedded Help Texts

Many parts of the parser package include special consideration for generation or other production of user-facing documentation. This includes interactive help messages, generated documentation of the set of available functions, or diagrams of the various expressions.

Generated documentation is produced and maintained at compile time, while the interactive, contextual help is returned at runtime.

We equip the generated parser with the ability to report contextual help in two circumstances:

• when the user explicitly requests help with the HELPTOKEN (current syntax: standalone "??")
• when the user makes a grammatical mistake (e.g. INSERT sometable INTO(x, y) ...)

We use the docgen tool to produce the generated documentation files that are then included in the broader (handwritten) published documentation.

Help texts embedded in the grammar

The help is embedded in the grammar using special markers in yacc comments, for example:

// %Help: HELPKEY - shortdescription
// %Category: SomeCat
// %Text: whatever until next %marker at start of line, or non-comment.
// %SeeAlso: whatever until next %marker at start of line, or non-comment.
// %End (optional)

The "HELPKEY" becomes the map key in the generated Go map.

These texts are extracted automatically by help.awk and converted into a Go data structure in help_messages.go.

Support in the parser

Primary mechanism - LALR error recovery

The primary mechanism is leveraging error recovery in LALR parsers using the special error token [1] [2]: when an unexpected token is encountered, the LALR parser will pop tokens on the stack until the prefix matches a grammar rule with the special "error" token (if any). If such a rule exists, its action is used to reduce and the erroneous tokens are discarded.

This mechanism is used both when the user makes a mistake, and when the user inserts the HELPTOKEN in the middle of a statement. When present in the middle of a statement, HELPTOKEN is considered an error and triggers the error recovery.

We use this for contextual help by providing error rules that generate a contextual help text during LALR error recovery.

For example:

backup_stmt:
  BACKUP targets TO string_or_placeholder opt_as_of_clause opt_incremental opt_with_options
  {
    $$.val = &Backup{Targets: $2.targetList(), To: $4.expr(), IncrementalFrom: $6.exprs(), AsOf: $5.asOfClause(), Options: $7.kvOptions()}
  }
| BACKUP error { return helpWith(sqllex, `BACKUP`) }

In this example, the grammar specifies that if the BACKUP keyword is followed by some input tokens such that the first (valid) grammar rule doesn't apply, the parser will "recover from the error" by backtracking up until the point it only sees BACKUP on the stack followed by non-parsable tokens, at which points it takes the error rule and executes its action.

The action is return helpWith(...). What this does is:

• halts parsing (the generated parser executes all actions in a big loop; a return interrupts this loop);
• makes the parser return with an error (the helpWith function returns non-zero);
• extends the parsing error message with a help text; this help text can subsequently be exploited in a client to display the help message in a friendly manner.

Code generation

Since the pattern "{ return helpWith(sqllex, ...) }" is common, we also implement a shorthand syntax based on comments, for example:

backup_stmt:
   ...
| BACKUP error // SHOW HELP: BACKUP

The special comment syntax "SHOW HELP: XXXX" is substituted by means of an auxiliary script (replace_help_rules.awk) into the form explained above.

Secondary mechanism - explicit help token

The mechanism described above works both when the user make a grammatical error and when they place the HELPTOKEN in the middle of a statement, rendering it invalid.

However for contextual help this is not sufficient: what happens if the user requests HELPTOKEN at a position in the grammar where everything before is a complete, valid SQL input?

For example: DELETE FROM foo ?

When encountering this input, the LALR parser will see DELETE FROM foo first, then reduce using the DELETE action because everything up to this point is a valid DELETE statement. When the HELPTOKEN is encountered, the statement has already been completed and the LALR parser doesn't 'know' any more that it was in the context of a DELETE statement.

If we try to place an error-based recovery rule at the top-level:

stmt:
  alter_stmt
| backup_stmt
| ...
| delete_stmt
| ...
| error { ??? }

This wouldn't work: the code inside the error action cannot "observe" the tokens observed so far and there would be no way to know whether the error should be about DELETE, or instead about ALTER, BACKUP, etc.

So in order to handle HELPTOKEN after a valid statement, we must place it in a rule where the context is still available, that is before the statement's grammar rule is reduced.

Where would that be? Suppose we had a simple statement rule:

somesimplestmt:
  SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE error { help ... }

We could extend with:

somesimplestmt:
  SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE DO SOMETHING HELPTOKEN { help ... }
| SIMPLE error { help ... }

(the alternative also works:

somesimplestmt:
  SIMPLE DO SOMETHING { $$ = reduce(...) }
| SIMPLE DO SOMETHING error { help ... }
| SIMPLE error { help ... }

)

That is all fine and dandy, but in SQL we have statements with many alternate forms, for example:

alter_rename_table_stmt:
  ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }

To add complementary handling of the help token at the end of valid statements we could, but would hate to, duplicate all the rules:

alter_rename_table_stmt:
  ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME TO qualified_name HELPTOKEN { help ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name HELPTOKEN { help ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name HELPTOKEN { help ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name HELPTOKEN { help ... }

This duplication is horrendous (not to mention hard to maintain), so instead we should attempt to factor the help token in a context where it is still known that we are dealing just with that statement.

The following works:

alter_rename_table_stmt:
  real_alter_rename_table_stmt { $$ = $1 }
| real_alter_rename_table_stmt HELPTOKEN { help ... }

real_alter_rename_table_stmt:
  ALTER TABLE relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME TO qualified_name { ... }
| ALTER TABLE relation_expr RENAME opt_column name TO name { ... }
| ALTER TABLE IF EXISTS relation_expr RENAME opt_column name TO name { ... }

Or does it? Without anything else, yacc complains with a "shift/reduce conflict". The reason is coming from the ambiguity: when the parsing stack contains everything sufficient to match a real_alter_rename_table_stmt, there is a choice between reducing the simple form alter_rename_table_stmt: real_alter_rename_table_stmt, or shifting into the more complex form alter_rename_table_stmt: real_alter_rename_table_stmt HELPTOKEN.

This is another form of the textbook situation when yacc is used to parse if-else statements in a programming language: the rule stmt: IF cond THEN body | IF cond THEN body ELSE body is ambiguous (and yields a shift/reduce conflict) for exactly the same reason.

The solution here is also straight out of a textbook: one simply informs yacc of the relative priority between the two candidate rules. In this case, when faced with a neutral choice, we encourage yacc to shift. The particular mechanism is to tell yacc that one rule has a higher priority than another.

It just so happens however that the yacc language only allows us to set relative priorities of tokens, not rules. And here we have a problem, of the two rules that need to be prioritized, only one has a token to work with (the one with HELPTOKEN). Which token should we prioritize for the other?

Conveniently yacc knows about this trouble and offers us an awkward, but working solution: we can tell it "use for this rule the same priority level as an existing token, even though the token is not part of the rule". The syntax for this is rule %prec TOKEN.

We can then use this as follows:

alter_rename_table_stmt:
  real_alter_rename_table_stmt           %prec LOWTOKEN { $$ = $1 }
| real_alter_rename_table_stmt HELPTOKEN %prec HIGHTOKEN { help ... }

We could create two new pseudo-tokens for this (called LOWTOKEN and HIGHTOKEN) however conveniently we can also reuse otherwise valid tokens that have known relative priorities. We settled in our case on VALUES (low priority) and UMINUS (high priority).

Code generation

With the latter mechanism presented above the pattern

rule:
  somerule           %prec VALUES
| somerule HELPTOKEN %prec UMINUS { help ...}`

becomes super common, so we automate it with the following special syntax:

rule:
  somerule // EXTEND WITH HELP: XXX

And the code replacement in replace_help_rules.awk expands this to the form above automatically.

Generated Documentation

Documentation of the SQL functions and operators is generated by the docgen utility, using make generate PKG=./docs/.... The markdown-formatted files are kept in docs/generated/sql and should be re-generated whenever the functions/operators they document change, and indeed if regenerating produces a diff, a CI failure is expected.

References

1. https://www.gnu.org/software/bison/manual/html_node/Error-Recovery.html
2. http://stackoverflow.com/questions/9796608/error-handling-in-yacc

Documentation

Overview

Package parser contains exposes a SQL parser for cockroach.

Index

• Constants
• Variables
• func GetTypeFromCastOrCollate(expr tree.Expr) (tree.ResolvableTypeReference, error)
• func GetTypeFromValidSQLSyntax(sql string) (tree.ResolvableTypeReference, error)
• func GetTypeReferenceFromName(typeName tree.Name) (tree.ResolvableTypeReference, error)
• func NakedIntTypeFromDefaultIntSize(defaultIntSize int32) *types.T
• func Parse(sql string) (statements.Statements, error)
• func ParseExpr(sql string) (tree.Expr, error)
• func ParseExprWithInt(sql string, nakedIntType *types.T) (tree.Expr, error)
• func ParseExprs(sql []string) (tree.Exprs, error)
• func ParseOne(sql string) (statements.Statement[tree.Statement], error)
• func ParseOneWithInt(sql string, nakedIntType *types.T) (statements.Statement[tree.Statement], error)
• func ParseQualifiedTableName(sql string) (*tree.TableName, error)
• func ParseTableName(sql string) (*tree.UnresolvedObjectName, error)
• func ParseTablePattern(sql string) (tree.TablePattern, error)
• func ParseWithInt(sql string, nakedIntType *types.T) (statements.Statements, error)
• func PopulateErrorDetails(tokID int32, lastTokStr string, lastTokPos int32, lastErr error, lIn string) error
• func RunShowSyntax(ctx context.Context, stmt string, ...)
• func SplitFirstStatement(sql string) (pos int, ok bool)
• type HelpMessage
  • func (h *HelpMessage) Format(w io.Writer)
  • func (h *HelpMessage) String() string
• type HelpMessageBody
• type Parser
  • func (p *Parser) Parse(sql string) (statements.Statements, error)
  • func (p *Parser) ParseWithInt(sql string, nakedIntType *types.T) (statements.Statements, error)
• type TokenString
  • func Tokens(sql string) (tokens []TokenString, ok bool)
  • func TokensIgnoreErrors(sql string) (tokens []TokenString)

Constants

const ABORT = 57363

const ABSOLUTE = 57364

const ACCESS = 57365

const ACTION = 57366

const ADD = 57367

const ADMIN = 57368

const AFTER = 57369

const AGGREGATE = 57370

const ALL = 57371

const ALTER = 57372

const ALWAYS = 57373

const ANALYSE = 57374

const ANALYZE = 57375

const AND = 57376

const AND_AND = 57377

const ANNOTATE_TYPE = 57379

const ANY = 57378

const ARRAY = 57380

const AS = 57381

const ASC = 57382

const ASENSITIVE = 57385

const ASYMMETRIC = 57386

const AS_JSON = 57383

const AS_LA = 58018

const AT = 57387

const ATOMIC = 57388

const ATTRIBUTE = 57389

const AT_AT = 57384

const AUTHORIZATION = 57390

const AUTOMATIC = 57391

const AVAILABILITY = 57392

const BACKUP = 57393

const BACKUPS = 57394

const BACKWARD = 57395

const BATCH = 57396

const BCONST = 57348

const BEFORE = 57397

const BEGIN = 57398

const BETWEEN = 57399

const BIGINT = 57400

const BIGSERIAL = 57401

const BINARY = 57402

const BIT = 57403

const BITCONST = 57349

const BOOLEAN = 57405

const BOTH = 57406

const BOX2D = 57407

const BUCKET_COUNT = 57404

const BUNDLE = 57408

const BY = 57409

const CACHE = 57410

const CALL = 57411

const CALLED = 57412

const CANCEL = 57413

const CANCELQUERY = 57414

const CAPABILITIES = 57415

const CAPABILITY = 57416

const CASCADE = 57417

const CASE = 57418

const CAST = 57419

const CBRT = 57420

const CHANGEFEED = 57421

const CHAR = 57422

const CHARACTER = 57423

const CHARACTERISTICS = 57424

const CHECK = 57425

const CHECK_FILES = 57426

const CLOSE = 57427

const CLUSTER = 57428

const CLUSTERS = 57429

const CLUSTER_ALL = 58026

const COALESCE = 57430

const COLLATE = 57431

const COLLATION = 57432

const COLUMN = 57433

const COLUMNS = 57434

const COMMENT = 57435

const COMMENTS = 57436

const COMMIT = 57437

const COMMITTED = 57438

const COMPACT = 57439

const COMPLETE = 57440

const COMPLETIONS = 57441

const CONCAT = 57442

const CONCURRENTLY = 57443

const CONFIGURATION = 57444

const CONFIGURATIONS = 57445

const CONFIGURE = 57446

const CONFLICT = 57447

const CONNECTION = 57448

const CONNECTIONS = 57449

const CONSTRAINT = 57450

const CONSTRAINTS = 57451

const CONTAINED_BY = 58028

const CONTAINS = 57452

const CONTROLCHANGEFEED = 57453

const CONTROLJOB = 57454

const CONVERSION = 57455

const CONVERT = 57456

const COPY = 57457

const COST = 57458

const COVERING = 57459

const CREATE = 57460

const CREATEDB = 57461

const CREATELOGIN = 57462

const CREATEROLE = 57463

const CROSS = 57464

const CSV = 57465

const CUBE = 57466

const CURRENT = 57467

const CURRENT_CATALOG = 57468

const CURRENT_DATE = 57469

const CURRENT_ROLE = 57471

const CURRENT_SCHEMA = 57470

const CURRENT_TIME = 57472

const CURRENT_TIMESTAMP = 57473

const CURRENT_USER = 57474

const CURSOR = 57475

const CYCLE = 57476

const DATA = 57477

const DATABASE = 57478

const DATABASES = 57479

const DATE = 57480

const DAY = 57481

const DEALLOCATE = 57490

const DEBUG_DUMP_METADATA_SST = 57485

const DEBUG_IDS = 57482

const DEBUG_PAUSE_ON = 57483

const DEC = 57484

const DECIMAL = 57486

const DECLARE = 57491

const DEFAULT = 57487

const DEFAULTS = 57488

const DEFERRABLE = 57492

const DEFERRED = 57493

const DEFINER = 57489

const DELETE = 57494

const DELIMITER = 57495

const DEPENDS = 57496

const DESC = 57497

const DESTINATION = 57498

const DETACHED = 57499

const DETAILS = 57500

const DISCARD = 57501

const DISTINCT = 57502

const DO = 57503

const DOMAIN = 57504

const DOT_DOT = 57355

const DOUBLE = 57505

const DROP = 57506

const ELSE = 57507

const ENCODING = 57508

const ENCRYPTED = 57509

const ENCRYPTION_INFO_DIR = 57510

const ENCRYPTION_PASSPHRASE = 57511

const END = 57512

const ENUM = 57513

const ENUMS = 57514

const ERROR = 57362

const ESCAPE = 57515

const EXCEPT = 57516

const EXCLUDE = 57517

const EXCLUDING = 57518

const EXECUTE = 57520

const EXECUTION = 57521

const EXISTS = 57519

const EXPERIMENTAL = 57522

const EXPERIMENTAL_AUDIT = 57525

const EXPERIMENTAL_FINGERPRINTS = 57523

const EXPERIMENTAL_RELOCATE = 57526

const EXPERIMENTAL_REPLICA = 57524

const EXPIRATION = 57527

const EXPLAIN = 57528

const EXPORT = 57529

const EXTENSION = 57530

const EXTERNAL = 57531

const EXTRACT = 57532

const EXTRACT_DURATION = 57533

const EXTREMES = 57534

const FAILURE = 57535

const FALSE = 57536

const FAMILY = 57537

const FCONST = 57351

const FETCH = 57538

const FETCHTEXT = 57540

const FETCHTEXT_PATH = 57542

const FETCHVAL = 57539

const FETCHVAL_PATH = 57541

const FILES = 57543

const FILTER = 57544

const FIRST = 57545

const FLOAT = 57546

const FLOAT4 = 57547

const FLOAT8 = 57548

const FLOORDIV = 57549

const FOLLOWING = 57550

const FOR = 57551

const FORCE = 57552

const FORCE_INDEX = 57553

const FORCE_NOT_NULL = 57554

const FORCE_NULL = 57555

const FORCE_QUOTE = 57556

const FORCE_ZIGZAG = 57557

const FOREIGN = 57558

const FORMAT = 57559

const FORWARD = 57560

const FREEZE = 57561

const FROM = 57562

const FULL = 57563

const FUNCTION = 57564

const FUNCTIONS = 57565

const GENERATED = 57566

const GENERATED_ALWAYS = 58019

const GENERATED_BY_DEFAULT = 58020

const GEOGRAPHY = 57567

const GEOMETRY = 57568

const GEOMETRYCOLLECTION = 57572

const GEOMETRYCOLLECTIONM = 57573

const GEOMETRYCOLLECTIONZ = 57574

const GEOMETRYCOLLECTIONZM = 57575

const GEOMETRYM = 57569

const GEOMETRYZ = 57570

const GEOMETRYZM = 57571

const GLOBAL = 57576

const GOAL = 57577

const GRANT = 57578

const GRANTEE = 57579

const GRANTS = 57580

const GREATER_EQUALS = 57357

const GREATEST = 57581

const GROUP = 57582

const GROUPING = 57583

const GROUPS = 57584

const HASH = 57586

const HAVING = 57585

const HEADER = 57587

const HELPTOKEN = 58032

const HIGH = 57588

const HISTOGRAM = 57589

const HOLD = 57590

const HOUR = 57591

const ICONST = 57350

const IDENT = 57346

const IDENTITY = 57592

const IF = 57593

const IFERROR = 57594

const IFNULL = 57595

const IGNORE_FOREIGN_KEYS = 57596

const ILIKE = 57597

const IMMEDIATE = 57598

const IMMUTABLE = 57599

const IMPORT = 57600

const IN = 57601

const INCLUDE = 57602

const INCLUDE_ALL_SECONDARY_TENANTS = 57604

const INCLUDE_ALL_VIRTUAL_CLUSTERS = 57605

const INCLUDING = 57603

const INCREMENT = 57606

const INCREMENTAL = 57607

const INCREMENTAL_LOCATION = 57608

const INDEX = 57612

const INDEXES = 57613

const INDEX_AFTER_ORDER_BY_BEFORE_AT = 57619

const INDEX_BEFORE_NAME_THEN_PAREN = 57618

const INDEX_BEFORE_PAREN = 57617

const INET = 57609

const INET_CONTAINED_BY_OR_EQUALS = 57610

const INET_CONTAINS_OR_EQUALS = 57611

const INHERITS = 57614

const INITIALLY = 57616

const INJECT = 57615

const INNER = 57620

const INOUT = 57621

const INPUT = 57622

const INSENSITIVE = 57623

const INSERT = 57624

const INT = 57625

const INTEGER = 57626

const INTERSECT = 57627

const INTERVAL = 57628

const INTERVAL_SIMPLE = 58030

const INTO = 57629

const INTO_DB = 57630

const INVERTED = 57631

const INVISIBLE = 57999

const INVOKER = 57632

const IS = 57633

const ISERROR = 57634

const ISNULL = 57635

const ISOLATION = 57636

const JOB = 57637

const JOBS = 57638

const JOIN = 57639

const JSON = 57640

const JSONB = 57641

const JSON_ALL_EXISTS = 57643

const JSON_SOME_EXISTS = 57642

const KEY = 57644

const KEYS = 57645

const KMS = 57646

const KV = 57647

const LABEL = 57648

const LANGUAGE = 57649

const LAST = 57650

const LATERAL = 57651

const LATEST = 57652

const LC_COLLATE = 57654

const LC_CTYPE = 57653

const LEADING = 57655

const LEAKPROOF = 57658

const LEASE = 57656

const LEAST = 57657

const LEFT = 57659

const LESS = 57660

const LESS_EQUALS = 57356

const LEVEL = 57661

const LIKE = 57662

const LIMIT = 57663

const LINESTRING = 57664

const LINESTRINGM = 57665

const LINESTRINGZ = 57666

const LINESTRINGZM = 57667

const LIST = 57668

const LOCAL = 57669

const LOCALITY = 57670

const LOCALTIME = 57671

const LOCALTIMESTAMP = 57672

const LOCKED = 57673

const LOGIN = 57674

const LOOKUP = 57675

const LOW = 57676

const LSHIFT = 57677

const MATCH = 57678

const MATERIALIZED = 57679

const MAXVALUE = 57682

const MERGE = 57680

const METHOD = 57683

const MINUTE = 57684

const MINVALUE = 57681

const MODIFYCLUSTERSETTING = 57685

const MODIFYSQLCLUSTERSETTING = 57686

const MONTH = 57687

const MOVE = 57688

const MULTILINESTRING = 57689

const MULTILINESTRINGM = 57690

const MULTILINESTRINGZ = 57691

const MULTILINESTRINGZM = 57692

const MULTIPOINT = 57693

const MULTIPOINTM = 57694

const MULTIPOINTZ = 57695

const MULTIPOINTZM = 57696

const MULTIPOLYGON = 57697

const MULTIPOLYGONM = 57698

const MULTIPOLYGONZ = 57699

const MULTIPOLYGONZM = 57700

const MaxInt = int(MaxUint >> 1)

const MaxUint = ^uint(0)

const NAME = 57702

const NAMES = 57703

const NAN = 57701

const NATURAL = 57704

const NEVER = 57705

const NEW_DB_NAME = 57706

const NEW_KMS = 57707

const NEXT = 57708

const NO = 57709

const NOCANCELQUERY = 57710

const NOCONTROLCHANGEFEED = 57711

const NOCONTROLJOB = 57712

const NOCREATEDB = 57713

const NOCREATELOGIN = 57714

const NOCREATEROLE = 57715

const NOLOGIN = 57716

const NOMODIFYCLUSTERSETTING = 57717

const NONE = 57723

const NONVOTERS = 57724

const NOREPLICATION = 57718

const NORMAL = 57725

const NOSQLLOGIN = 57719

const NOT = 57726

const NOTHING = 57727

const NOTHING_AFTER_RETURNING = 57728

const NOTNULL = 57729

const NOT_EQUALS = 57358

const NOT_LA = 58015

const NOT_REGIMATCH = 57361

const NOT_REGMATCH = 57359

const NOVIEWACTIVITY = 57730

const NOVIEWACTIVITYREDACTED = 57731

const NOVIEWCLUSTERSETTING = 57732

const NOWAIT = 57733

const NO_FULL_SCAN = 57722

const NO_INDEX_JOIN = 57720

const NO_ZIGZAG_JOIN = 57721

const NULL = 57734

const NULLIF = 57735

const NULLS = 57736

const NULLS_LA = 58016

const NUMERIC = 57737

const OF = 57738

const OFF = 57739

const OFFSET = 57740

const OID = 57741

const OIDS = 57742

const OIDVECTOR = 57743

const OLD_KMS = 57744

const ON = 57745

const ONLY = 57746

const ON_LA = 58024

const OPERATOR = 57761

const OPT = 57747

const OPTION = 57748

const OPTIONS = 57749

const OR = 57750

const ORDER = 57751

const ORDINALITY = 57752

const OTHERS = 57753

const OUT = 57754

const OUTER = 57755

const OVER = 57756

const OVERLAPS = 57757

const OVERLAY = 57758

const OWNED = 57759

const OWNER = 57760

const PARALLEL = 57762

const PARENT = 57763

const PARTIAL = 57764

const PARTITION = 57765

const PARTITIONS = 57766

const PASSWORD = 57767

const PAUSE = 57768

const PAUSED = 57769

const PHYSICAL = 57770

const PLACEHOLDER = 57352

const PLACEMENT = 57771

const PLACING = 57772

const PLAN = 57773

const PLANS = 57774

const POINT = 57775

const POINTM = 57776

const POINTZ = 57777

const POINTZM = 57778

const POLYGON = 57779

const POLYGONM = 57780

const POLYGONZ = 57781

const POLYGONZM = 57782

const POSITION = 57783

const POSTFIXOP = 58029

const PRECEDING = 57784

const PRECISION = 57785

const PREPARE = 57786

const PRESERVE = 57787

const PRIMARY = 57788

const PRIOR = 57789

const PRIORITY = 57790

const PRIVILEGES = 57791

const PROCEDURAL = 57792

const PROCEDURE = 57793

const PROCEDURES = 57794

const PUBLIC = 57795

const PUBLICATION = 57796

const QUERIES = 57797

const QUERY = 57798

const QUOTE = 57799

const RANGE = 57800

const RANGES = 57801

const READ = 57802

const REAL = 57803

const REASON = 57804

const REASSIGN = 57805

const RECURRING = 57807

const RECURSIVE = 57806

const REDACT = 57808

const REF = 57809

const REFERENCES = 57810

const REFRESH = 57811

const REGCLASS = 57812

const REGIMATCH = 57360

const REGION = 57813

const REGIONAL = 57814

const REGIONS = 57815

const REGNAMESPACE = 57816

const REGPROC = 57817

const REGPROCEDURE = 57818

const REGROLE = 57819

const REGTYPE = 57820

const REINDEX = 57821

const RELATIVE = 57822

const RELEASE = 57830

const RELOCATE = 57823

const REMOVE_PATH = 57824

const REMOVE_REGIONS = 57825

const RENAME = 57826

const REPEATABLE = 57827

const REPLACE = 57828

const REPLICATION = 57829

const RESET = 57831

const RESET_ALL = 58021

const RESTART = 57832

const RESTORE = 57833

const RESTRICT = 57834

const RESTRICTED = 57835

const RESUME = 57836

const RETENTION = 57837

const RETRY = 57841

const RETURN = 57839

const RETURNING = 57838

const RETURNS = 57840

const REVISION_HISTORY = 57842

const REVOKE = 57843

const RIGHT = 57844

const ROLE = 57845

const ROLES = 57846

const ROLE_ALL = 58022

const ROLLBACK = 57847

const ROLLUP = 57848

const ROUTINES = 57849

const ROW = 57850

const ROWS = 57851

const RSHIFT = 57852

const RULE = 57853

const RUNNING = 57854

const SAVEPOINT = 57855

const SCANS = 57856

const SCATTER = 57857

const SCHEDULE = 57858

const SCHEDULES = 57859

const SCHEMA = 57861

const SCHEMAS = 57863

const SCHEMA_ONLY = 57862

const SCONST = 57347

const SCROLL = 57860

const SCRUB = 57864

const SEARCH = 57865

const SECOND = 57866

const SECONDARY = 57867

const SECURITY = 57868

const SELECT = 57869

const SEQUENCE = 57870

const SEQUENCES = 57871

const SERIALIZABLE = 57872

const SERVER = 57873

const SERVICE = 57874

const SESSION = 57875

const SESSIONS = 57876

const SESSION_USER = 57877

const SET = 57878

const SETOF = 57879

const SETS = 57880

const SETTING = 57881

const SETTINGS = 57882

const SET_TRACING = 58027

const SHARE = 57883

const SHARED = 57884

const SHOW = 57885

const SIMILAR = 57886

const SIMPLE = 57887

const SIZE = 57888

const SKIP = 57889

const SKIP_LOCALITIES_CHECK = 57890

const SKIP_MISSING_FOREIGN_KEYS = 57891

const SKIP_MISSING_SEQUENCES = 57892

const SKIP_MISSING_SEQUENCE_OWNERS = 57893

const SKIP_MISSING_UDFS = 57895

const SKIP_MISSING_VIEWS = 57894

const SMALLINT = 57896

const SMALLSERIAL = 57897

const SNAPSHOT = 57898

const SOME = 57899

const SPLIT = 57900

const SQL = 57901

const SQLLOGIN = 57902

const SQRT = 57926

const STABLE = 57903

const START = 57904

const STATE = 57905

const STATEMENTS = 57928

const STATISTICS = 57906

const STATUS = 57907

const STDIN = 57908

const STDOUT = 57909

const STOP = 57910

const STORAGE = 57914

const STORE = 57915

const STORED = 57916

const STORING = 57917

const STREAM = 57911

const STRICT = 57912

const STRING = 57913

const SUBSCRIPTION = 57927

const SUBSTRING = 57918

const SUPER = 57919

const SUPPORT = 57920

const SURVIVAL = 57922

const SURVIVE = 57921

const SYMMETRIC = 57923

const SYNTAX = 57924

const SYSTEM = 57925

const TABLE = 57929

const TABLES = 57930

const TABLESPACE = 57931

const TEMP = 57932

const TEMPLATE = 57933

const TEMPORARY = 57934

const TENANT = 57935

const TENANTS = 57937

const TENANT_ALL = 58025

const TENANT_NAME = 57936

const TESTING_RELOCATE = 57938

const TEXT = 57939

const THEN = 57940

const THROTTLING = 57947

const TIES = 57941

const TIME = 57942

const TIMESTAMP = 57944

const TIMESTAMPTZ = 57945

const TIMETZ = 57943

const TO = 57946

const TRACE = 57949

const TRACING = 57962

const TRAILING = 57948

const TRANSACTION = 57950

const TRANSACTIONS = 57951

const TRANSFER = 57952

const TRANSFORM = 57953

const TREAT = 57954

const TRIGGER = 57955

const TRIM = 57956

const TRUE = 57957

const TRUNCATE = 57958

const TRUSTED = 57959

const TYPE = 57960

const TYPEANNOTATE = 57354

const TYPECAST = 57353

const TYPES = 57961

const UMINUS = 58031

const UNBOUNDED = 57963

const UNCOMMITTED = 57964

const UNION = 57965

const UNIQUE = 57966

const UNKNOWN = 57967

const UNLISTEN = 57968

const UNLOGGED = 57969

const UNSAFE_RESTORE_INCOMPATIBLE_VERSION = 57970

const UNSET = 57975

const UNSPLIT = 57971

const UNTIL = 57976

const UPDATE = 57972

const UPDATES_CLUSTER_MONITORING_METRICS = 57973

const UPSERT = 57974

const USE = 57977

const USER = 57978

const USERS = 57979

const USER_ALL = 58023

const USING = 57980

const UUID = 57981

const VALID = 57982

const VALIDATE = 57983

const VALUE = 57984

const VALUES = 57985

const VARBIT = 57986

const VARCHAR = 57987

const VARIADIC = 57988

const VARYING = 57991

const VERIFY_BACKUP_TABLE_DATA = 57989

const VIEW = 57990

const VIEWACTIVITY = 57992

const VIEWACTIVITYREDACTED = 57993

const VIEWCLUSTERMETADATA = 57995

const VIEWCLUSTERSETTING = 57996

const VIEWDEBUG = 57994

const VIRTUAL = 57997

const VIRTUAL_CLUSTER = 58004

const VIRTUAL_CLUSTER_NAME = 58003

const VISIBILITY = 58000

const VISIBLE = 57998

const VOLATILE = 58001

const VOTERS = 58002

const WHEN = 58005

const WHERE = 58006

const WINDOW = 58007

const WITH = 58008

const WITHIN = 58009

const WITHOUT = 58010

const WITH_LA = 58017

const WORK = 58011

const WRITE = 58012

const YEAR = 58013

const ZONE = 58014

Variables

var AllHelp = func(h map[string]HelpMessageBody) string {

	cmds := make(map[string][]string)
	for c, details := range h {
		if details.Category == "" {
			continue
		}
		cmds[details.Category] = append(cmds[details.Category], c)
	}

	// Ensure the result is deterministic.
	var categories []string
	for c, l := range cmds {
		categories = append(categories, c)
		sort.Strings(l)
	}
	sort.Strings(categories)

	// Compile the final help index.
	var buf bytes.Buffer
	w := tabwriter.NewWriter(&buf, 0, 0, 1, ' ', 0)
	for _, cat := range categories {
		fmt.Fprintf(w, "%s:\n", cases.Title(language.English, cases.NoLower).String(cat))
		for _, item := range cmds[cat] {
			fmt.Fprintf(w, "\t\t%s\t%s\n", item, h[item].ShortDescription)
		}
		fmt.Fprintln(w)
	}
	_ = w.Flush()
	return buf.String()
}(helpMessages)

AllHelp contains an overview of all statements with help messages. For example, displayed in the CLI shell with \h without additional parameters.

var HelpMessages = func(h map[string]HelpMessageBody) map[string]HelpMessageBody {
	appendSeeAlso := func(newItem, prevItems string) string {

		if prevItems != "" {
			return newItem + "\n  " + prevItems
		}
		return newItem
	}
	reformatSeeAlso := func(seeAlso string) string {
		return strings.Replace(
			strings.Replace(seeAlso, ", ", "\n  ", -1),
			"WEBDOCS", docs.URLBase, -1)
	}
	srcMsg := h["<SOURCE>"]
	srcMsg.SeeAlso = reformatSeeAlso(strings.TrimSpace(srcMsg.SeeAlso))
	selectMsg := h["<SELECTCLAUSE>"]
	selectMsg.SeeAlso = reformatSeeAlso(strings.TrimSpace(selectMsg.SeeAlso))
	for k, m := range h {
		m = h[k]
		m.ShortDescription = strings.TrimSpace(m.ShortDescription)
		m.Text = strings.TrimSpace(m.Text)
		m.SeeAlso = strings.TrimSpace(m.SeeAlso)

		if strings.Contains(m.Text, "<source>") && k != "<SOURCE>" {
			m.Text = strings.TrimSpace(m.Text) + "\n\n" + strings.TrimSpace(srcMsg.Text)
			m.SeeAlso = appendSeeAlso(srcMsg.SeeAlso, m.SeeAlso)
		}

		if strings.Contains(m.Text, "<selectclause>") && k != "<SELECTCLAUSE>" {
			m.Text = strings.TrimSpace(m.Text) + "\n\n" + strings.TrimSpace(selectMsg.Text)
			m.SeeAlso = appendSeeAlso(selectMsg.SeeAlso, m.SeeAlso)
		}

		if strings.Contains(m.Text, "<tablename>") {
			m.SeeAlso = appendSeeAlso("SHOW TABLES", m.SeeAlso)
		}
		m.SeeAlso = reformatSeeAlso(m.SeeAlso)
		h[k] = m
	}
	return h
}(helpMessages)

HelpMessages is the registry of all help messages, keyed by the top-level statement that they document. The key is intended for use via the \h client-side command.

Functions

func GetTypeFromCastOrCollate

func GetTypeFromCastOrCollate(expr tree.Expr) (tree.ResolvableTypeReference, error)

GetTypeFromCastOrCollate returns the type of the given tree.Expr. The method assumes that the expression is either tree.CastExpr or tree.CollateExpr (which wraps the tree.CastExpr).

func GetTypeFromValidSQLSyntax

func GetTypeFromValidSQLSyntax(sql string) (tree.ResolvableTypeReference, error)

GetTypeFromValidSQLSyntax retrieves a type from its SQL syntax. The caller is responsible for guaranteeing that the type expression is valid SQL (or handling the resulting error). This includes verifying that complex identifiers are enclosed in double quotes, etc.

func GetTypeReferenceFromName

func GetTypeReferenceFromName(typeName tree.Name) (tree.ResolvableTypeReference, error)

GetTypeReferenceFromName turns a type name into a type reference. This supports only “simple” (single-identifier) references to built-in types, when the identifer has already been parsed away from the input SQL syntax.

func NakedIntTypeFromDefaultIntSize

func NakedIntTypeFromDefaultIntSize(defaultIntSize int32) *types.T

NakedIntTypeFromDefaultIntSize given the size in bits or bytes (preferred) of how a "naked" INT type should be parsed returns the corresponding integer type.

func Parse

func Parse(sql string) (statements.Statements, error)

Parse parses a sql statement string and returns a list of Statements.

func ParseExpr

func ParseExpr(sql string) (tree.Expr, error)

ParseExpr parses a SQL scalar expression. The caller is responsible for ensuring that the input is, in fact, a valid SQL scalar expression — the results are undefined if the string contains invalid SQL syntax.

func ParseExprWithInt

func ParseExprWithInt(sql string, nakedIntType *types.T) (tree.Expr, error)

ParseExprWithInt parses a SQL scalar expression, using the given type when INT is used as type name in the SQL syntax. The caller is responsible for ensuring that the input is, in fact, a valid SQL scalar expression — the results are undefined if the string contains invalid SQL syntax.

func ParseExprs

func ParseExprs(sql []string) (tree.Exprs, error)

ParseExprs parses a comma-delimited sequence of SQL scalar expressions. The caller is responsible for ensuring that the input is, in fact, a comma-delimited sequence of SQL scalar expressions — the results are undefined if the string contains invalid SQL syntax.

func ParseOne

func ParseOne(sql string) (statements.Statement[tree.Statement], error)

ParseOne parses a sql statement string, ensuring that it contains only a single statement, and returns that Statement. ParseOne will always interpret the INT and SERIAL types as 64-bit types, since this is used in various internal-execution paths where we might receive bits of SQL from other nodes. In general,earwe expect that all user-generated SQL has been run through the ParseWithInt() function.

func ParseOneWithInt

func ParseOneWithInt(
	sql string, nakedIntType *types.T,
) (statements.Statement[tree.Statement], error)

ParseOneWithInt is similar to ParseOn but interprets the INT and SERIAL types as the provided integer type.

func ParseQualifiedTableName

func ParseQualifiedTableName(sql string) (*tree.TableName, error)

ParseQualifiedTableName parses a possibly qualified table name. The table name must contain one or more name parts, using the full input SQL syntax: each name part containing special characters, or non-lowercase characters, must be enclosed in double quote. The name may not be an invalid table name (the caller is responsible for guaranteeing that only valid table names are provided as input).

func ParseTableName

func ParseTableName(sql string) (*tree.UnresolvedObjectName, error)

ParseTableName parses a table name. The table name must contain one or more name parts, using the full input SQL syntax: each name part containing special characters, or non-lowercase characters, must be enclosed in double quote. The name may not be an invalid table name (the caller is responsible for guaranteeing that only valid table names are provided as input).

func ParseTablePattern

func ParseTablePattern(sql string) (tree.TablePattern, error)

ParseTablePattern parses a table pattern. The table name must contain one or more name parts, using the full input SQL syntax: each name part containing special characters, or non-lowercase characters, must be enclosed in double quote. The name may not be an invalid table name (the caller is responsible for guaranteeing that only valid table names are provided as input). The last part may be '*' to denote a wildcard.

func ParseWithInt

func ParseWithInt(sql string, nakedIntType *types.T) (statements.Statements, error)

ParseWithInt parses a sql statement string and returns a list of Statements. The INT token will result in the specified TInt type.

func PopulateErrorDetails

func PopulateErrorDetails(
	tokID int32, lastTokStr string, lastTokPos int32, lastErr error, lIn string,
) error

PopulateErrorDetails properly wraps the "last error" field in the lexer.

func RunShowSyntax

func RunShowSyntax(
	ctx context.Context,
	stmt string,
	report func(ctx context.Context, field, msg string),
	reportErr func(ctx context.Context, err error),
)

RunShowSyntax analyzes the syntax and reports its structure as data for the client. Even an error is reported as data.

Since errors won't propagate to the client as an error, but as a result, the usual code path to capture and record errors will not be triggered. Instead, the caller can pass a reportErr closure to capture errors instead. May be nil.

func SplitFirstStatement

func SplitFirstStatement(sql string) (pos int, ok bool)

SplitFirstStatement returns the length of the prefix of the string up to and including the first semicolon that separates statements. If there is no including the first semicolon that separates statements. If there is no semicolon, returns ok=false.

Types

type HelpMessage

type HelpMessage struct {
	// Command is set if the message is about a statement.
	Command string
	// Function is set if the message is about a built-in function.
	Function string

	// HelpMessageBody contains the details of the message.
	HelpMessageBody
}

HelpMessage describes a contextual help message.

func (*HelpMessage) Format

func (h *HelpMessage) Format(w io.Writer)

Format prints out details about the message onto the specified output stream.

func (*HelpMessage) String

func (h *HelpMessage) String() string

String implements the fmt.String interface.

type HelpMessageBody

type HelpMessageBody struct {
	Category         string
	ShortDescription string
	Text             string
	SeeAlso          string
}

HelpMessageBody defines the body of a help text. The messages are structured to facilitate future help navigation functionality.

type Parser

type Parser struct {
	// contains filtered or unexported fields
}

Parser wraps a scanner, parser and other utilities present in the parser package.

func (*Parser) Parse

func (p *Parser) Parse(sql string) (statements.Statements, error)

Parse parses the sql and returns a list of statements.

func (*Parser) ParseWithInt

func (p *Parser) ParseWithInt(sql string, nakedIntType *types.T) (statements.Statements, error)

ParseWithInt parses a sql statement string and returns a list of Statements. The INT token will result in the specified TInt type.

type TokenString

type TokenString struct {
	TokenID int32
	Str     string
}

TokenString is the unit value returned by Tokens.

func Tokens

func Tokens(sql string) (tokens []TokenString, ok bool)

Tokens decomposes the input into lexical tokens.

func TokensIgnoreErrors

func TokensIgnoreErrors(sql string) (tokens []TokenString)

TokensIgnoreErrors decomposes the input into lexical tokens and ignores errors.