Documentation ¶
Overview ¶
Package syntax provides a Starlark parser and abstract syntax tree.
Index ¶
- func Quote(s string, b bool) string
- func Walk(n Node, f func(Node) bool)
- type AssignStmt
- type BinaryExpr
- type BranchStmt
- type CallExpr
- type Comment
- type Comments
- type Comprehension
- type CondExpr
- type DefStmt
- type DictEntry
- type DictExpr
- type DotExpr
- type Error
- type Expr
- type ExprStmt
- type File
- type FileOptions
- func (opts *FileOptions) Parse(filename string, src interface{}, mode Mode) (f *File, err error)
- func (opts *FileOptions) ParseCompoundStmt(filename string, readline func() ([]byte, error)) (f *File, err error)
- func (opts *FileOptions) ParseExpr(filename string, src interface{}, mode Mode) (expr Expr, err error)
- type FilePortion
- type ForClause
- type ForStmt
- type Ident
- type IfClause
- type IfStmt
- type IndexExpr
- type LambdaExpr
- type ListExpr
- type Literal
- type LoadStmt
- type Mode
- type Node
- type ParenExpr
- type Position
- type ReturnStmt
- type SliceExpr
- type Stmt
- type Token
- type TupleExpr
- type UnaryExpr
- type WhileStmt
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Walk ¶
Walk traverses a syntax tree in depth-first order. It starts by calling f(n); n must not be nil. If f returns true, Walk calls itself recursively for each non-nil child of n. Walk then calls f(nil).
Example ¶
ExampleWalk demonstrates the use of Walk to enumerate the identifiers in a Starlark source file containing a nonsense program with varied grammar.
package main import ( "fmt" "log" "strings" "go.starlark.net/syntax" ) func main() { const src = ` load("library", "a") def b(c, *, d=e): f += {g: h} i = -(j) return k.l[m + n] for o in [p for q, r in s if t]: u(lambda: v, w[x:y:z]) ` f, err := syntax.Parse("hello.star", src, 0) if err != nil { log.Fatal(err) } var idents []string syntax.Walk(f, func(n syntax.Node) bool { if id, ok := n.(*syntax.Ident); ok { idents = append(idents, id.Name) } return true }) fmt.Println(strings.Join(idents, " ")) // The identifier 'a' appears in both LoadStmt.From[0] and LoadStmt.To[0]. }
Output: a a b c d e f g h i j k l m n o p q r s t u v w x y z
Types ¶
type AssignStmt ¶
type AssignStmt struct { OpPos Position Op Token // = EQ | {PLUS,MINUS,STAR,PERCENT}_EQ LHS Expr RHS Expr // contains filtered or unexported fields }
An AssignStmt represents an assignment:
x = 0 x, y = y, x x += 1
func (*AssignStmt) AllocComments ¶
func (cr *AssignStmt) AllocComments()
AllocComments enables comments to be associated with a syntax node.
func (AssignStmt) Comments ¶
func (cr AssignStmt) Comments() *Comments
Comments returns the comments associated with a syntax node, or nil if AllocComments has not yet been called.
func (*AssignStmt) Span ¶
func (x *AssignStmt) Span() (start, end Position)
type BinaryExpr ¶
type BinaryExpr struct { X Expr OpPos Position Op Token Y Expr // contains filtered or unexported fields }
A BinaryExpr represents a binary expression: X Op Y.
As a special case, BinaryExpr{Op:EQ} may also represent a named argument in a call f(k=v) or a named parameter in a function declaration def f(param=default).
func (*BinaryExpr) AllocComments ¶
func (cr *BinaryExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
func (BinaryExpr) Comments ¶
func (cr BinaryExpr) Comments() *Comments
Comments returns the comments associated with a syntax node, or nil if AllocComments has not yet been called.
func (*BinaryExpr) Span ¶
func (x *BinaryExpr) Span() (start, end Position)
type BranchStmt ¶
type BranchStmt struct { Token Token // = BREAK | CONTINUE | PASS TokenPos Position // contains filtered or unexported fields }
A BranchStmt changes the flow of control: break, continue, pass.
func (*BranchStmt) AllocComments ¶
func (cr *BranchStmt) AllocComments()
AllocComments enables comments to be associated with a syntax node.
func (BranchStmt) Comments ¶
func (cr BranchStmt) Comments() *Comments
Comments returns the comments associated with a syntax node, or nil if AllocComments has not yet been called.
func (*BranchStmt) Span ¶
func (x *BranchStmt) Span() (start, end Position)
type CallExpr ¶
type CallExpr struct { Fn Expr Lparen Position Args []Expr // arg = expr | ident=expr | *expr | **expr Rparen Position // contains filtered or unexported fields }
A CallExpr represents a function call expression: Fn(Args).
func (*CallExpr) AllocComments ¶
func (cr *CallExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type Comments ¶
type Comments struct { Before []Comment // whole-line comments before this expression Suffix []Comment // end-of-line comments after this expression (up to 1) // For top-level expressions only, After lists whole-line // comments following the expression. After []Comment }
Comments collects the comments associated with an expression.
type Comprehension ¶
type Comprehension struct { Curly bool // {x:y for ...} or {x for ...}, not [x for ...] Lbrack Position Body Expr Clauses []Node // = *ForClause | *IfClause Rbrack Position // contains filtered or unexported fields }
A Comprehension represents a list or dict comprehension: [Body for ... if ...] or {Body for ... if ...}
func (*Comprehension) AllocComments ¶
func (cr *Comprehension) AllocComments()
AllocComments enables comments to be associated with a syntax node.
func (Comprehension) Comments ¶
func (cr Comprehension) Comments() *Comments
Comments returns the comments associated with a syntax node, or nil if AllocComments has not yet been called.
func (*Comprehension) Span ¶
func (x *Comprehension) Span() (start, end Position)
type CondExpr ¶
type CondExpr struct { If Position Cond Expr True Expr ElsePos Position False Expr // contains filtered or unexported fields }
CondExpr represents the conditional: X if COND else ELSE.
func (*CondExpr) AllocComments ¶
func (cr *CondExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type DefStmt ¶
type DefStmt struct { Def Position Name *Ident Lparen Position Params []Expr // param = ident | ident=expr | * | *ident | **ident Rparen Position Body []Stmt Function interface{} // a *resolve.Function, set by resolver // contains filtered or unexported fields }
A DefStmt represents a function definition.
func (*DefStmt) AllocComments ¶
func (cr *DefStmt) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type DictEntry ¶
type DictEntry struct { Key Expr Colon Position Value Expr // contains filtered or unexported fields }
A DictEntry represents a dictionary entry: Key: Value. Used only within a DictExpr.
func (*DictEntry) AllocComments ¶
func (cr *DictEntry) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type DictExpr ¶
type DictExpr struct { Lbrace Position List []Expr // all *DictEntrys Rbrace Position // contains filtered or unexported fields }
A DictExpr represents a dictionary literal: { List }.
func (*DictExpr) AllocComments ¶
func (cr *DictExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type DotExpr ¶
type DotExpr struct { X Expr Dot Position NamePos Position Name *Ident // contains filtered or unexported fields }
A DotExpr represents a field or method selector: X.Name.
func (*DotExpr) AllocComments ¶
func (cr *DotExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type Expr ¶
type Expr interface { Node // contains filtered or unexported methods }
An Expr is a Starlark expression.
type ExprStmt ¶
type ExprStmt struct { X Expr // contains filtered or unexported fields }
An ExprStmt is an expression evaluated for side effects.
func (*ExprStmt) AllocComments ¶
func (cr *ExprStmt) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type File ¶
type File struct { Path string Stmts []Stmt Module interface{} // a *resolve.Module, set by resolver Options *FileOptions // contains filtered or unexported fields }
A File represents a Starlark file.
func ParseCompoundStmt
deprecated
func (*File) AllocComments ¶
func (cr *File) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type FileOptions ¶
type FileOptions struct { // resolver Set bool // allow references to the 'set' built-in function While bool // allow 'while' statements TopLevelControl bool // allow if/for/while statements at top-level GlobalReassign bool // allow reassignment to top-level names LoadBindsGlobally bool // load creates global not file-local bindings (deprecated) // compiler Recursion bool // disable recursion check for functions in this file }
FileOptions specifies various per-file options that affect static aspects of an individual file such as parsing, name resolution, and code generation. (Options that affect global dynamics are typically controlled through go.starlark.net/starlark.Thread.)
The zero value of FileOptions is the default behavior.
Many functions in this package come in two versions: the legacy standalone function (such as Parse) uses LegacyFileOptions, whereas the more recent method (such as FileOptions.Parse) honors the provided options. The second form is preferred. In other packages, the modern version is a standalone function with a leading FileOptions parameter and the name suffix "Options", such as go.starlark.net/starlark.ExecFileOptions.
func LegacyFileOptions ¶
func LegacyFileOptions() *FileOptions
LegacyFileOptions returns a new FileOptions containing the current values of the resolver package's legacy global variables such as [resolve.AllowRecursion], etc. These variables may be associated with command-line flags.
func (*FileOptions) Parse ¶
func (opts *FileOptions) Parse(filename string, src interface{}, mode Mode) (f *File, err error)
Parse parses the input data and returns the corresponding parse tree.
If src != nil, Parse parses the source from src and the filename is only used when recording position information. The type of the argument for the src parameter must be string, []byte, io.Reader, or FilePortion. If src == nil, Parse parses the file specified by filename.
func (*FileOptions) ParseCompoundStmt ¶
func (opts *FileOptions) ParseCompoundStmt(filename string, readline func() ([]byte, error)) (f *File, err error)
ParseCompoundStmt parses a single compound statement: a blank line, a def, for, while, or if statement, or a semicolon-separated list of simple statements followed by a newline. These are the units on which the REPL operates. ParseCompoundStmt does not consume any following input. The parser calls the readline function each time it needs a new line of input.
type FilePortion ¶
A FilePortion describes the content of a portion of a file. Callers may provide a FilePortion for the src argument of Parse when the desired initial line and column numbers are not (1, 1), such as when an expression is parsed from within larger file.
type ForClause ¶
type ForClause struct { For Position Vars Expr // name, or tuple of names In Position X Expr // contains filtered or unexported fields }
A ForClause represents a for clause in a list comprehension: for Vars in X.
func (*ForClause) AllocComments ¶
func (cr *ForClause) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type ForStmt ¶
type ForStmt struct { For Position Vars Expr // name, or tuple of names X Expr Body []Stmt // contains filtered or unexported fields }
A ForStmt represents a loop: for Vars in X: Body.
func (*ForStmt) AllocComments ¶
func (cr *ForStmt) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type Ident ¶
type Ident struct { NamePos Position Name string Binding interface{} // a *resolver.Binding, set by resolver // contains filtered or unexported fields }
An Ident represents an identifier.
func (*Ident) AllocComments ¶
func (cr *Ident) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type IfClause ¶
An IfClause represents an if clause in a list comprehension: if Cond.
func (*IfClause) AllocComments ¶
func (cr *IfClause) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type IfStmt ¶
type IfStmt struct { If Position // IF or ELIF Cond Expr True []Stmt ElsePos Position // ELSE or ELIF False []Stmt // optional // contains filtered or unexported fields }
An IfStmt is a conditional: If Cond: True; else: False. 'elseif' is desugared into a chain of IfStmts.
func (*IfStmt) AllocComments ¶
func (cr *IfStmt) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type IndexExpr ¶
type IndexExpr struct { X Expr Lbrack Position Y Expr Rbrack Position // contains filtered or unexported fields }
An IndexExpr represents an index expression: X[Y].
func (*IndexExpr) AllocComments ¶
func (cr *IndexExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type LambdaExpr ¶
type LambdaExpr struct { Lambda Position Params []Expr // param = ident | ident=expr | * | *ident | **ident Body Expr Function interface{} // a *resolve.Function, set by resolver // contains filtered or unexported fields }
A LambdaExpr represents an inline function abstraction.
func (*LambdaExpr) AllocComments ¶
func (cr *LambdaExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
func (LambdaExpr) Comments ¶
func (cr LambdaExpr) Comments() *Comments
Comments returns the comments associated with a syntax node, or nil if AllocComments has not yet been called.
func (*LambdaExpr) Span ¶
func (x *LambdaExpr) Span() (start, end Position)
type ListExpr ¶
type ListExpr struct { Lbrack Position List []Expr Rbrack Position // contains filtered or unexported fields }
A ListExpr represents a list literal: [ List ].
func (*ListExpr) AllocComments ¶
func (cr *ListExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type Literal ¶
type Literal struct { Token Token // = STRING | BYTES | INT | FLOAT TokenPos Position Raw string // uninterpreted text Value interface{} // = string | int64 | *big.Int | float64 // contains filtered or unexported fields }
A Literal represents a literal string or number.
func (*Literal) AllocComments ¶
func (cr *Literal) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type LoadStmt ¶
type LoadStmt struct { Load Position Module *Literal // a string From []*Ident // name defined in loading module To []*Ident // name in loaded module Rparen Position // contains filtered or unexported fields }
A LoadStmt loads another module and binds names from it: load(Module, "x", y="foo").
The AST is slightly unfaithful to the concrete syntax here because Starlark's load statement, so that it can be implemented in Python, binds some names (like y above) with an identifier and some (like x) without. For consistency we create fake identifiers for all the strings.
func (*LoadStmt) AllocComments ¶
func (cr *LoadStmt) AllocComments()
AllocComments enables comments to be associated with a syntax node.
func (LoadStmt) Comments ¶
func (cr LoadStmt) Comments() *Comments
Comments returns the comments associated with a syntax node, or nil if AllocComments has not yet been called.
func (*LoadStmt) ModuleName ¶
ModuleName returns the name of the module loaded by this statement.
type Mode ¶
type Mode uint
A Mode value is a set of flags (or 0) that controls optional parser functionality.
type Node ¶
type Node interface { // Span returns the start and end position of the expression. Span() (start, end Position) // Comments returns the comments associated with this node. // It returns nil if RetainComments was not specified during parsing, // or if AllocComments was not called. Comments() *Comments // AllocComments allocates a new Comments node if there was none. // This makes possible to add new comments using Comments() method. AllocComments() }
A Node is a node in a Starlark syntax tree.
type ParenExpr ¶
type ParenExpr struct { Lparen Position X Expr Rparen Position // contains filtered or unexported fields }
A ParenExpr represents a parenthesized expression: (X).
func (*ParenExpr) AllocComments ¶
func (cr *ParenExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type Position ¶
type Position struct { Line int32 // 1-based line number; 0 if line unknown Col int32 // 1-based column (rune) number; 0 if column unknown // contains filtered or unexported fields }
A Position describes the location of a rune of input.
func MakePosition ¶
MakePosition returns position with the specified components.
type ReturnStmt ¶
type ReturnStmt struct { Return Position Result Expr // may be nil // contains filtered or unexported fields }
A ReturnStmt returns from a function.
func (*ReturnStmt) AllocComments ¶
func (cr *ReturnStmt) AllocComments()
AllocComments enables comments to be associated with a syntax node.
func (ReturnStmt) Comments ¶
func (cr ReturnStmt) Comments() *Comments
Comments returns the comments associated with a syntax node, or nil if AllocComments has not yet been called.
func (*ReturnStmt) Span ¶
func (x *ReturnStmt) Span() (start, end Position)
type SliceExpr ¶
type SliceExpr struct { X Expr Lbrack Position Lo, Hi, Step Expr // all optional Rbrack Position // contains filtered or unexported fields }
A SliceExpr represents a slice or substring expression: X[Lo:Hi:Step].
func (*SliceExpr) AllocComments ¶
func (cr *SliceExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type Stmt ¶
type Stmt interface { Node // contains filtered or unexported methods }
A Stmt is a Starlark statement.
type Token ¶
type Token int8
A Token represents a Starlark lexical token.
const ( ILLEGAL Token = iota EOF NEWLINE INDENT OUTDENT // Tokens with values IDENT // x INT // 123 FLOAT // 1.23e45 STRING // "foo" or 'foo' or ”'foo”' or r'foo' or r"foo" BYTES // b"foo", etc // Punctuation PLUS // + MINUS // - STAR // * SLASH // / SLASHSLASH // // PERCENT // % AMP // & PIPE // | CIRCUMFLEX // ^ LTLT // << GTGT // >> TILDE // ~ DOT // . COMMA // , EQ // = SEMI // ; COLON // : LPAREN // ( RPAREN // ) LBRACK // [ RBRACK // ] LBRACE // { RBRACE // } LT // < GT // > GE // >= LE // <= EQL // == NEQ // != PLUS_EQ // += (keep order consistent with PLUS..GTGT) MINUS_EQ // -= STAR_EQ // *= SLASH_EQ // /= SLASHSLASH_EQ // //= PERCENT_EQ // %= AMP_EQ // &= PIPE_EQ // |= CIRCUMFLEX_EQ // ^= LTLT_EQ // <<= GTGT_EQ // >>= STARSTAR // ** // Keywords AND BREAK CONTINUE DEF ELIF ELSE FOR IF IN LAMBDA LOAD NOT NOT_IN // synthesized by parser from NOT IN OR PASS RETURN WHILE )
type TupleExpr ¶
type TupleExpr struct { Lparen Position // optional (e.g. in x, y = 0, 1), but required if List is empty List []Expr Rparen Position // contains filtered or unexported fields }
A TupleExpr represents a tuple literal: (List).
func (*TupleExpr) AllocComments ¶
func (cr *TupleExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type UnaryExpr ¶
type UnaryExpr struct { OpPos Position Op Token X Expr // may be nil if Op==STAR // contains filtered or unexported fields }
A UnaryExpr represents a unary expression: Op X.
As a special case, UnaryOp{Op:Star} may also represent the star parameter in def f(*args) or def f(*, x).
func (*UnaryExpr) AllocComments ¶
func (cr *UnaryExpr) AllocComments()
AllocComments enables comments to be associated with a syntax node.
type WhileStmt ¶
type WhileStmt struct { While Position Cond Expr Body []Stmt // contains filtered or unexported fields }
A WhileStmt represents a while loop: while X: Body.
func (*WhileStmt) AllocComments ¶
func (cr *WhileStmt) AllocComments()
AllocComments enables comments to be associated with a syntax node.