Documentation ¶
Overview ¶
Package asp implements an experimental BUILD-language parser. Parsing is doing using Participle (github.com/alecthomas/participle) in native Go, with a custom and also native partial Python interpreter.
Index ¶
- Constants
- Variables
- func AddReader(err error, r io.ReadSeeker) error
- func AddStackFrame(pos Position, err interface{}) error
- func GetExtents(statements []*Statement, statement *Statement, max int) (int, int)
- func NameOfReader(r io.Reader) string
- func StatementOrExpressionFromAst(stmts []*Statement, position Position) (statement *Statement, expression *Expression)
- type Argument
- type Call
- type CallArgument
- type Comprehension
- type Dict
- type DictItem
- type Expression
- type FString
- type FileInput
- type ForStatement
- type FuncDef
- type IdentExpr
- type IdentStatement
- type IdentStatementAction
- type IfStatement
- type InlineIf
- type Lambda
- type List
- type OpExpression
- type Operator
- type OptimisedExpression
- type Parser
- func (p *Parser) LoadBuiltins(filename string, contents, encoded []byte) error
- func (p *Parser) MustLoadBuiltins(filename string, contents, encoded []byte)
- func (p *Parser) ParseData(data []byte, filename string) ([]*Statement, error)
- func (p *Parser) ParseFile(pkg *core.Package, filename string) error
- func (p *Parser) ParseFileOnly(filename string) ([]*Statement, error)
- func (p *Parser) ParseReader(pkg *core.Package, r io.ReadSeeker) (bool, error)
- func (p *Parser) ParseToFile(input, output string) error
- type Position
- type ReturnStatement
- type Slice
- type Statement
- type Token
- type UnaryOp
- type ValueExpression
Constants ¶
const ( // Add etc are arithmetic operators - these are implemented on a per-type basis Add Operator = '+' // Subtract implements binary - (only works on integers) Subtract = '-' // Modulo implements % (including string interpolation) Modulo = '%' // LessThan implements < LessThan = '<' // GreaterThan implements > GreaterThan = '>' // LessThanOrEqual implements <= LessThanOrEqual = '≤' // GreaterThanOrEqual implements >= GreaterThanOrEqual = '≥' // Equal etc are comparison operators - also on a per-type basis but have slightly different rules. Equal = '=' // NotEqual implements != NotEqual = '≠' // In implements the in operator In = '∈' // NotIn implements "not in" as a single operator. NotIn = '∉' // And etc are logical operators - these are implemented type-independently And Operator = '&' // Or implements the or operator Or = '|' // Is implements type identity. Is = '≡' // Index is used in the parser, but not when parsing code. Index = '[' )
const ( EOF = -(iota + 1) Ident Int String LexOperator EOL Unindent )
Token types.
Variables ¶
var ( True pyBool = 1 False pyBool None pyBool = -1 FileNotFound pyBool = -2 )
True, False and None are the singletons representing those values in Python. None isn't really a bool of course, but it's easier to use an instance of bool than another custom type.
Functions ¶
func AddReader ¶
func AddReader(err error, r io.ReadSeeker) error
AddReader adds an io.Reader to an errStack, which will allow it to recover more information from that file.
func AddStackFrame ¶
AddStackFrame adds a new stack frame to the given errorStack, or wraps an existing error if not.
func GetExtents ¶
GetExtents returns the "extents" of a statement, i.e. the lines that it covers in source. The caller must pass a value for the maximum extent of the file; we can't detect it here because the AST only contains positions for the beginning of the statements.
func NameOfReader ¶
NameOfReader returns a name for the given reader, if one can be determined.
func StatementOrExpressionFromAst ¶
func StatementOrExpressionFromAst(stmts []*Statement, position Position) (statement *Statement, expression *Expression)
StatementOrExpressionFromAst recursively finds asp.IdentStatement and asp.Expression in the ast and returns a valid statement pointer if within range
Types ¶
type Argument ¶
type Argument struct { Name string `@Ident` Type []string `[ ":" @( { ( "bool" | "str" | "int" | "list" | "dict" | "function" ) [ "|" ] } ) ]` // Aliases are an experimental non-Python concept where function arguments can be aliased to different names. // We use this to support compatibility with Bazel & Buck etc in some cases. Aliases []string `[ "&" ( { @Ident [ "&" ] } ) ]` Value *Expression `[ "=" @@ ]` }
An Argument represents an argument to a function definition.
type Call ¶
type Call struct {
Arguments []CallArgument `[ @@ ] { "," [ @@ ] }`
}
A Call represents a call site of a function.
type CallArgument ¶
type CallArgument struct { Pos Position Name string `[ @@ "=" ]` Value Expression `@@` }
A CallArgument represents a single argument at a call site of a function.
func FindArgument ¶
func FindArgument(statement *Statement, args ...string) *CallArgument
FindArgument finds an argument of any one of the given names, or nil if there isn't one. The statement must be a function call (e.g. as returned by FindTarget).
type Comprehension ¶
type Comprehension struct { Names []string `"for" @Ident [ { "," @Ident } ] "in"` Expr *Expression `@@` Second *struct { Names []string `"for" @Ident [ { "," @Ident } ] "in"` Expr *Expression `@@` } `[ @@ ]` If *Expression `[ "if" @@ ]` }
A Comprehension represents a list or dict comprehension clause.
type Dict ¶
type Dict struct { Items []*DictItem `[ @@ ] { "," [ @@ ] }` Comprehension *Comprehension `[ @@ ]` }
A Dict represents a dict literal, either with or without a comprehension clause.
type DictItem ¶
type DictItem struct { Key Expression `@( Ident | String ) ":"` Value Expression `@@` }
A DictItem represents a single key-value pair in a dict literal.
type Expression ¶
type Expression struct { Pos Position EndPos Position UnaryOp *UnaryOp `( @@` Val *ValueExpression `| @@ )` Op []OpExpression `{ @@ }` If *InlineIf `[ @@ ]` // For internal optimisation - do not use outside this package. Optimised *OptimisedExpression }
An Expression is a generalised Python expression, i.e. anything that can appear where an expression is allowed (including the extra parts like inline if-then-else, operators, etc).
type FString ¶
type FString struct { Vars []struct { Prefix string // Preceding string bit Var string // Variable name to interpolate Config string // Config variable to look up } Suffix string // Following string bit }
A FString represents a minimal version of a Python literal format string. Note that we only support a very small subset of what Python allows there; essentially only variable substitution, which gives a much simpler AST structure here.
type FileInput ¶
type FileInput struct {
Statements []*Statement `{ @@ } EOF`
}
A FileInput is the top-level structure of a BUILD file.
type ForStatement ¶
type ForStatement struct { Names []string `"for" @Ident [ { "," @Ident } ] "in"` Expr Expression `@@ Colon EOL` Statements []*Statement `{ @@ } Unindent` }
A ForStatement implements the 'for' statement. Note that it does not support Python's "for-else" construction.
type FuncDef ¶
type FuncDef struct { Name string `"def" @Ident` Arguments []Argument `"(" [ @@ { "," @@ } ] ")" Colon EOL` Docstring string `[ @String EOL ]` Statements []*Statement `{ @@ } Unindent` EoDef Position // Not part of the grammar. Used to indicate internal targets that can only // be called using keyword arguments. KeywordsOnly bool }
A FuncDef implements definition of a new function.
type IdentExpr ¶
type IdentExpr struct { Pos Position EndPos Position Name string `@Ident` Action []struct { Property *IdentExpr ` "." @@` Call *Call `| "(" @@ ")"` } `{ @@ }` }
An IdentExpr implements parts of an expression that begin with an identifier (i.e. anything that might be a variable name).
type IdentStatement ¶
type IdentStatement struct { Name string `@Ident` Unpack *struct { Names []string `@Ident { "," @Ident }` Expr *Expression `"=" @@` } `( "," @@ ` Index *struct { Expr *Expression `@@ "]"` Assign *Expression `( "=" @@` AugAssign *Expression `| "+=" @@ )` } `| "[" @@` Action *IdentStatementAction `| @@ )` }
An IdentStatement implements a statement that begins with an identifier (i.e. anything that starts off with a variable name). It is a little fiddly due to parser limitations.
type IdentStatementAction ¶
type IdentStatementAction struct { Property *IdentExpr ` "." @@` Call *Call `| "(" @@ ")"` Assign *Expression `| "=" @@` AugAssign *Expression `| "+=" @@` }
An IdentStatementAction implements actions on an IdentStatement.
type IfStatement ¶
type IfStatement struct { Condition Expression `"if" @@ Colon EOL` Statements []*Statement `{ @@ } Unindent` Elif []struct { Condition Expression `"elif" @@ Colon EOL` Statements []*Statement `{ @@ } Unindent` } `{ @@ }` ElseStatements []*Statement `[ "else" Colon EOL { @@ } Unindent ]` }
An IfStatement implements the if-elif-else statement.
type InlineIf ¶
type InlineIf struct { Condition *Expression `"if" @@` Else *Expression `[ "else" @@ ]` }
An InlineIf implements the single-line if-then-else construction
type Lambda ¶
type Lambda struct { Arguments []Argument `[ @@ { "," @@ } ] Colon` Expr Expression `@@` }
A Lambda is the inline lambda function.
type List ¶
type List struct { Values []*Expression `[ @@ ] { "," [ @@ ] }` Comprehension *Comprehension `[ @@ ]` }
A List represents a list literal, either with or without a comprehension clause.
type OpExpression ¶
type OpExpression struct { Op Operator `@("+" | "-" | "%" | "<" | ">" | "and" | "or" | "is" | "in" | "not" "in" | "==" | "!=" | ">=" | "<=")` Expr *Expression `@@` }
An OpExpression is a operator combined with its following expression.
type OptimisedExpression ¶
type OptimisedExpression struct { // Used to optimise constant expressions. Constant pyObject // Similarly applied to optimise simple lookups of local variables. Local string // And similarly applied to optimise lookups into configuration. Config string }
An OptimisedExpression contains information to optimise certain aspects of execution of an expression. It must be public for serialisation but shouldn't be used outside this package.
type Parser ¶
type Parser struct {
// contains filtered or unexported fields
}
A Parser implements parsing of BUILD files.
func NewParser ¶
func NewParser(state *core.BuildState) *Parser
NewParser creates a new parser instance. One is normally sufficient for a process lifetime.
func (*Parser) LoadBuiltins ¶
LoadBuiltins instructs the parser to load rules from this file as built-ins. Optionally the file contents can be supplied directly. Also optionally a previously parsed form (acquired from ParseToFile) can be supplied.
func (*Parser) MustLoadBuiltins ¶
MustLoadBuiltins calls LoadBuiltins, and dies on any errors.
func (*Parser) ParseData ¶
ParseData reads the given byteslice and parses it into a set of statements. The 'filename' argument is only used in case of errors so doesn't necessarily have to correspond to a real file.
func (*Parser) ParseFile ¶
ParseFile parses the contents of a single file in the BUILD language. It returns true if the call was deferred at some point awaiting target to build, along with any error encountered.
func (*Parser) ParseFileOnly ¶
ParseFileOnly parses the given file but does not interpret it.
func (*Parser) ParseReader ¶
ParseReader parses the contents of the given ReadSeeker as a BUILD file. The first return value is true if parsing succeeds - if the error is still non-nil that indicates that interpretation failed.
type Position ¶
A Position describes a position in a source file. All properties in Position are one(1) indexed
type ReturnStatement ¶
type ReturnStatement struct {
Values []*Expression `[ @@ { "," @@ } ]`
}
A ReturnStatement implements the Python 'return' statement.
type Slice ¶
type Slice struct { Start *Expression `"[" [ @@ ]` Colon string `[ @":" ]` End *Expression `[ @@ ] "]"` }
A Slice represents a slice or index expression (e.g. [1], [1:2], [2:], [:], etc).
type Statement ¶
type Statement struct { Pos Position EndPos Position FuncDef *FuncDef `| @@` For *ForStatement `| @@` If *IfStatement `| @@` Return *ReturnStatement `| "return" @@ EOL` Raise *Expression `| "raise" @@ EOL` Assert *struct { Expr *Expression `@@` Message string `["," @String]` } `| "assert" @@ EOL` Ident *IdentStatement `| @@ EOL` Literal *Expression `| @@ EOL)` Pass bool `( @"pass" EOL` Continue bool `| @"continue" EOL` }
A Statement is the type we work with externally the most; it's a single Python statement. Note that some mildly excessive fiddling is needed since the parser we're using doesn't support backoff (i.e. if an earlier entry matches to its completion but can't consume following tokens, it doesn't then make another choice :( )
func FindTarget ¶
FindTarget returns the top-level call in a BUILD file that corresponds to a target of the given name (or nil if one does not exist).
type Token ¶
type Token struct { // Type of token. If > 0 this is the literal character value; if < 0 it is one of the types above. Type rune // The literal text of the token. Strings are lightly normalised to always be surrounded by quotes (but only one). Value string // The position in the input that the token occurred at. Pos Position }
A Token describes each individual lexical element emitted by the lexer.
type UnaryOp ¶
type UnaryOp struct { Op string `@( "-" | "not" )` Expr ValueExpression `@@` }
A UnaryOp represents a unary operation - in our case the only ones we support are negation and not.
type ValueExpression ¶
type ValueExpression struct { String string `( @String` FString *FString `| @FString` Int *struct { Int int `@Int` } `| @@` // Should just be *int, but https://github.com/golang/go/issues/23498 :( Bool string `| @( "True" | "False" | "None" )` List *List `| "[" @@ "]"` Dict *Dict `| "{" @@ "}"` Tuple *List `| "(" @@ ")"` Lambda *Lambda `| "lambda" @@` Ident *IdentExpr `| @@ )` Slice *Slice `[ @@ ]` Property *IdentExpr `[ ( "." @@` Call *Call `| "(" @@ ")" ) ]` }
A ValueExpression is the value part of an expression, i.e. without surrounding operators.
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
Package fuzz implements a fuzzing entry point for asp using go-fuzz.
|
Package fuzz implements a fuzzing entry point for asp using go-fuzz. |
Package main implements a compiler for the builtin build rules, which is used at bootstrap time.
|
Package main implements a compiler for the builtin build rules, which is used at bootstrap time. |