Documentation
¶
Overview ¶
Package syntax implements the Reflow language.
It is a simple, type-safe, applicative domain specific language that compiles to Reflow's Flow IR.
A Reflow type is one of the following (where t1, t2, etc. are themselves types):
int // the type of arbitrary precision integers float // the type of arbitrary precision floats string // the type of (UTF-8 encoded) strings bool // the type of booleans file // the type of files dir // the type of directories (t1, t2, .., tn) // the type of the tuple consisting of types // t1, t2, t3... (id1, id2 t1, .., idn tn) // the type of the tuple (t1, t1, ..., tn), but // with optional labels (syntactic affordance) [t] // the type of list of type t [k:v] // the type of map with keys of type k // and values of type v {id1 t1, id2 t2} // the type of a struct with fields id1, id2 // of types t1, t2 respectively {id1, id2 t1, id3 t3} // the type of struct{id1 t1, id2 t1, id3 t3} // (syntactic affordance) func(t1, t2, ..., tn) tr // the type of a function with argument types // t1, t2, ..., tn, and return type tr func(a1, a2 t1, ..., an tn) tr // a function of type func(t1, t1, ..., tn) tr // with labels (syntactic affordance)
A Reflow expression is one of the following (where e1, e2, .. are themselves expressions, d1, d2, .. are declarations; t1, t2, .. are types):
123 // a literal (arbitrary precision) integer "abc" // a literal UTF-8 encoded string ident // an identifier [e1, e2, e3, ..] // a list of e1, e2, e3.. [e1, e2, ...e3] // list e1, e2, concatenated with list e3 [e1:e2, e3:e4, ..] // a map key e1 to value e2, etc. [e1:e2, e3:e4, ...e5] // as above, with map e5 appended (e1, e2, e3, ..) // a tuple of e1, e2, e3, .. {id1: e1, id2: e2, ..} // a struct with fields id1 with value e1, id2 with value e2, .. {id1, id2, ..} // a shorthand for {id1: id1, id2: id2} {d1; d2; ...; dn; e1} // a block of declarations usable by expression e1 func(id1, id2 t1, id3 t3) t4 => e1 // a function literal with arguments and return type; evaluates e1 func(id1, id2 t1, id3 t3) => e1 // a function literal with arguments, return type omitted exec(d1, d2, ..) t1 {{ template }} // an exec with declarations d1, d2, .., returning t1 with template // identifiers are valid declarations in this context; they are // deparsed as id := id. e1 <op> e2 // a binary op (||, &&, <, >, <=, >=, !=, ==, +, /, %, &, <<, >>) <op> e1 // unary expression (!) if e1 { d1; d2; ..; e2 } else { d3; d4; ..; e3 } // conditional expression (with declaration blocks) (e1) // parenthesized expression to control precedence int(e1) // builtin float to int type conversion float(e1) // builtin int to float type conversion len(e1) // builtin length operator zip(e1, e2) // builtin zip operator unzip(e1) // builtin unzip operator map(e1) // convert e1 to a map list(e1) // convert e1 to a list make(strlit, d1, ..., dn) // builtin make primitive. identifiers are valid declarations in // this context; they are deparsed as id := id. panic(e1) // terminate the program with error e1 [e1 | c1, c2,..., cn] // list comprehension: evaluate e1 in the environment provided by // the given clauses (see below) e1 ~> e2 // force evaluation of e1, ignore its result, then evaluate e2. flatten(e1) // flatten a list of lists into a single list trace(e1) // trace expression e1: evaluate it, print it to console, // and return it. Can be used for debugging. range(e1, e2) // produce a list of integers with the range of the two expressions.
A comprehension clause is one of the following:
pat <- e1 // bind a pattern to each of the list or map e1 if e1 // filter entries that do not meet the predicate e1
A Reflow declaration is one of the following:
val id = e1 or id := e1 // assign e1 to id val id t1 = e1 // assign e1 to id with type t1 type id t1 // declare id as a type alias to t1 func id(a1, a2 t1) r1 = e1 // sugar for id := func(a1, a2 t1) r1 => e1 func id(a1, a2 t1) = e1 // sugar for id := func(a1, a2 t1) => e1
Value declarations may be preceded by one of the following annoations, each of which takes a list of declarations.
@requires(...) // resource requirement annotation, // takes declarations mem int, // cpu int or cpu float, disk int, // cpufeatures[string, and wide // bool. They indicate resource // requirements for computing the // declaration; if wide is set to // true, then the resource // requirements have no // theoretical upper bound. Wide // is thus useful for // declarations whose // parallelization factor is not // known statically, for example // when processing sharded data.
Value declarations can take destructive pattern bindings, mimicing value constructors. Currently tuples and lists are supported. Patterns accept identifiers and "_" (ignore), but not yet literal values. Patterns follow the grammar:
_ [p1, p2, ..., pn] (p1, p2, ..., pn) {id1: p1, ..., idn: pn} {id1, ..., id3} // sugar for {id1: id1, ..., idn: idn}
For example, the declaration
val [(x1, y1), (x2, y2), _] = e1
pattern matches e1 to a list of length three, whose elements are 2-tuples. The first two tuples are bound; the third is ignored.
A Reflow module consists of, in order: a optional keyspace, a set of optional parameters, and a set of declarations.
keyspace "com.grail.WGSv1" // defines the keyspace to "com.grail.WGS1" param id1 = e1 // defines parameter id1 with default e1 param id2 string // defines parameter without default, of type t1 param ( // block version of parameters id3 int id4 stirng ) declarations // declarations as above
The Reflow language infers types, except for in function arguments and non-default params. Everywhere else, types can safely be omitted. When a type is supplied, it is used as an ascription: the expression is ascribed to the given type; the type checker fails if the expression's type is incompatible with its ascription.
Following Go, semicolons are inserted if a newline appears after the following tokens: identifier, string, integer, template, ')', '}', or ']'
All bytes in a line following the characters "//" are commentary.
Index ¶
- func Force(v values.T, t *types.T) values.T
- func Modules() (names []string)
- func RegisterModule(name string, m *ModuleImpl)
- func Stdlib() (*types.Env, *values.Env)
- type ComprClause
- type ComprKind
- type Decl
- type DeclKind
- type Expr
- type ExprKind
- type ExprValue
- type FieldExpr
- type MatchKind
- type Matcher
- type Module
- type ModuleImpl
- func (m *ModuleImpl) Doc(ident string) string
- func (m *ModuleImpl) Eager() bool
- func (m *ModuleImpl) FlagEnv(flags *flag.FlagSet, env *values.Env) error
- func (m *ModuleImpl) Flags(sess *Session, env *values.Env) (*flag.FlagSet, error)
- func (m *ModuleImpl) Init(sess *Session, env *types.Env) error
- func (m *ModuleImpl) Make(sess *Session, params *values.Env) (values.T, error)
- func (m *ModuleImpl) Param(id string) (*types.T, bool)
- func (m *ModuleImpl) ParamErr(env *types.Env) error
- func (m *ModuleImpl) Params() []Param
- func (m *ModuleImpl) String() string
- func (m *ModuleImpl) Type() *types.T
- type Param
- type Parser
- type ParserMode
- type Pat
- type PatKind
- type Path
- type Session
- type SystemFunc
- type Template
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Force ¶
Force produces a strict version of v. Force either returns an immediate value v, or else a *reflow.Flow that will produce the immediate value.
func Modules ¶
func Modules() (names []string)
Modules returns the names of the available systems modules.
func RegisterModule ¶
func RegisterModule(name string, m *ModuleImpl)
RegisterModule is a hook to register custom reflow intrinsics.
Types ¶
type ComprClause ¶
type ComprClause struct { // Kind is the clause's kind. Kind ComprKind // Pat is the clause's pattern (ComprEnum). Pat *Pat // Expr is the clause's expression (ComprEnum, ComprFilter). Expr *Expr }
A ComprClause is a single clause in a comprehension expression.
type Decl ¶
type Decl struct { // Position contains the source position of the node. // It is set by the parser. scanner.Position // Comment stores an optional comment attached to the declaration. Comment string // Kind is the Decl's op; see above. Kind DeclKind // Ident is the identifier in a DeclDeclare and DeclType. Ident string // Pat stores the pattern for this declaration. Pat *Pat // Expr stores the rvalue expression for this declaration. Expr *Expr // Type stores the type for DeclDeclare and DeclType. Type *types.T }
A Decl is a Reflow declaration.
type Expr ¶
type Expr struct { // Position contains the source position of the node. // It is set by the parser. scanner.Position // Comment is the commentary text that precedes this expression, // if any. Comment string // Kind is the expression's op; see above. Kind ExprKind // Cond is the condition expression in ExprCond. Cond *Expr // Left is the "left" operand for expressions. Left *Expr // Right is the "right" operand for expressions. Right *Expr // Op is the binary operation in ExprBinop, unary operation in // ExprUnop, and builtin in ExprBuiltin. Op string // Args holds function arguments in an ExprFunc. Args []*types.Field // List holds expressions for list literals. List []*Expr // Map holds expressions for map literals. Map map[*Expr]*Expr // Decls holds declarations for ExprBlock and ExprExec. Decls []*Decl // Fields holds field definitions (identifiers and expressions) // in ExprStruct, ExprTuple Fields []*FieldExpr // Ident stores the identifier for ExprIdent Ident string // Val stores constant values in ExprConst. Val values.T // Type holds the Type in ExprAscribe, ExprExec, and ExprConst. Type *types.T // Template is the exec template in ExprExec. Template *Template ComprExpr *Expr ComprClauses []*ComprClause // Env stores a value environmetn for ExprThunk. Env *values.Env // Pat stores the bind pattern in a comprehension. Pat *Pat // Module stores the module as opened during type checking. Module Module }
An Expr is a node in Reflow's expression AST.
func (*Expr) Abbrev ¶
Abbrev shows an "abbreviated" pretty-printed version of expression e. These are useful when showing expression values in documentary output; but they do not necessarily parse. Abbrev strips unnecessary parentheses from arithmetic expressions.
func (*Expr) Digest ¶
Digest computes an identifier that uniquely identifies what Expr e will compute with the given environment. i.e., it can be used as a key to identify the computation represented by Expr e and an environment.
On a semantic level, Digest is one-to-many: that is, there are many candidate digests for a given semantic computation. However, we go through some lengths to normalize, for example by using De Bruijn indices (levels) to remove dependence on concrete names. In the future, we could consider canonicalizing the expression tree as well (e.g., by exploiting commutatvity, etc.)
type ExprKind ¶
type ExprKind int
ExprKind is the kind of an expression.
const ( // ExprError indicates an erroneous expression (e.g., through a parse error) ExprError ExprKind = iota // ExprIdent is an identifier reference. ExprIdent // ExprBinop is a binary operation. ExprBinop // ExprUnop is a unary operation. ExprUnop // ExprApply is function application. ExprApply // ExprConst is a const (literal). ExprConst // ExprAscribe is an ascription (static type assertion). ExprAscribe // ExprBlock is a declaration-block. ExprBlock // ExprFunc is a function definition. ExprFunc // ExprTuple is a tuple literal. ExprTuple // ExprStruct is a struct literal. ExprStruct // ExprList is a list literal. ExprList // ExprMap is a map literal. ExprMap // ExprExec is an exec expression. ExprExec // ExprCond is a conditional expression. ExprCond // ExprDeref is a struct derefence expression. ExprDeref // ExprIndex is an indexing (map or list) expression. ExprIndex // ExprCompr is a comprehension expression. ExprCompr // ExprMake is a module instantiation expression. ExprMake // ExprBuiltin is a builtin expression (e.g., len, zip, unzip). ExprBuiltin // ExprRequires assigns resources to the underlying expression. // It also necessarily forces the value. ExprRequires // ExprThunk is a delayed evaluation (expression + environment). // These are never produced from parsing--they are used internally // by the evaluator. (But see note there.) ExprThunk )
type ExprValue ¶
type ExprValue struct {
Left, Right values.T
Cond values.T
List []values.T
Map []struct{ K, V *values.T }
Fields []values.T
Extras []values.T
}
ExprValue stores the evaluated values associated with the dependencies of an Expr node. It is used while evaluating an expression tree.
type Matcher ¶
type Matcher struct { // Kind is the kind of matcher. Kind MatchKind // Index is the index of the match (MatchTuple, MatchList). Index int // Parent is this matcher's parent. Parent *Matcher // Field holds a struct field (MatchStruct). Field string }
A Matcher binds individual pattern components (identifiers) in a pattern. Matchers form a tree; their interpretation (through method Match) performs value destructuring.
type Module ¶
type Module interface { // Make creates a new module instance in the provided session // with the provided parameters. Make(sess *Session, params *values.Env) (values.T, error) // ParamErr type-checks parameter types, returning an error on failure. ParamErr(env *types.Env) error // Flags returns the set of flags provided by this module. // Note that not all modules may have parameters that are supported // by the regular flag types. These return an error. Flags(sess *Session, env *values.Env) (*flag.FlagSet, error) // FlagEnv adds flags from the FlagSet to value environment env. // The FlagSet should be produced by Module.Flags. FlagEnv(flags *flag.FlagSet, env *values.Env) error // Params returns the parameter descriptors for this module. Params() []Param // Doc returns the docstring for a toplevel identifier. Doc(string) string // Type returns the type of the module. Type() *types.T // Eager tells whether the module requires eager parameters. // When it does, all parameters are forced and fully evaluated // before instantiating a new module instance. Eager() bool }
Module abstracts a Reflow module, having the ability to type check parameters, inspect its type, and mint new instances.
type ModuleImpl ¶
type ModuleImpl struct { // Keyspace is the (optional) key space of this module. Keyspace *Expr // Reservation is the set of reservation declarations. Reservation []*Decl // ParamDecls is the set of declared parameters for this module. ParamDecls []*Decl // Decls is the set of declarations in this module. Decls []*Decl Docs map[string]string // contains filtered or unexported fields }
ModuleImpl defines a Reflow module comprising: a keyspace, a set of parameters, and a set of declarations.
func (*ModuleImpl) Doc ¶
func (m *ModuleImpl) Doc(ident string) string
Doc returns the documentation for the provided identifier.
func (*ModuleImpl) FlagEnv ¶
FlagEnv adds flags from the FlagSet to value environment env. The FlagSet should be produced by (*Module).Flags.
func (*ModuleImpl) Flags ¶
Flags returns a FlagSet that captures the parameters of this module. This can be used to parameterize a module from the command line. The returned FlagSet uses parameter documentation as the help text.
func (*ModuleImpl) Init ¶
func (m *ModuleImpl) Init(sess *Session, env *types.Env) error
Init type checks this module and returns any type checking errors.
func (*ModuleImpl) Make ¶
Make creates a new instance of this module. ParamDecls contains the value environment storing parameter values.
func (*ModuleImpl) Param ¶
func (m *ModuleImpl) Param(id string) (*types.T, bool)
Param returns the type of the module parameter with identifier id, and whether it is mandatory.
func (*ModuleImpl) ParamErr ¶
func (m *ModuleImpl) ParamErr(env *types.Env) error
ParamErr type checks the type environment env against the parameters of this module. It returns any type checking errors (e.g., badly typed parameters, or missing ones).
func (*ModuleImpl) Params ¶
func (m *ModuleImpl) Params() []Param
Params returns the parameter metadata for this module.
func (*ModuleImpl) String ¶
func (m *ModuleImpl) String() string
String renders a tree-formatted version of m.
type Parser ¶
type Parser struct { // File is prefixed to parser error locations. File string // Body is the io.Reader that is parsed. Body io.Reader // Mode governs how the parser is started. See documentation above. // The fields Module, Decls, Expr, and Type are set depending on the // parser mode. Mode ParserMode // Module contains the parsed module (LexerModule). Module *ModuleImpl // Decls contains the parsed declarations (LexerDecls). Decls []*Decl // Expr contains the parsed expression (LexerExpr). Expr *Expr // Type contains the pased type (LexerType). Type *types.T // contains filtered or unexported fields }
Parser is a Reflow lexer. It composes an (internal) scanner to produce tokens for the YACC grammar. Parser inserts semicolons following the rules outlined in the package docs. Lex implements the (internal) yyParser.
type ParserMode ¶
type ParserMode int
ParserMode determines the lexer's entry behavior.
const ( // ParseModule parses a module. ParseModule ParserMode = iota // ParseDecls parses a set of declarations. ParseDecls // ParseExpr parses an expression. ParseExpr // ParseType parses a type. ParseType )
type Pat ¶
A Pat stores a pattern tree used in destructuring operations. Patterns can bind type and value environments. They can also produce matchers that can be used to selectively match and bind identifiers.
func (*Pat) BindTypes ¶
BindTypes binds the pattern's identifier's types in the passed environment, given the type of binding value t.
func (*Pat) BindValues ¶
BindValues binds this pattern's values in the given value environment.
type PatKind ¶
type PatKind int
PatKind is the kind of pattern.
const ( // PatError is an erroneous pattern // (e.g., uninitialized or parse error). PatError PatKind = iota // PatIdent is an identifier pattern. PatIdent // PatTuple is a tuple pattern. PatTuple // PatList is a list pattern. PatList // PatStruct is a struct pattern. PatStruct // PatIgnore is an ignore pattern. PatIgnore )
type Path ¶
type Path []*Matcher
Path represents a path to a value.
type Session ¶
type Session struct {
// Stdout and stderr is the writer to which standard output and error are written.
Stdout, Stderr io.Writer
Types *types.Env
Values *values.Env
// contains filtered or unexported fields
}
A Session is a compiler session. It's responsible for opening, parsing and type checking modules.
type SystemFunc ¶
type SystemFunc struct { Module string Id string Doc string Type *types.T Force bool Do func(loc values.Location, args []values.T) (values.T, error) }
SystemFunc is utility to define a reflow intrinsic.
func (SystemFunc) Decl ¶
func (s SystemFunc) Decl() *Decl
Decl returns the intrinsic as a reflow declaration.
func (SystemFunc) Digest ¶
func (s SystemFunc) Digest() digest.Digest
Digest computes the digest of the intrinsic.
type Template ¶
Template is an exec template and its interpolation arguments. The template is stored as a number of fragments interspersed by argument expressions to be rendered.
The following are guaranteed invariant:
len(Frags) > 0 len(Frags) == len(Args)+1
func (*Template) FormatString ¶
FormatString returns a format string that can be used to render the final template. It's provided for backwards compatibility only.