README
¶
wdte
WDTE is a simple, functional-ish, embedded scripting language.
Why does this exist?
Good question. In fact, I found myself asking the same thing, hence the name.
I had a number of design goals in mind when I started working on this project:
- Extremely simple. Entire grammar is less than 20-30 lines of specification.
- Grammar is LL(1) parseable.
- Functional-ish, but not particularly strict about it.
- Designed primarily for embedding. No command-line interpreter by default.
- Extremely easy to use from the binding side. In this case, that's primarily Go.
If you want to try the language yourself, feel free to take a look at the playground. It shows not only some of the features of the language in terms of actually writing code in it, but also how embeddable it is. The playground runs entirely in the browser on the client's end thanks to WebAssembly.
Example
package main
import (
"fmt"
"os"
"strings"
"github.com/DeedleFake/wdte"
)
const src = `
let i => import 'some/import/path/or/another';
i.print 3;
+ 5 2 -> i.print;
`
func im(from string) (*wdte.Scope, error) {
var print wdte.GoFunc
print = wdte.GoFunc(func(frame wdte.Frame, args ...wdte.Func) wdte.Func {
if len(args) < 1 {
return print
}
a := args[0].Call(frame)
fmt.Println(a)
return a
})
return wdte.S().Map(map[wdte.ID]wdte.Func{
"print": print,
}), nil
}
func main() {
m, err := wdte.Parse(strings.NewReader(src), wdte.ImportFunc(im))
if err != nil {
fmt.Fprintf(os.Stderr, "Error parsing module: %v\n", err)
os.Exit(1)
}
scope := wdte.S().Sub("+", wdte.GoFunc(func(frame wdte.Frame, args ...wdte.Func) wdte.Func {
var sum wdte.Number
for _, arg := range args {
sum += arg.Call(frame).(wdte.Number)
}
return sum
}))
m.Call(wdte.F().WithScope(scope))
}
3
7
Documentation
For an overview of the language's design and features, see the GitHub wiki.
Status
WDTE is in a pre-alpha state. It is filled with bugs and large amounts of stuff are subject to change without warning. That being said, if you're interested in anything, feel free to submit a pull request and get things fixed and/or implemented faster.
Documentation
¶
Overview ¶
Package wdte implements the WDTE scripting language.
WDTE is an embeddable, functionalish scripting language with a primary goal of simplicity of use from the embedding side, which is what this package provides.
Quick Language Overview ¶
In order to understand how this package works, an overview of the language itself is first necessary. WDTE is functional-ish, with some emphasis on the "-ish". Although it generally follows a functional design, it is not purely functional. In WDTE, everything is a function in that everything can be "called", optionally with arguments. Value types return themselves, however, allowing them to be passed around.
WDTE contains a construct called a "compound" which is similar to a function body in most languages. It is surrounded by parentheses and contains a semicolon separated list of expressions, with the last semicolon being optional. The top-level of a WDTE script is a compound without the parentheses. When a compound is executed, each expression is evaluated in turn. If any yield an error, that error is immediately returned from the entire compound. If not, the result of the last expression is returned.
Example 1
# Declare a function called print3 that takes no arguments. let print3 => ( let x => 3; io.writeln io.stdout x; );
There are very few functions built-in in WDTE, but the standard library, found in the std directory and its subdirectories, contains a number of useful functions and definitions. For example, the stream module contains iterator functionality, which provides a means of looping over expressions, something which is otherwise not possible.
Example 2
# Import 'stream' and 'array' and assign them to s and a,
# respectively. Note that an import is a compile-time operation,
# unlike normal functions. As such, it must be passed a string
# literal, not a variable.
let s => import 'stream';
let a => import 'arrays';
# Create a function called flatten that takes one argument,
# array.
let flatten array =>
# Create a new stream that iterates over array.
a.stream array
# Create a stream from the previous one that performs a flat
# map operation. The (@ name arg => ...) syntax is a lambda
# declaration.
-> s.flatMap (@ f v => v {
# If the current element of the stream, v, is an array,
# recursively flatten it into the stream.
reflect 'Array' => a.stream v -> s.flatMap f;
})
# Collect the previous stream into an array.
-> s.collect
;
This example also demonstrates "chains" and "switches", some features that seem complicated at first but quickly become second nature so with some practice.
A chain is a series of expressions separated by either the chain operator, "->", or the ignored chain operator, "--". Each piece of the chain is executed in turn, and the output of the previous section is passed as an argument to the output of the current section. In other words, in the previous example, the chain's execution matches the following pseudocode
r1 = a.stream(array) r2 = s.flatMap(<lambda>) r1 = r2(r1) r2 = s.collect return r2(r1)
A chain with a use of "--" operates in much the same way, but the output of the piece of the chain immediately following the operator is ignored, meaning that it doesn't affect the remainder of the chain.
Chains can also have "slots" assigned to each piece. This is an identifier immediately following the expression of a piece of chain. This identifier is inserted into the scope for the remainder of the chain, allowing manual access to earlier sections of the chain. For example
let io => import 'io'; let file => import 'io/file'; let readFile path => file.open path : f -> io.string -- io.close f ;
The aforementioned switch expression is the only conditional provided by WDTE. It looks like an expression followed by a semicolon separated series of cases in squiggly braces. A case is two expressions separated by the assignment operator, "=>". The original expression is first evaluated, following which each case's left-hand side is evaluated and the result of the original expression's evaluation is passed to it. If and only if this call results in the boolean value true, the right-hand side of that case is returned. If no cases match, the original expression is returned. For example,
func arg1 {
lhs1 arg2 => rhs1 arg3;
lhs2 arg4 => rhs2 arg6;
}
This is analogous to the following pseudocode
check = func(arg1)
if lhs := lhs1(arg2); lhs(check) {
return rhs1(arg3)
}
if lhs := lhs2(arg4); lhs(check) {
return rhs2(arg6)
}
return check
A few more minor points exist as well:
Array literals are a semicolon list of expression surrounded by square brackets. Like in compounds and switches, the last semicolon is optional. Identifier parsing rules are very loose; essentially, anything that isn't ambiguous with an existing keyword, operator, or other syntactic construct is allowed. All strings are essentially heredocs, allowing newlines like they're any other character. There's no difference between single-quoted and double-quoted strings. There are no boolean literals, but the standard library provides true and false functions that are essentially the same thing.
Embedding ¶
As previously mentioned, everything in WDTE is a function. In Go terms, everything in WDTE implements the Func type defined in this package. This includes syntactic constructs as well, such as compounds, switches, and chains.
When a script is parsed by one of the parsing functions in this package, it is translated into a recursive series of Func implementations. The specific types that it is translated to are all defined in and exported by this package. For example, the top-level of a script, being itself a compound, results in the instantiation of a Compound.
What this means in terms of embedding is that the only thing required for interaction between Go and WDTE is an interoperative layer of Func implementations. As a functional language, WDTE is stateless; there is no global interpreter state to keep track of at all. Systems for tracking interpreter state, should they be required, are provided by the repl package.
When a Func is called, it is passed a Frame. A Frame keeps track of anything the function needs that isn't directly an argument to the function. This includes the scope in which the Func call should be evaluated. For example, the expression
func arg1 arg2
translates to an instance of the FuncCall implementation of Func. When the FuncCall is "called", it must be given a scope which contains, at a minimum, "func", "arg1", and "arg2", or the call will fail with an error. It is through this mechanism that new functions can be provided to WDTE. A custom scope can be created with new implementations of Func inserted into it. If this scope is inserted into a Frame which is then passed to a call of, for example, the top-level compound created by parsing a script, they will be available during the evaluation.
Example:
const src = `
let io => import 'io';
io.writeln io.stdout example;
`
c, _ := wdte.Parse(strings.NewReader(src), std.Import)
scope := std.Scope.Add("example", wdte.String("This is an example."))
r := c.Call(std.F().WithScope(scope))
if err, ok := r.(error); ok {
log.Fatalln(err)
}
This will print "This is an example." to stdout.
For convenience, a simple function wrapper around the single method required by Func is provided in the form of GoFunc. GoFunc provides a number of extra features, such as automatically converting panics into errors, but for the most part is just a simple wrapper around manual implementations of Func. If more automatic behavior is required, possibly at the cost of some runtime performance, functions for automatically wrapping Go functions are provided in the wdteutil package.
One final note: WDTE is lazily-evaluated. Very, very lazily-evaluated. Until Go code manually calls a Func implementation, most WDTE code is never evaluated at all past the initial parsing. In some cases some code may get called more times than expected as well. Because of this, it is highly recommended that any Go code that is expected to be directly called by WDTE provide a purely functional interface, deterministically returning the same thing for the same arguments. If code does not follow this guideline, expect occasional odd behavior for seemingly no reason.
Index ¶
- Constants
- func AssignPattern(frame Frame, scope *Scope, ids []ID, val Func) (*Scope, Func)
- func AssignSimple(frame Frame, scope *Scope, ids []ID, val Func) (*Scope, Func)
- func Reflect(f Func, name string) bool
- type Array
- type AssignFunc
- type Assigner
- type Atter
- type Bool
- type Chain
- type ChainPiece
- type Collector
- type Comparer
- type Compound
- type Error
- type Frame
- type Func
- type FuncCall
- type GoFunc
- type ID
- type ImportFunc
- type Importer
- type Lambda
- type Lenner
- type Memo
- type Number
- type Reflector
- type Scope
- func (s *Scope) Add(id ID, val Func) *Scope
- func (s *Scope) At(i Func) (Func, error)
- func (s *Scope) Call(frame Frame, args ...Func) Func
- func (s *Scope) Custom(getFunc func(ID) Func, known func(map[ID]struct{})) *Scope
- func (s *Scope) Get(id ID) Func
- func (s *Scope) Known() []ID
- func (s *Scope) Map(vars map[ID]Func) *Scope
- func (s *Scope) Parent() *Scope
- func (s *Scope) Reflect(name string) bool
- func (s *Scope) Set(k, v Func) (Func, error)
- func (s *Scope) String() string
- func (s *Scope) Sub(sub *Scope) *Scope
- type Setter
- type String
- type Sub
- type Switch
- type Var
Constants ¶
const ( NormalChain = 0 IgnoredChain = 1 << (iota - 1) ErrorChain )
Variables ¶
This section is empty.
Functions ¶
func AssignPattern ¶ added in v0.5.3
AssignPattern performs a pattern matching assignment, placing values retrieved from an Atter into the corresponding provided IDs.
func AssignSimple ¶ added in v0.5.3
AssignSimple is an AssignFunc which places a single value into the scope with a single ID.
func Reflect ¶ added in v0.4.4
Reflect checks if a Func can be considered to be of a given type. If v implements Reflector, v.Reflect(name) is used to check for compatability. If not, a simple string comparison is done against whatever Go's reflect package claims the short name of the underlying type to be.
Types ¶
type Array ¶
type Array []Func
An Array represents a WDTE array type. It's similar to a Compound, but when evaluated, it returns itself with its own members replaced with their own evaluations. This allows it to be passed around as a value in the same way as strings and numbers.
type AssignFunc ¶ added in v0.5.3
AssignFunc places items into a scope. How exactly it does this differs, but the general idea is that it should return a scope which contains the IDs given with data somehow gotten from the provided Func, possibly involving calls using the given Frame. It returns the new scope and a Func. Ideally, this should be the Func that was originally provided, possibly wrapped in something, but it may not be.
In the event of an error, an AssignFunc should return a nil scope alongside the returned Func to indicate that it didn't simply store an error value in the scope, which would be completely valid.
type Assigner ¶ added in v0.5.2
type Assigner struct {
AssignFunc AssignFunc
IDs []ID
Expr Func
}
An Assigner bundles a known list of IDs and an expression with an AssignFunc.
type Bool ¶
type Bool bool
Bool is a boolean. Like other primitive types, it simply returns itself when called.
type ChainPiece ¶ added in v0.5.3
type ChainPiece struct {
Expr Func
Flags uint
Slots []ID
AssignFunc AssignFunc
}
A ChainPiece is, as you can probably guess from the name, a piece of a Chain. It stores the underlying expression as well as some extra information necessary for properly evaluating the Chain.
func (ChainPiece) String ¶ added in v0.5.5
func (p ChainPiece) String() string
type Collector ¶ added in v0.10.0
type Collector struct {
Compound Compound
}
Collector wraps a compound, causing it to return its collected scope instead of the last result. If any expression in the compound returns an error, however, then that error is returned instead.
type Comparer ¶
type Comparer interface {
// Compare returns two values. The meaning of the first is dependent
// upon the second. If the second is true, then the first indicates
// ordering via the standard negative, positive, and zero results to
// indicate less than, greater than, and equal, respectively. If the
// second is false, then the first indicates only equality, with
// zero still meaning equal, but other values simply meaning unequal.
Compare(other Func) (int, bool)
}
A Comparer is a Func that is able to be compared to other functions.
type Compound ¶
type Compound []Func
A Compound represents a compound expression. Calling it calls each of the expressions in the compound, returning the value of the last one. If the compound is empty, nil is returned.
If an element of a compound is an Assigner, it is used to build a new subscope under which the remainder of the elements of the compound will be evaluated. If the element is the last element of the compound, the Func returned by its assignment is returned from the whole compound.
func FromAST ¶
FromAST translates an AST into a top-level compound. im is used to handle import statements. If im is nil, a no-op importer is used.
func Parse ¶
Parse parses an AST from r and then translates it into a top-level compound. im is used to handle import statements. If im is nil, a no-op importer is used. In most cases, std.Import is a good default.
type Error ¶
type Error struct {
// Err is the error that generated the Error. In a lot of cases,
// this is just a simple error message.
Err error
// Frame is the frame of the function that the error was first
// generated in.
Frame Frame
}
An Error is returned by any of the built-in functions when they run into an error.
type Frame ¶
type Frame struct {
// contains filtered or unexported fields
}
A Frame tracks information about the current function call, such as the scope that the function is being executed in and debugging info.
func F ¶
func F() Frame
F returns a top-level frame. This can be used by Go code calling WDTE functions directly if another frame is not available.
In many cases, it may be preferable to use std.F() instead.
func (Frame) ID ¶
ID returns the ID of the frame. This is generally the function that created the frame.
func (Frame) Parent ¶
Parent returns the frame that this frame was created from, or a blank frame if there was none.
func (Frame) Sub ¶
Sub returns a new child frame of f with the given ID and the same scope as f.
Under most circumstances, a GoFunc should call this before calling any WDTE functions, as it is useful for debugging. For example:
func Example(frame wdte.Frame, args ...wdte.Func) wdte.Func {
frame = frame.Sub("example")
...
}
func (Frame) WithContext ¶
WithContext returns a copy of f with the given context.
type Func ¶
type Func interface {
// Call calls the function with the given arguments, returning its
// return value. frame represents the current call frame, which
// tracks scope as well as debugging info.
Call(frame Frame, args ...Func) Func
}
Func is the base type through which all data is handled by WDTE. It represents everything that can be passed around in the language. This includes functions, of course, expressions, strings, numbers, Go functions, and anything else the client wants to pass into WDTE.
type FuncCall ¶ added in v0.2.1
A FuncCall is an unevaluated function call. This is usually the right-hand side of a function declaration, but could also be any of various pieces of switches, compounds, or arrays.
type GoFunc ¶
A GoFunc is an implementation of Func that calls a Go function. This is the easiest way to implement lower-level systems for WDTE scripts to make use of.
For example, to implement a simple, non-type-safe addition function:
GoFunc(func(frame wdte.Frame, args ...wdte.Func) wdte.Func {
frame = frame.Sub("+")
var sum wdte.Number
for _, arg := range(args) {
sum += arg.Call(frame).(wdte.Number)
}
return sum
})
If placed into a scope with the ID "+", this function can then be called from WDTE as follows:
- 3 6 9
As shown, it is recommended that arguments be passed the given frame when evaluating them. Failing to do so without knowing what you're doing can cause unexpected behavior, including sending the evaluation system into infinite loops or causing panics.
In the event that a GoFunc panics with an error value, it will be automatically caught and converted into an Error, which will then be returned.
type ImportFunc ¶
ImportFunc is a wrapper around simple functions to allow them to be used as Importers.
type Importer ¶
An Importer creates scopes from strings. When parsing a WDTE script, an importer is used to import scopes into namespaces.
When the WDTE import expression
import 'example'
is parsed, the associated Importer will be invoked as follows:
im.Import("example")
type Lambda ¶
A Lambda is a closure. When called, it calls its inner expression with itself and its own arguments placed into the scope. In other words, given the lambda
(@ ex x y => + x y)
it will create a new subscope containing itself under the ID "ex", and its first and second arguments under the IDs "x" and "y", respectively. It will then evaluate `+ x y` in that new scope.
The arguments in the subscope, not including the self-reference, are contained in the boundary "args". The self-reference is contained in the boundary "self".
type Lenner ¶
type Lenner interface {
Len() int
}
A Lenner is a Func that has a length, such as arrays and strings.
type Number ¶
type Number float64
A Number is a number, as parsed from a number literal. That's about it. Like everything else, it's a function. It simply returns itself when called.
type Reflector ¶ added in v0.4.2
A Reflector is a Func that can determine if it can be treated as the named type or not. For example,
s := wdte.String("example")
return s.Reflect("string")
returns true.
type Scope ¶
type Scope struct {
// contains filtered or unexported fields
}
Scope is a tiered storage space for local variables. This includes function parameters and chain slots. A nil *Scope is equivalent to a blank, top-level scope.
func (*Scope) Custom ¶
Custom returns a new subscope that uses the given lookup function to retrieve values. If getFunc returns nil, the parent of s will be searched. known is an optional function which adds all variables known to this layer of the scope into the map that it is passed as keys.
func (*Scope) Get ¶
Get returns the value of the variable with the given ID. If the variable doesn't exist in either the current scope or any of its parent scopes, nil is returned.
func (*Scope) Map ¶
Map returns a subscope that includes the given mapping of variable names to functions. Note that no copy is made of vars, so changing the map after passing it to this method may result in undefined behavior.
type Setter ¶ added in v0.8.0
A Setter is a Func that can produce a new Func from itself with a key-value mapping applied in some way. For example, a scope can produce a subscope with a new variable added to it, or an array can produce a new array with an index modified.
type String ¶
type String string
A String is a string, as parsed from a string literal. That's about it. Like everything else, it's a function. It simply returns itself when called.
type Sub ¶
type Sub []Func
A Sub is a function that is in a subscope. This is most commonly an imported function.
type Switch ¶
type Switch struct {
// Check is the condition at the front of the switch.
Check Func
// Cases is the switch's cases. Each contains two functions. The
// first index is the left-hand side, while the second is the
// right-hand side. When the switch is evaluated, the cases are run
// in order. If any matches, the right-hand side is evaluated and
// its return value is returned.
Cases [][2]Func
}
Switch represents a switch expression.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package ast provides the parser for WDTE.
|
Package ast provides the parser for WDTE. |
|
cmd
|
|
|
Package repl provides a layer intended to help with the development of a read-eval-print loop.
|
Package repl provides a layer intended to help with the development of a read-eval-print loop. |
|
Package scanner provides a scanner for WDTE tokens.
|
Package scanner provides a scanner for WDTE tokens. |
|
Package std provides a number of basic WDTE functions.
|
Package std provides a number of basic WDTE functions. |
|
all
Package all is a convenience package that imports the entire standard library, thus registering it with std.Import.
|
Package all is a convenience package that imports the entire standard library, thus registering it with std.Import. |
|
arrays
Package arrays contains functions for manipulating arrays.
|
Package arrays contains functions for manipulating arrays. |
|
io
Package io contains WDTE functions for dealing with files and other types of data streams.
|
Package io contains WDTE functions for dealing with files and other types of data streams. |
|
io/file
Package file provides functions for dealing with files.
|
Package file provides functions for dealing with files. |
|
math
Package math contains wdte.Funcs for performing mathematical operations.
|
Package math contains wdte.Funcs for performing mathematical operations. |
|
rand
Package rand provides functions for generating and dealing with random numbers.
|
Package rand provides functions for generating and dealing with random numbers. |
|
stream
Package stream provides WDTE functions for manipulating streams of data.
|
Package stream provides WDTE functions for manipulating streams of data. |
|
strings
Package strings contains functions for dealing with strings.
|
Package strings contains functions for dealing with strings. |
|
Package wdteutil provides higher-level automatic wrappers and convenience functions for Go/WDTE interoperability.
|
Package wdteutil provides higher-level automatic wrappers and convenience functions for Go/WDTE interoperability. |