Documentation
¶
Overview ¶
Package parse provides support functionality for writing scannerless parsers.
The main entry point is the Parse function. It works by setting up a parser on the supplied content, and then invoking the supplied root parsing function. The CST is built for you inside the ParseLeaf and ParseBranch methods of the parser, but it is up to the supplied parsing functions to hold on to the CST if you want it, and also to build the AST.
Index ¶
- Constants
- Variables
- type Branch
- type BranchParser
- type CSTMap
- type Error
- type ErrorList
- type Fragment
- type Leaf
- type LeafParser
- type Node
- type NumberKind
- type Parser
- func (p *Parser) Error(message string, args ...interface{})
- func (p *Parser) ErrorAt(at interface{}, message string, args ...interface{})
- func (p *Parser) Expected(value string)
- func (p *Parser) Extend(ast interface{}, do BranchParser)
- func (p *Parser) ParseBranch(cst *Branch, do BranchParser)
- func (p *Parser) ParseLeaf(cst *Branch, do LeafParser)
- type Reader
- func (r *Reader) Advance()
- func (r *Reader) AdvanceN(n int)
- func (r *Reader) AlphaNumeric() bool
- func (r *Reader) Consume() Token
- func (r *Reader) EOL() bool
- func (r *Reader) GuessNextToken() Token
- func (r *Reader) IsEOF() bool
- func (r *Reader) IsEOL() bool
- func (r *Reader) NotSpace() bool
- func (r *Reader) Numeric() NumberKind
- func (r *Reader) Peek() rune
- func (r *Reader) PeekN(n int) rune
- func (r *Reader) Rollback()
- func (r *Reader) Rune(value rune) bool
- func (r *Reader) SeekRune(value rune) bool
- func (r *Reader) Space() bool
- func (r *Reader) String(value string) bool
- func (r *Reader) Token() Token
- type Separator
- type Skip
- type SkipMode
- type Source
- type Token
Constants ¶
const ( // AbortParse is paniced when a parse cannot continue. It is recovered at the // top level, to allow the errors to be cleanly returned to the caller. AbortParse = fault.Const("abort") )
Variables ¶
var (
// ParseErrorLimit is the maximum number of errors before a parse is aborted.
ParseErrorLimit = 10
)
Functions ¶
This section is empty.
Types ¶
type Branch ¶
type Branch struct {
// Children is the slice of child nodes for this Branch.
Children []Node
// contains filtered or unexported fields
}
Branch is a CST node that can have children.
type BranchParser ¶
BranchParser is a function that is passed to ParseBranch. It is handed the Branch to fill in and the Parser to fill it from, and must either succeed or add an error to the parser.
type CSTMap ¶
type CSTMap interface {
// SetCST is called to map an ast node to a CST node.
SetCST(ast interface{}, cst Node)
// CST is called to lookup the CST mapping for an AST node.
CST(ast interface{}) Node
}
CSTMap is the interface to an object into which ast<->cts mappings are stored.
type Error ¶
type Error struct {
// At is the parse fragment that was being processed when the error was encountered.
At Fragment
// Message is the message associated with the error.
Message string
// Stack is the captured stack trace at the point the error was noticed.
Stack []byte
}
Error represents the information that us useful in debugging a parse failure.
func Parse ¶
func Parse(root BranchParser, filename, data string, skip Skip, m CSTMap) []Error
Parse is the main entry point to the parse library. Given a root parse function, the input string and the Skip controller, it builds and initializes a Parser, runs the root using it, verifies it worked correctly and then returns the errors generated if any.
type ErrorList ¶
type ErrorList []Error
ErrorList is a convenience type for managing lists of errors.
type Fragment ¶
type Fragment interface {
// Token returns the underlying token of this node.
Token() Token
// Write is used to write the underlying token out to the writer.
WriteTo(io.Writer) error
}
Fragment is a component of a cst that is backed by a token. This includes Nodes and all forms of space and comment.
func NewFragment ¶
type Leaf ¶
type Leaf struct {
// contains filtered or unexported fields
}
Leaf nodes are part of the cst that cannot have child nodes, they represent a single token from the input.
type LeafParser ¶
LeafParser is a function that is passed to ParseLeaf. It is handed the Leaf to fill in and the Parser to fill it from, and must either succeed or add an error to the parser.
type Node ¶
type Node interface {
Fragment
// Parent returns the Branch that this node is under.
Parent() *Branch
// Prefix returns the set of skippable fragments associated with this Node
// that precede it in the stream. Association is defined by the Skip function
// in use.
Prefix() Separator
// AddPrefix adds more fragments to the Prefix list.
AddPrefix(Separator)
// Suffix returns the set of skippable fragments associated with this Node
// that follow it in the stream. Association is defined by the Skip function
// in use, the default is until the end of the line.
Suffix() Separator
// AddSuffix adds more fragments to the Suffix list.
AddSuffix(Separator)
}
Node is a Fragment in a cst that represents unskipped tokens.
type NumberKind ¶
type NumberKind uint8
NumberKind is a type used by Reader.Numeric for identifying various kinds of numbers.
const ( // No number was found. NotNumeric NumberKind = iota // A decimal number. Decimal // An octal number, starting with "0". PS: A lone "0" is classified as octal. Octal // A hexadecimal number, starting with "0x". Hexadecimal // A floating point number: "123.456". Whole and the fractional parts are optional (but // not both at the same time). Floating // A floating point number in scientific notation: "123.456e±789". The fractional part, // the dot and the exponent sign are all optional. Scientific )
type Parser ¶
type Parser struct {
Reader // The token reader for this parser.
CSTMap // A map of cst nodes to ast nodes
Errors ErrorList // The set of errors generated during the parse.
// contains filtered or unexported fields
}
Parser contains all the context needed while parsing. They are built for you by the Parse function.
func (*Parser) Error ¶
Error adds a new error to the parser error list. It will attempt to consume a token from the reader to act as a place holder, and also to ensure progress is made in the presence of errors.
func (*Parser) ErrorAt ¶
ErrorAt is like Error, except because it is handed a fragment, it will not try to consume anything itself.
func (*Parser) Expected ¶
Expected is a wrapper around p.ErrorAt for the very common case of an unexpected input. It uses value as the expected input, and parses a token of the stream for the unexpected actual input.
func (*Parser) Extend ¶
func (p *Parser) Extend(ast interface{}, do BranchParser)
Extend inserts a new branch between ast and its parent, and calls do() with the newly insterted branch.
Extend will transform:
n.parent ──> n
to:
n.parent ──> b ──> n
where b is passed as the second argument to do().
func (*Parser) ParseBranch ¶
func (p *Parser) ParseBranch(cst *Branch, do BranchParser)
ParseBranch adds a new Branch to cst and then calls the do function to parse the branch. This is called recursively to build the node tree.
func (*Parser) ParseLeaf ¶
func (p *Parser) ParseLeaf(cst *Branch, do LeafParser)
ParseLeaf adds a new Leaf to cst and then calls the do function to parse the Leaf. If do is nil, a leaf will be built with the current unconsumed input.
type Reader ¶
type Reader struct {
Source *Source // The source being parsed.
// contains filtered or unexported fields
}
Reader is the interface to an object that converts a rune array into tokens.
func (*Reader) AlphaNumeric ¶
AlphaNumeric moves past anything that starts with a letter or underscore, and consists of letters, numbers or underscores. It returns true if the pattern was matched, false otherwise.
func (*Reader) EOL ¶
EOL moves the cursor past the EOL and returns true if the cursor is at the end of a line (\n or \r\n), otherwise EOL returns false and does nothing.
func (*Reader) GuessNextToken ¶
GuessNextToken attempts to do a general purpose consume of a single arbitrary token from the stream. It is used by error handlers to indicate where the error occurred. It guarantees that if the stream is not finished, it will consume at least one character.
func (*Reader) IsEOL ¶
IsEOL returns true when the cursor is at the end of a line (\n or \r\n). IsEOL does not move the cursor.
func (*Reader) NotSpace ¶
Space skips over any non whitespace, returning true if it advanced the cursor.
func (*Reader) Numeric ¶
func (r *Reader) Numeric() NumberKind
Numeric tries to move past the common number pattern. It returns a constant of type NumberKind describing the kind of number it found.
func (*Reader) Rollback ¶
func (r *Reader) Rollback()
Rollback sets the cursor back to the last consume point.
func (*Reader) Rune ¶
Rune advances and returns true if the next rune after the cursor matches value.
func (*Reader) SeekRune ¶
SeekRune advances the cursor until either the value is found or the end of stream is reached. It returns true if it found value, false otherwise.
type Separator ¶
type Separator []Fragment
Separator is a list type to manage fragments that were skipped.
type Skip ¶
Skip is the function used to skip separating tokens. A separating token is one where, as far as the parser is concerned, the tokens do not exist, even though the tokens may have been necessary to separate the lexical tokens (whitespace), or carry useful information (comments).
type SkipMode ¶
type SkipMode int
const ( // SkipPrefix is the skip mode that skips tokens that are associated with // the following lexically relevant token. This is mostly important for // comment association. SkipPrefix SkipMode = iota // SkipSuffix is the skip mode that skips tokens that are associated with // the preceding lexically relevant token. This is mostly important for // comment association. SkipSuffix )
type Source ¶
type Source struct {
Filename string // The path which was used to load the source (if any).
Runes []rune // The full content of the source file.
}
func (Source) RelativeFilename ¶
RelativeFilename returns the filename relative to the current working directory.
type Token ¶
type Token struct {
Source *Source // The source object this token is from (including the full rune array).
Start int // The start of the token in the full rune array.
End int // One past the end of the token.
}
A Token represents the smallest consumed unit input.
func (Token) Cursor ¶
Cursor is used to calculate the line and column of the start of the token. It may be very expensive to call, and is intended to be used sparingly in producing human readable error messages only.
Source Files
Directories