lexer

Documentation

Overview

Package lexer provides a lexer that parses templates into a sequence of lexical tokens. The tokens are then to be further processed by a parser into an abstract syntax tree.

Index

• Constants
• func IsParseError(e error) bool
• type Lexer
  • func New(r io.Reader, opts ...Opt) *Lexer
  • func (l *Lexer) Tokens() (<-chan *Token, chan<- struct{})
• type Opt
  • func WithStartInCodeMode() Opt
• type Token
  • func (t Token) String() string
• type TokenType
  • func (t TokenType) String() string

Constants

const (
	// EOF is the token type returned when a lexer has reached the end of its input.
	EOF = iota

	// Illegal is the token type used for unknown tokens.
	Illegal

	// True is the token type used for a literal true bool value.
	True

	// True is the token type used for a literal false bool value.
	False

	// Nil is the token type used for a literal nil value.
	Nil

	// Ident is the token type used for an identifier.
	Ident

	// Int is the token type used for a signed integer value.
	Int

	// String is the token type used for a literal string value.
	String

	// Assign is the token type used for the assignment character '='.
	// If the character is part of the sequences "==", "!=", "<=", or ">=",
	// the token types Equal, NotEqual, LessOrEqual, or GreaterOrEqual are used for
	// the whole sequence instead, respectively.
	Assign

	// Bang is the token type used for the bang character '!'.
	Bang

	// Plus is the token type used for the plus character '+'.
	Plus

	// Minus is the token type used for the minus character '-'.
	Minus

	// Asterisk is the token type used for the asterisk character '*'.
	Asterisk

	// Slash is the token type used for the slash character '/'.
	Slash

	// Mod is the token type used for the modulo character '%'.
	Mod

	// Equal is the token type used for the equals comparison character sequence "==".
	Equal

	// NotEqual is the token type used for the not equals comparison character sequence "!=".
	NotEqual

	// LessThan is the token type used for the less than character '<'. If the character is followed by
	// the equals character '=', the token type LessOrEqual is used for the whole sequence instead.
	LessThan

	// GreaterThan is the token type used for the greater than character '>'. If the character is followed by
	// the equals character '=', the token type GreaterOrEqual is used for the whole sequence instead.
	GreaterThan

	// LessOrEqual is the token type used for the less or equals character sequence "<=".
	LessOrEqual

	// GreaterOrEqual is the token type used for the greater or equals character sequence ">=".
	GreaterOrEqual

	// Or is the token type used for the boolean OR character sequence "||".
	Or

	// And is the token type used for the boolean AND character sequence "&&".
	And

	// Dot is the token type used for the dot character '.'.
	Dot

	// Comma is the token type used for the modulo character '%'.
	Comma

	// Colon is the token type used for the colon character ':'.
	Colon

	// LeftParen is the token type used for the left parenthesis character '('.
	LeftParen

	// RightParen is the token type used for the right parenthesis character ')'.
	RightParen

	// LeftBracket is the token type used for the left square bracket character '['.
	LeftBracket

	// RightBracket is the token type used for the right square bracket character ']'.
	RightBracket

	// LeftBrace is the token type used for the left curly brace character '{'.
	LeftBrace

	// RightBrace is the token type used for the right curly brace character '}'.
	RightBrace

	// Let is the token type used for the let keyword.
	Let

	// If is the token type used for the if keyword.
	If

	// Else is the token type used for the else keyword.
	Else

	// ElseIf is the token type used for the elseif keyword.
	ElseIf

	// End is the token type used for the end keyword.
	End

	// For is the token type used for the for keyword.
	For

	// Break is the token type used for the break keyword.
	Break

	// Continue is the token type used for the continue keyword.
	Continue

	// In is the token type used for the in keyword.
	In

	// Capture is the token type used for the capture keyword.
	Capture

	// Literal is the token type used for literal strings in the template, outside of code blocks.
	Literal

	Error
)

Functions

func IsParseError

func IsParseError(e error) bool

Types

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer parses a series of statements or expressions, a template, from a reader and returns them as a sequence of lexical tokens.

func New

func New(r io.Reader, opts ...Opt) *Lexer

New returns a new lexer, configured with opts, that reads a template from r.

func (*Lexer) Tokens

func (l *Lexer) Tokens() (<-chan *Token, chan<- struct{})

Tokens reads from the lexer's input and writes a sequence of tokens into tCh. If an error occurs when producing tokens, the error is associated with the next token in the channel. Token production stops when there was an error, or when the done channel is closed.

type Opt

type Opt func(l *Lexer)

Opt is the type of a function that configures an option of l.

func WithStartInCodeMode

func WithStartInCodeMode() Opt

WithStartInCodeMode configures a lexer to start in code mode. The default is to start in literal mode. If the lexer starts in literal mode, code blocks (<% %>) must be used to switch to code mode.

type Token

type Token struct {
	Type    TokenType
	Literal string
	Line    int
	Col     int
	Err     error
}

func (Token) String

func (t Token) String() string

type TokenType

type TokenType int

func (TokenType) String

func (t TokenType) String() string