lexer

package
v11.1.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 18, 2018 License: BSD-3-Clause, BSD-3-Clause Imports: 1 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// Begin is the symbol which lexer should scan forward to.
	Begin = '{' // token.LBRACE
	// End is the symbol which lexer should stop scanning.
	End = '}' // token.RBRACE
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer helps us to read/scan characters of a source and resolve their token types.

func New

func New(src string) *Lexer

New takes a source, series of chars, and returns a new, ready to read from the first letter, lexer.

func (*Lexer) NextDynamicToken

func (l *Lexer) NextDynamicToken() (t token.Token)

NextDynamicToken doesn't cares about the grammar. It reads numbers or any unknown symbol, it's being used by parser to skip all characters between parameter function's arguments inside parenthesis, in order to allow custom regexp on the end-language too.

It moves the cursor forward.

func (*Lexer) NextToken

func (l *Lexer) NextToken() (t token.Token)

NextToken returns the next token in the series of characters. It can be a single symbol, a token type or a literal. It's able to return an EOF token too.

It moves the cursor forward.

func (*Lexer) PeekNextTokenType

func (l *Lexer) PeekNextTokenType() token.Type

PeekNextTokenType returns only the token type of the next character and it does not move forward the cursor. It's being used by parser to recognise empty functions, i.e `even()` as valid functions with zero input arguments.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL