lexer

Documentation

Overview

Package lexer implements a lexical analyzer used by the Elk interpreter.

Lexer expects a slice of bytes containing Elk source code analyses it and returns a stream of tokens/tokens.

Tokens are returned on demand.

Index

• func Colorize(source string) string
• func Lex(source string) []*token.Token
• type Lexer
  • func New(source string) *Lexer
  • func NewWithName(sourceName string, source string) *Lexer
  • func (l *Lexer) Next() *token.Token

Functions

func Colorize

func Colorize(source string) string

Lex the given string and construct a new one colouring every token.

func Lex

func Lex(source string) []*token.Token

Lex the given string and return a slice containing all the tokens.

Types

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

Holds the current state of the lexing process.

func New

func New(source string) *Lexer

Instantiates a new lexer for the given source code.

func NewWithName

func NewWithName(sourceName string, source string) *Lexer

Same as New but lets you specify the path to the source code file.

func (*Lexer) Next

func (l *Lexer) Next() *token.Token

Returns the next token or an error if the input is malformed.