lexer

Documentation

Overview

Package lexer handles the interpreter's first phase of operation. Lexer takes some input and interpret it into 'tokens'.

Smoosh (and Monkey) builds up its input into an ordered list of tokens without attmpting to make sense of the structure and relationship. These tokens are operators, keywords, identifiers or values (strings, numbers, etc).

Index

• type Lexer
  • func New(input string) *Lexer
  • func (l *Lexer) NextToken() token.Token

Types

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer tokenises input

func New

func New(input string) *Lexer

New creates and initialises a new lexer

func (*Lexer) NextToken

func (l *Lexer) NextToken() token.Token

NextToken attempts to find the next token in the program's input