lexer

Documentation

Overview

Package lexer is the lexer for the arbor project. The lexer can either be run asyncrounously or syncrounously

Index

• func Lex(in io.Reader) func() Lexeme
• func LexAsync(in io.Reader) chan internal.Lexeme
• func NewLexer(in io.Reader) *internal.Lexer
• func RunMachine(lex *internal.Lexer)
• type Definitions
  • func (d *Definitions) Lex(r io.Reader) (participle.Lexer, error)
  • func (d *Definitions) Symbols() map[string]rune
• type Lexeme
  • func (lexeme Lexeme) String() string
• type PLexer
  • func (p *PLexer) Next() (participle.Token, error)

Functions

func Lex

func Lex(in io.Reader) func() Lexeme

Lex lexes the input syncrounously by returning a function that will listen for input on the channel else it will call the state function.

This function returns a function that can be used to get the next lexeme

func LexAsync

func LexAsync(in io.Reader) chan internal.Lexeme

LexAsync lexs the input asyncronously. This returns a channel that will contain the identified lexemes

func NewLexer

func NewLexer(in io.Reader) *internal.Lexer

NewLexer creates and returns a new lexer instance

func RunMachine

func RunMachine(lex *internal.Lexer)

RunMachine runs the lexer state machine

Types

type Definitions

type Definitions struct {
}

func (*Definitions) Lex

func (d *Definitions) Lex(r io.Reader) (participle.Lexer, error)

func (*Definitions) Symbols

func (d *Definitions) Symbols() map[string]rune

type Lexeme

type Lexeme internal.Lexeme

Lexeme is the public lexeme for everyone

func (Lexeme) String

func (lexeme Lexeme) String() string

type PLexer

type PLexer struct {
	// contains filtered or unexported fields
}

func (*PLexer) Next

func (p *PLexer) Next() (participle.Token, error)