lexer

Documentation

Index

• type Lex
  • func New(r io.Reader) *Lex
  • func (l *Lex) IsTokenString() bool
  • func (l *Lex) Next() (string, error)
  • func (lex *Lex) Scan()
  • func (l *Lex) TokenAll() (tokens []string, err error)

Types

type Lex

type Lex struct {
	*scanner.Scanner
	Token rune
}

Lex is lexer for gigue. Lex provides token for scheme.

func New

func New(r io.Reader) *Lex

New returns new lexer

func (*Lex) IsTokenString

func (l *Lex) IsTokenString() bool

IsTokenString check if token is string. It's use for get unquoted string in parser.

func (*Lex) Next

func (l *Lex) Next() (string, error)

Next returns next scheme token Parser calls this Next() iteratively for fetching token.

func (*Lex) Scan

func (lex *Lex) Scan()

Scan makes scan

func (*Lex) TokenAll

func (l *Lex) TokenAll() (tokens []string, err error)

TokenAll returns all token generated by lexer. It is useful for testing lexer.