lexer

Documentation

Overview

Package lexer implements the lexical scanner for the Suneido language

Index

• func IsIdentifier(s string) bool
• type Item
  • func (it *Item) String() string
• type Lexer
  • func NewLexer(src string) *Lexer
  • func NewQueryLexer(src string) *Lexer
  • func (lxr *Lexer) Ahead(i int) Item
  • func (lxr *Lexer) AheadSkip(i int) Item
  • func (lxr *Lexer) Dup() *Lexer
  • func (lxr *Lexer) Next() Item
  • func (lxr *Lexer) Position() int
  • func (lxr *Lexer) Remainder() string
  • func (lxr *Lexer) Source() string

Functions

func IsIdentifier

func IsIdentifier(s string) bool

Types

type Item

type Item struct {
	Text  string
	Pos   int32
	Token tok.Token
}

Item is the return value from Lexer.Next

func (*Item) String

func (it *Item) String() string

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer implements the lexical scanner for Suneido It is designed so the sequence of values returned forms the complete source.

func NewLexer

func NewLexer(src string) *Lexer

NewLexer returns a new Lexer

func NewQueryLexer

func NewQueryLexer(src string) *Lexer

func (*Lexer) Ahead

func (lxr *Lexer) Ahead(i int) Item

Ahead provides lookahead, 0 is the next item Items are buffered so they can be used by Next

func (*Lexer) AheadSkip

func (lxr *Lexer) AheadSkip(i int) Item

AheadSkip provides lookahead like Ahead but skips Whitespace, Newline, and Comment

func (*Lexer) Dup

func (lxr *Lexer) Dup() *Lexer

func (*Lexer) Next

func (lxr *Lexer) Next() Item

Next returns the next Item

func (*Lexer) Position

func (lxr *Lexer) Position() int

Position will be out of sync if using Ahead

func (*Lexer) Remainder

func (lxr *Lexer) Remainder() string

Remainder is used by ParseAdmin to get view definitions

func (*Lexer) Source

func (lxr *Lexer) Source() string