jalex

README

jaLex

JaLex is an extensible lexer built for educational purposes.

Inspiration

Usage

Please refer to lexer_test.go for a simple usage example.

Documentation

Index

• Constants
• type Item
  • func (i *Item) String() string
• type ItemType
• type Lexer
  • func NewLexer(name string, initial StateFn) (*Lexer, error)
  • func (l *Lexer) Run(text string) chan Item
• type State
  • func (s *State) Back()
  • func (s *State) Emit(t ItemType)
  • func (s *State) Errorf(format string, args ...interface{}) StateFn
  • func (s *State) Next() rune
  • func (s *State) Skip()
• type StateFn

Constants

const EOF = -1

Types

type Item

type Item struct {
	T ItemType
	V string
}

Item represents the structure sent to the client(e.g Parser).

func (*Item) String

func (i *Item) String() string

type ItemType

type ItemType string

ItemType represents the type of lexed item.

const (
	ItemUnknown ItemType = "unknown"
	ItemError   ItemType = "error"
	ItemEOF     ItemType = "EOF"
)

Common items.

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer holds the required information to run several state machines.

func NewLexer

func NewLexer(name string, initial StateFn) (*Lexer, error)

NewLexer creates a new Lexer with the given name and an initial state function.

func (*Lexer) Run

func (l *Lexer) Run(text string) chan Item

Run runs the state machine for a given text string and returns a channel of Item.

type State

type State struct {
	Text    string
	Start   int // the beginning cursor position of a selection of the Text field
	Current int // the current cursor position of a selection of the Text field
	Width   int
	// contains filtered or unexported fields
}

State holds the current state of the currently running state machine.

func (*State) Back

func (s *State) Back()

Back moves the current cursor position backward by one rune.

func (*State) Emit

func (s *State) Emit(t ItemType)

Emit send an item on the items channel and move s.start to s.current.

func (*State) Errorf

func (s *State) Errorf(format string, args ...interface{}) StateFn

Errorf send an ItemError on the items channel and returns an nil StateFn.

func (*State) Next

func (s *State) Next() rune

Next moves the current cursor position forward by one rune.

func (*State) Skip

func (s *State) Skip()

Skip skips the current selection of text.

type StateFn

type StateFn func(*State) StateFn

StateFn function represents one state function.