lexer

Documentation

Index

• type Lexer
  • func New(input string) *Lexer
  • func (l *Lexer) Index() int
  • func (l *Lexer) Next() token.Token
  • func (l *Lexer) Reposition(idx int)

Types

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer converts an input string into its constituent tokens.

func New

func New(input string) *Lexer

New creates a lexer from the given input.

func (*Lexer) Index

func (l *Lexer) Index() int

Index returns the current position of the lexer.

func (*Lexer) Next

func (l *Lexer) Next() token.Token

Next returns the next token from the input and moves the current position of the lexer ahead.

If encountering the end of the input, token.EOF is returned.

func (*Lexer) Reposition

func (l *Lexer) Reposition(idx int)

Reposition positions the lexer to the given index.