scanner

Documentation

Index

• type Scanner
  • func New(r io.Reader) *Scanner
  • func (s *Scanner) Err() error
  • func (s *Scanner) Pos() (line, col int)
  • func (s *Scanner) Scan() bool
  • func (s *Scanner) Tok() Token
• type Token
• type TokenType
  • func (t TokenType) String() string

Examples

• Scanner

Types

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

A Scanner tokenizes runes from an io.Reader.

Example

package main

import (
	"io"

	"github.com/DeedleFake/wdte/scanner"
)

var r io.Reader

func main() {
	s := scanner.New(r)
	for s.Scan() {
		/* Do something with s.Tok(). */
	}
	if err := s.Err(); err != nil {
		panic(err)
	}
}

func New

func New(r io.Reader) *Scanner

New returns a new Scanner that reads from r.

func (*Scanner) Err

func (s *Scanner) Err() error

Err returns the error that stopped the scanner, if any.

func (*Scanner) Pos

func (s *Scanner) Pos() (line, col int)

Pos returns the line and column of the input that the scanner is currently on.

func (*Scanner) Scan

func (s *Scanner) Scan() bool

Scan reads the next token from the underlying io.Reader. If a token was successfully read, it returns true. It is designed to be used in a loop, similarly to bufio.Scanner's API.

func (*Scanner) Tok

func (s *Scanner) Tok() Token

Tok returns the latest token scanned. If there was an error or a token hasn't been scanned yet, its return value is undefined.

type Token

type Token struct {
	Line, Col int
	Type      TokenType
	Val       interface{}
}

A Token is a usable element parsed from a string.

type TokenType

type TokenType uint

TokenType is the type of a token.

const (
	Invalid TokenType = iota // nolint
	Number
	String
	ID
	Keyword
	EOF
)

func (TokenType) String

func (t TokenType) String() string