scanner

Documentation

Index

• type Context
• type IndentState
• type InvalidTokenError
  • func ErrInvalidToken(tk *token.Token) *InvalidTokenError
  • func (e *InvalidTokenError) Error() string
• type Scanner
  • func (s *Scanner) Init(text string)
  • func (s *Scanner) Scan() (token.Tokens, error)

Types

type Context

type Context struct {
	// contains filtered or unexported fields
}

Context context at scanning

type IndentState

type IndentState int

IndentState state for indent

const (
	// IndentStateEqual equals previous indent
	IndentStateEqual IndentState = iota
	// IndentStateUp more indent than previous
	IndentStateUp
	// IndentStateDown less indent than previous
	IndentStateDown
	// IndentStateKeep uses not indent token
	IndentStateKeep
)

type InvalidTokenError

type InvalidTokenError struct {
	Token *token.Token
}

func ErrInvalidToken

func ErrInvalidToken(tk *token.Token) *InvalidTokenError

func (*InvalidTokenError) Error

func (e *InvalidTokenError) Error() string

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner holds the scanner's internal state while processing a given text. It can be allocated as part of another data structure but must be initialized via Init before use.

func (*Scanner) Init

func (s *Scanner) Init(text string)

Init prepares the scanner s to tokenize the text src by setting the scanner at the beginning of src.

func (*Scanner) Scan

func (s *Scanner) Scan() (token.Tokens, error)

Scan scans the next token and returns the token collection. The source end is indicated by io.EOF.