scanner

Documentation

Index

• type ErrorHandler
• type Ruleset
• type Scanner
  • func (s *Scanner) Init(src []byte, err ErrorHandler, rules Ruleset)
  • func (s *Scanner) Pos() token.Position
  • func (s *Scanner) Scan() (pos int, tok token.Token, lit string)

Types

type ErrorHandler

type ErrorHandler func(pos token.Position, msg string)

An ErrorHandler may be provided to Scanner.Init. If a syntax error is encountered and a handler was installed, the handler is called with a position and an error message. The position points to the beginning of the offending token.

type Ruleset

type Ruleset struct {
	BracketsAreQuotes   bool
	BacktickIsQuotemark bool
	DoubleQuoteIsString bool

	DollarIsLetter bool
}

A scanner.Ruleset specifies the dialect specific tokenizing rules for a SQL dialect

type Scanner

type Scanner struct {

	// public state
	ErrorCount int // number of errors encountered
	// contains filtered or unexported fields
}

A Scanner holds the scanner's internal state.

func (*Scanner) Init

func (s *Scanner) Init(src []byte, err ErrorHandler, rules Ruleset)

Init prepares the scanner s to tokenize the text src by setting the scanner at the beginning of src.

Calls to Scan will invoke the error handler err if they encounter a syntax error and err is not nil. Also, for each error encountered, the Scanner field ErrorCount is incremented by one.

Note that Init may call err if there is an error in the first character of the file.

func (*Scanner) Pos

func (s *Scanner) Pos() token.Position

func (*Scanner) Scan

func (s *Scanner) Scan() (pos int, tok token.Token, lit string)

Scan scans the next token and returns the token position, the token, and its literal string if applicable. The source end is indicated by the EOS token.

If the returned token is a literal the literal string has the corresponding value.

If the returned token is a keyword, the literal string is the keyword.

If the returned token is an identifier, the literal string is the identifier.

If the returned token is a quoted identifier, the literal string is the identifier without the quotes.

If the returned token is invalid, the literal string is the offending character.

In all other cases, Scan returns an empty literal string.