parseutil

Documentation

Index

• func IsWhitespace(char byte) bool
• func MaybeTokenizeSymbol(reader *LocationReader, sortedSymbols Symbols) (*Symbol, Location, error)
• func StripLeadingWhitespaces(reader *LocationReader) error
• func StripLeadingWhitespacesAndComments(reader *LocationReader) error
• type Location
  • func MaybeTokenizeCharacter(reader *LocationReader) (string, Location, error)
  • func MaybeTokenizeIdentifier(reader *LocationReader) (string, Location, error)
  • func (l Location) ShortString() string
  • func (l Location) String() string
• type LocationReader
  • func NewLocationReader(fileName string, reader io.Reader) *LocationReader
  • func (r *LocationReader) Peek(n int) ([]byte, error)
  • func (r *LocationReader) Read(buf []byte) (int, error)
  • func (r *LocationReader) ReadByte() (byte, error)
  • func (r *LocationReader) ReadString(delim byte) (string, error)
• type Symbol
• type Symbols
  • func (s Symbols) Len() int
  • func (s Symbols) Less(i int, j int) bool
  • func (s Symbols) Swap(i int, j int)

Functions

func IsWhitespace

func IsWhitespace(char byte) bool

func MaybeTokenizeSymbol

func MaybeTokenizeSymbol(
	reader *LocationReader,
	sortedSymbols Symbols) (
	*Symbol,
	Location,
	error)

func StripLeadingWhitespaces

func StripLeadingWhitespaces(reader *LocationReader) error

Strip all leading whitespaces

func StripLeadingWhitespacesAndComments

func StripLeadingWhitespacesAndComments(reader *LocationReader) error

Strip all leading whitespaces as well as golang style comments (i.e., /* */ and //)

Types

type Location

type Location struct {
	FileName string
	Line     int // one-base
	Column   int // zero-base
}

maybe track end line / end column?

func MaybeTokenizeCharacter

func MaybeTokenizeCharacter(reader *LocationReader) (string, Location, error)

If the reader's leading bytes are ascii character of the form 'x' (or '\t' for escaped characters), pop those bytes off the reader and return the value. Otherwise, return a nil slice.

func MaybeTokenizeIdentifier

func MaybeTokenizeIdentifier(reader *LocationReader) (string, Location, error)

If the reader's leading bytes are identifer of the form [a-zA-Z_]\w* , pop those bytes off the reader and return the value. Otherwise, return a nil slice.

func (Location) ShortString

func (l Location) ShortString() string

func (Location) String

func (l Location) String() string

type LocationReader

type LocationReader struct {
	Location
	// contains filtered or unexported fields
}

Not efficient, but probably good enough

func NewLocationReader

func NewLocationReader(fileName string, reader io.Reader) *LocationReader

func (*LocationReader) Peek

func (r *LocationReader) Peek(n int) ([]byte, error)

func (*LocationReader) Read

func (r *LocationReader) Read(buf []byte) (int, error)

func (*LocationReader) ReadByte

func (r *LocationReader) ReadByte() (byte, error)

func (*LocationReader) ReadString

func (r *LocationReader) ReadString(delim byte) (string, error)

type Symbol

type Symbol struct {
	Value string // can't be empty string
	Id    int
}

type Symbols

type Symbols []Symbol

func (Symbols) Len

func (s Symbols) Len() int

func (Symbols) Less

func (s Symbols) Less(i int, j int) bool

longer symbol before shorter symbol symbol value as tie break

func (Symbols) Swap

func (s Symbols) Swap(i int, j int)