scan

Documentation

Overview

Package scan implements a scanner for the SubC language. It takes an interface that can return bytes and transforms it into tokens.

Index

• Variables
• type Config
• type ErrorList
  • func (l *ErrorList) Add(msg ErrorMessage)
  • func (l *ErrorList) Err() error
  • func (l *ErrorList) Error() string
• type ErrorMessage
  • func (e ErrorMessage) Error() string
• type Interface
• type Loader
• type Reader
  • func OpenFile(name string) (Reader, error)
  • func StringReader(pos scanner.Position, src string, lockedPos bool) Reader
• type Scanner
  • func New(conf Config, name string, r Reader) *Scanner
  • func (l *Scanner) Close() error
  • func (l *Scanner) Scan() Token
• type Span
  • func (s Span) IsValid() bool
• type Token
  • func (op Token) IsLiteral() bool
  • func (op Token) Precedence() int
  • func (i Token) Span() Span
  • func (i Token) String() string
• type Type
  • func (t Type) String() string

Variables

var (
	NoSpan = Span{}             // A zero value for span, considered to be invalid span.
	NoPos  = scanner.Position{} // A zero value for position, considered to be an invalid position.
)

var DefaultConfig = Config{
	ApplyPreprocessor: true,
	Loader:            fileLoader{},
	MaxIncludes:       16,
	Macros: [][2]string{
		{"__SUBC__", ""},
		{"__" + runtime.GOOS + "__", ""},
		{"__DATE__", "\"" + time.Now().Format("Jan 2 2006") + "\""},
		{"__TIME__", "\"" + time.Now().Format("15:04:05") + "\""},
	},
	ScanComments: false,
}

DefaultConfig specifies a reasonable default for the scanner. It is intended to be extended by making a copy of this struct and then customizing some of the fields.

Types

type Config

type Config struct {
	ApplyPreprocessor bool        // applies the preprocessor upon encountering macros
	Loader            Loader      // the loader interface is used for loading include files
	MaxIncludes       int         // max number of nested includes during macro expansion before aborting
	IncludePaths      []string    // paths to look for include files
	Macros            [][2]string // macro definitions in the form of (macro, text expansion)
	ScanComments      bool        // scan comments as tokens when turned on, otherwise it is ignored
	// contains filtered or unexported fields
}

Config specifies the behavior of the scanner.

type ErrorList

type ErrorList struct {
	Messages    []ErrorMessage
	NumWarnings int
	NumErrors   int
}

ErrorList keeps a list of error messages

func (*ErrorList) Add

func (l *ErrorList) Add(msg ErrorMessage)

func (*ErrorList) Err

func (l *ErrorList) Err() error

Err returns an error to satisfy the error interface.

func (*ErrorList) Error

func (l *ErrorList) Error() string

type ErrorMessage

type ErrorMessage struct {
	Pos     scanner.Position
	Text    string
	Warning bool
}

ErrorMessage is a struct that contains the error message and whether or not if it is critical.

func (ErrorMessage) Error

func (e ErrorMessage) Error() string

type Interface

type Interface interface {
	Scan() Token
	Close() error
}

Interface defines an interface for scanning.

type Loader

type Loader interface {
	Open(name string) (Reader, error)
}

Loader is an interface for opening files that are used by the #include mechanism.

type Reader

type Reader interface {
	io.ByteReader
	io.Closer
	Pos() scanner.Position
	LockedPos() bool
}

Reader is an interface that provides input to the scanner.

func OpenFile

func OpenFile(name string) (Reader, error)

OpenFile returns a reader made from opening a file on the filesystem.

func StringReader

func StringReader(pos scanner.Position, src string, lockedPos bool) Reader

StringReader returns a reader out of a string.

type Scanner

type Scanner struct {
	Tokens chan Token // channel used for returning tokens.
	// contains filtered or unexported fields
}

Scanner is the scanner structure for scanning tokens.

func New

func New(conf Config, name string, r Reader) *Scanner

New returns a scanner for reading tokens using name as the filename identifier and r as a reader. It starts a goroutine and runs concurrently to get the next token which must be drained by user.

func (*Scanner) Close

func (l *Scanner) Close() error

Close drains the tokens from the channel, it is used for cleaning up.

func (*Scanner) Scan

func (l *Scanner) Scan() Token

Scan reads the next token from the channel.

type Span

type Span struct {
	Start scanner.Position
	End   scanner.Position
}

Span stores the beginning and end position boundary of a token or block.

func (Span) IsValid

func (s Span) IsValid() bool

type Token

type Token struct {
	Type Type
	Pos  scanner.Position
	Text string
}

func (Token) IsLiteral

func (op Token) IsLiteral() bool

IsLiteral returns whether or not an op was a literal

func (Token) Precedence

func (op Token) Precedence() int

Precedence returns the operator precedence of a given operator. The operators covered are not the full set, because some of them need more context like whether or not it is prefix/postfix/unary, If op is not an arithmetic or logical operator, it returns the lowest precedence.

func (Token) Span

func (i Token) Span() Span

func (Token) String

func (i Token) String() string

type Type

type Type int

Type is a token type.

const (
	EOF Type = iota
	Error
	Warning
	Comment
	Preprocessor
	Pragma

	Div
	Mul
	Mod
	Plus
	Minus
	Lsh
	Rsh
	Gt
	Geq
	Lt
	Leq
	Eq
	Neq
	And
	Xor
	Or
	Land
	Lor

	Arrow
	AndEq
	XorEq
	LshEq
	MinusEq
	ModEq
	OrEq
	PlusEq
	RshEq
	DivEq
	MulEq
	Assign
	Auto
	Break
	Case
	Char
	Colon
	Comma
	Const
	Continue
	Dec
	Default
	Do
	Dot
	Double
	Ellipsis
	Else
	Enum
	Extern
	Float
	For
	Goto
	Ident
	If
	Inc
	Inline
	Int
	Lbrace
	Lbrack
	Long
	Lparen
	Not
	Qmark
	Rbrace
	Rbrack
	Register
	Restrict
	Return
	Rparen
	Semi
	Short
	Signed
	Sizeof
	Static
	Struct
	Switch
	Typedef
	Negate
	Union
	Unsigned
	Void
	Volatile
	While

	Alignas
	Alignof
	Atomic
	Bool
	Complex
	Generic
	Imaginary
	Noreturn
	Static_assert
	Thread_local

	Number
	Rune
	String
)

All of the token types.

func (Type) String

func (t Type) String() string