token

Documentation

Index

• Constants
• type Token
  • func Lookup(lit string) Token
  • func (tok Token) IsLiteral() bool
  • func (tok Token) Precedence() int
  • func (i Token) String() string

Constants

const (
	LowestPrec  = 0 // non-operators
	UnaryPrec   = 6
	HighestPrec = 7
)

A set of constants for precedence-based expression parsing. Non-operators have lowest precedence, followed by operators starting with precedence 1 up to unary operators. The highest precedence serves as "catch-all" precedence for selector, indexing, and other operator and delimiter tokens.

Types

type Token

type Token int

Token encodes a scanned token.

const (
	// special tokens.
	ILLEGAL Token = iota // #illegal
	EOF                  // #eof
	IDENT                // #ident
	COMMENT              // #comment

	// keywords.
	BREAK       // break
	CASE        // case
	CHAN        // chan
	CONST       // const
	CONTINUE    // continue
	DEFAULT     // default
	DEFER       // defer
	ELSE        // else
	FALLTHROUGH // fallthrough
	FOR         // for
	FUNC        // func
	GO          // go
	GOTO        // goto
	IF          // if
	IMPORT      // import
	INTERFACE   // interface
	MAP         // map
	PACKAGE     // package
	RANGE       // range
	RETURN      // return
	SELECT      // select
	STRUCT      // struct
	SWITCH      // switch
	TYPE        // type
	VAR         // var

	INT    // <int>
	FLOAT  // <float>
	STRING // <string>

	SEMICOLON // ;
	COLON     // :
	PERIOD    // .
	ELLIPSIS  // ...
	COMMA     // ,
	DEFINE    // :=
	ARROW     // <-

	INC // ++
	DEC // --

	ASSIGN // =
	EQL    // ==
	NEQ    // !=
	NOT    // !

	AND_NOT // &^
	AND     // &
	XOR     // ^
	OR      // |
	LAND    // &&
	LOR     // ||

	LPAREN // (
	RPAREN // )
	LBRACK // [
	RBRACK // ]
	LBRACE // {
	RBRACE // }

	LSS // <
	LEQ // <=
	SHL // <<
	GTR // >
	GEQ // >=
	SHR // >>

	QUO // /
	MUL // *
	ADD // +
	SUB // -
	REM // %

	QUO_ASSIGN     // /=
	MUL_ASSIGN     // *=
	ADD_ASSIGN     // +=
	SUB_ASSIGN     // -=
	AND_NOT_ASSIGN // &^=
	AND_ASSIGN     // &=
	OR_ASSIGN      // |=
	SHR_ASSIGN     // >>=
	SHL_ASSIGN     // <<=
	REM_ASSIGN     // %=
	XOR_ASSIGN     // ^=
)

func Lookup

func Lookup(lit string) Token

Lookup determines if the literal is a keyword token, or else an identifier.

func (Token) IsLiteral

func (tok Token) IsLiteral() bool

IsLiteral returns true if the token represents a literal token.

func (Token) Precedence

func (tok Token) Precedence() int

Precedence returns the operator precedence of the binary operator op. If op is not a binary operator, the result is LowestPrecedence.

func (Token) String

func (i Token) String() string