Documentation
¶
Overview ¶
Package token defines constants representing the lexical tokens of the Go programming language and basic operations on tokens (printing, predicates).
Index ¶
- Constants
- func IsBreakerKeyword(ident string) bool
- func IsClosingKeyword(ident string) bool
- func IsExported(name string) bool
- func IsIdentifier(name string) bool
- func IsKeyword(name string) bool
- func IsSplittingKeyword(ident string) bool
- func ToWaTok(tok Token) watoken.Token
- type Token
- func (tok Token) IsKeyword() bool
- func (tok Token) IsLiteral() bool
- func (tok Token) IsOperator() bool
- func (op Token) Precedence() int
- func (tok Token) String() string
- func (tok Token) WaGoKeykword() string
- func (tok Token) WaGoString() string
- func (tok Token) WaZhString() string
- func (tok Token) ZhKeykword() string
Constants ¶
const ( LowestPrec = 0 // non-operators UnaryPrec = 6 HighestPrec = 7 )
A set of constants for precedence-based expression parsing. Non-operators have lowest precedence, followed by operators starting with precedence 1 up to unary operators. The highest precedence serves as "catch-all" precedence for selector, indexing, and other operator and delimiter tokens.
Variables ¶
This section is empty.
Functions ¶
func IsBreakerKeyword ¶
func IsClosingKeyword ¶
func IsIdentifier ¶
IsIdentifier reports whether name is a Go identifier, that is, a non-empty string made up of letters, digits, and underscores, where the first character is not a digit. Keywords are not identifiers.
func IsSplittingKeyword ¶
Types ¶
type Token ¶
type Token int
Token is the set of lexical tokens of the Go programming language.
const ( // Special tokens ILLEGAL Token = iota EOF COMMENT // Identifiers and basic type literals // (these tokens stand for classes of literals) IDENT // main INT // 12345 FLOAT // 123.45 IMAG // 123.45i CHAR // 'a' STRING // "abc" // Operators and delimiters ADD // + SUB // - MUL // * QUO // / REM // % AND // & OR // | XOR // ^ SHL // << SHR // >> AND_NOT // &^ ADD_ASSIGN // += SUB_ASSIGN // -= MUL_ASSIGN // *= QUO_ASSIGN // /= REM_ASSIGN // %= AND_ASSIGN // &= OR_ASSIGN // |= XOR_ASSIGN // ^= SHL_ASSIGN // <<= SHR_ASSIGN // >>= AND_NOT_ASSIGN // &^= LAND // && LOR // || INC // ++ DEC // -- EQL // == LSS // < GTR // > ASSIGN // = NOT // ! NEQ // != LEQ // <= GEQ // >= DEFINE // := ELLIPSIS // ... LPAREN // ( LBRACK // [ LBRACE // { COMMA // , PERIOD // . RPAREN // ) RBRACK // ] RBRACE // } SEMICOLON // ; COLON // : ARROW // => TK_左方 // 【 TK_右方 // 】 TK_冒号 // ： TK_句号 // 。 TK_顿号 // 、 TK_叹号 // ！ TK_点号 // · TK_左书 // 《 TK_右书 // 》 TK_逗号 // ， // 类型 TK_数 TK_浮点数 TK_布尔 TK_引于 TK_归于 TK_若 TK_则 TK_否则 TK_设 TK_之 TK_行 TK_自 TK_至 TK_有 TK_当 TK_为 // Keywords BREAK CASE CONST CONTINUE DEFAULT DEFER ELSE FOR FUNC IF IMPORT INTERFACE MAP PACKAGE RANGE RETURN STRUCT SWITCH TYPE VAR // reserved keywords CLASS ENUM )
The list of tokens.
func (Token) IsKeyword ¶
IsKeyword returns true for tokens corresponding to keywords; it returns false otherwise.
func (Token) IsLiteral ¶
IsLiteral returns true for tokens corresponding to identifiers and basic type literals; it returns false otherwise.
func (Token) IsOperator ¶
IsOperator returns true for tokens corresponding to operators and delimiters; it returns false otherwise.
func (Token) Precedence ¶
Precedence returns the operator precedence of the binary operator op. If op is not a binary operator, the result is LowestPrecedence.
func (Token) String ¶
String returns the string corresponding to the token tok. For operators, delimiters, and keywords the string is the actual token character sequence (e.g., for the token ADD, the string is "+"). For all other tokens the string corresponds to the token constant name (e.g. for the token IDENT, the string is "IDENT").
Source Files