token

package
v0.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 11, 2023 License: AGPL-3.0 Imports: 4 Imported by: 0

Documentation

Overview

Package token defines constants representing the lexical tokens of the Go programming language and basic operations on tokens (printing, predicates).

Index

Constants

View Source
const (
	LowestPrec  = 0 // non-operators
	UnaryPrec   = 6
	HighestPrec = 7
)

A set of constants for precedence-based expression parsing. Non-operators have lowest precedence, followed by operators starting with precedence 1 up to unary operators. The highest precedence serves as "catch-all" precedence for selector, indexing, and other operator and delimiter tokens.

Variables

This section is empty.

Functions

func IsBreakerKeyword

func IsBreakerKeyword(ident string) bool

func IsClosingKeyword

func IsClosingKeyword(ident string) bool

func IsExported

func IsExported(name string) bool

IsExported reports whether name is exported.

func IsIdentifier

func IsIdentifier(name string) bool

IsIdentifier reports whether name is a Go identifier, that is, a non-empty string made up of letters, digits, and underscores, where the first character is not a digit. Keywords are not identifiers.

func IsKeyword

func IsKeyword(name string) bool

IsKeyword reports whether name is a Go keyword, such as "func" or "return".

func IsSplittingKeyword

func IsSplittingKeyword(ident string) bool

func ToWaTok

func ToWaTok(tok Token) watoken.Token

Types

type Token

type Token int

Token is the set of lexical tokens of the Go programming language.

const (
	// Special tokens
	ILLEGAL Token = iota
	EOF
	COMMENT

	// Identifiers and basic type literals
	// (these tokens stand for classes of literals)
	IDENT  // main
	INT    // 12345
	FLOAT  // 123.45
	IMAG   // 123.45i
	CHAR   // 'a'
	STRING // "abc"

	// Operators and delimiters
	ADD // +
	SUB // -
	MUL // *
	QUO // /
	REM // %

	AND     // &
	OR      // |
	XOR     // ^
	SHL     // <<
	SHR     // >>
	AND_NOT // &^

	ADD_ASSIGN // +=
	SUB_ASSIGN // -=
	MUL_ASSIGN // *=
	QUO_ASSIGN // /=
	REM_ASSIGN // %=

	AND_ASSIGN     // &=
	OR_ASSIGN      // |=
	XOR_ASSIGN     // ^=
	SHL_ASSIGN     // <<=
	SHR_ASSIGN     // >>=
	AND_NOT_ASSIGN // &^=

	LAND // &&
	LOR  // ||
	INC  // ++
	DEC  // --

	EQL    // ==
	LSS    // <
	GTR    // >
	ASSIGN // =
	NOT    // !

	NEQ      // !=
	LEQ      // <=
	GEQ      // >=
	DEFINE   // :=
	ELLIPSIS // ...

	LPAREN // (
	LBRACK // [
	LBRACE // {
	COMMA  // ,
	PERIOD // .

	RPAREN    // )
	RBRACK    // ]
	RBRACE    // }
	SEMICOLON // ;
	COLON     // :
	ARROW     // =>

	TK_左方 // 【
	TK_右方 // 】
	TK_冒号 // :
	TK_句号 // 。
	TK_顿号 // 、
	TK_叹号 // !
	TK_点号 // ·
	TK_左书 // 《
	TK_右书 // 》
	TK_逗号 // ,

	// 类型
	TK_数
	TK_浮点数
	TK_布尔

	TK_引于
	TK_归于
	TK_若
	TK_则
	TK_否则
	TK_设
	TK_之
	TK_从
	TK_到
	TK_直
	TK_自
	TK_至
	TK_有
	TK_当
	TK_为

	// Keywords
	BREAK
	CASE
	CONST
	CONTINUE

	DEFAULT
	DEFER
	ELSE
	FOR

	FUNC
	IF
	IMPORT

	INTERFACE
	MAP
	PACKAGE
	RANGE
	RETURN

	STRUCT
	SWITCH
	TYPE
	VAR

	// reserved keywords
	CLASS
	ENUM
)

The list of tokens.

func Lookup

func Lookup(ident string) Token

Lookup maps an identifier to its keyword token or IDENT (if not a keyword).

func LookupWaGo

func LookupWaGo(ident string) Token

解析 WaGo 关键字 WaGo 不支持中文关键字

func (Token) IsKeyword

func (tok Token) IsKeyword() bool

IsKeyword returns true for tokens corresponding to keywords; it returns false otherwise.

func (Token) IsLiteral

func (tok Token) IsLiteral() bool

IsLiteral returns true for tokens corresponding to identifiers and basic type literals; it returns false otherwise.

func (Token) IsOperator

func (tok Token) IsOperator() bool

IsOperator returns true for tokens corresponding to operators and delimiters; it returns false otherwise.

func (Token) Precedence

func (op Token) Precedence() int

Precedence returns the operator precedence of the binary operator op. If op is not a binary operator, the result is LowestPrecedence.

func (Token) String

func (tok Token) String() string

String returns the string corresponding to the token tok. For operators, delimiters, and keywords the string is the actual token character sequence (e.g., for the token ADD, the string is "+"). For all other tokens the string corresponds to the token constant name (e.g. for the token IDENT, the string is "IDENT").

func (Token) WaGoKeykword

func (tok Token) WaGoKeykword() string

返回 WaGo 关键字名字

func (Token) WaGoString

func (tok Token) WaGoString() string

func (Token) WaZhString

func (tok Token) WaZhString() string

func (Token) ZhKeykword

func (tok Token) ZhKeykword() string

返回中文关键字名字

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL