token

Documentation

Index

• type Token
  • func LookupWordToken(w string) Token
• type Type
  • func (i Type) String() string

Types

type Token

type Token struct {
	Type  Type
	Value string
}

Token consists of a type and value; it's the unit output of a lexer.

func LookupWordToken

func LookupWordToken(w string) Token

LookupWordToken returns the token for the given word. It checks against a static list of reserved keywords, and if the input matches, returns the corresponding type. If it doesn't, an WORD token is returned.

type Type

type Type int

Type represents the type of a given token.

const (
	ILLEGAL Type = iota + 128
	EOF

	// Words and digits.
	WORD   // x, yyz, ...
	DIGITS // 42, 1343, ...

	// Operations.
	PLUS     // +
	MINUS    // -
	BANG     // !
	ASTERISK // *
	SLASH    // /
	IMPL     // →
	MOD      // %
	LT       // <
	GT       // >
	EXISTS   // ∈
	NEXISTS  // ∉
	UNION    // ∪
	EQ       // ==
	NEQ      // !=

	// Delimiters.
	DOT      // .
	COLON    // :
	COMMA    // ,
	PIPE     // |
	SUM      // Σ
	LPAREN   // (
	RPAREN   // )
	LBRACKET // [
	RBRACKET // ]

	// Keywords.
	AS   // "as"
	IF   // "if"
	IN   // "in"
	MAX  // "max"
	MIN  // "min"
	TO   // "to"
	BOOL // "true" or "false"
)

func (Type) String

func (i Type) String() string