token

Documentation

Overview

Package token defines constants representing the lexical tokens of the Ghost programming language and basic operations on tokens.

Index

• Constants
• type Token
• type TokenType
  • func LookupIdentifier(identifier string) TokenType

Constants

const (
	// Special tokens
	ILLEGAL = "ILLEGAL"
	EOF     = "EOF"

	// Basic type literals
	IDENTIFIER = "IDENTIFIER"
	NUMBER     = "NUMBER"
	STRING     = "STRING"

	// Operators and delimiters
	PLUS     = "+"
	MINUS    = "-"
	BANG     = "!"
	ASTERISK = "*"
	SLASH    = "/"
	PERCENT  = "%"
	DOT      = "."
	HASH     = "#"

	LT  = "<"
	GT  = ">"
	LTE = "<="
	GTE = ">="

	COMMA     = ","
	SEMICOLON = ";"
	COLON     = ":"

	LPAREN   = "("
	RPAREN   = ")"
	LBRACE   = "{"
	RBRACE   = "}"
	LBRACKET = "["
	RBRACKET = "]"

	// Keywords
	FUNCTION = "FUNCTION"
	TRUE     = "TRUE"
	FALSE    = "FALSE"
	FOR      = "FOR"
	IN       = "IN"
	IF       = "IF"
	ELSE     = "ELSE"
	RETURN   = "RETURN"
	WHILE    = "WHILE"
	AND      = "AND"
	OR       = "OR"
	EXPORT   = "EXPORT"
	IMPORT   = "IMPORT"

	EQ    = "=="
	NOTEQ = "!="

	ASSIGN         = ":="
	PLUSASSIGN     = "+="
	MINUSASSIGN    = "-="
	ASTERISKASSIGN = "*="
	SLASHASSIGN    = "/="

	PLUSPLUS   = "++"
	MINUSMINUS = "--"

	RANGE = ".."
)

The list of tokens.

Types

type Token

type Token struct {
	Type    TokenType
	Literal string
	Line    int
}

Token represents the scanned token from source.

type TokenType

type TokenType string

TokenType represents the type of token.

func LookupIdentifier

func LookupIdentifier(identifier string) TokenType

LookupIdentifier checks the `keywords` table to see whether the given identifier is in fact a keyword.