etoken

Documentation

Overview

Package token defines constants representing the lexical tokens of the Go programming language and basic operations on tokens (printing, predicates).

Index

• Variables
• func IsKeyword(tok Token) bool
• func IsLiteral(tok Token) bool
• func IsMacroKeyword(tok Token) bool
• func IsOperator(tok Token) bool
• func String(tok Token) string
• type File
  • func (f *File) Position(p token.Pos) (pos token.Position)
  • func (f *File) PositionFor(p token.Pos, adjusted bool) (pos token.Position)
  • func (f *File) SetSource(source []string)
  • func (f *File) SetSourceForContent(content []byte)
  • func (f *File) Source(p token.Pos) (line string, pos token.Position)
• type FileSet
  • func NewFileSet() *FileSet
  • func (s *FileSet) AddFile(filename string, base, size, line int) *File
  • func (s *FileSet) File(p token.Pos) (f *File)
  • func (s *FileSet) Position(p token.Pos) (pos token.Position)
  • func (s *FileSet) PositionFor(p token.Pos, adjusted bool) (pos token.Position)
  • func (s *FileSet) Source(p token.Pos) (line string, pos token.Position)
• type Generics
  • func (g Generics) V1_CXX() bool
  • func (g Generics) V2_CTI() bool
• type Token
  • func Lookup(lit string) Token
  • func LookupSpecial(lit string) Token

Variables

var GENERICS = GENERICS_NONE

can be changed at runtime. useful to enable them by in gomacro, without affecting packages that depend on gomacro, as Gophernotes

Functions

func IsKeyword

func IsKeyword(tok Token) bool

IsKeyword returns true for tokens corresponding to keywords; it returns false otherwise.

func IsLiteral

func IsLiteral(tok Token) bool

IsLiteral returns true for tokens corresponding to identifiers and basic type literals; it returns false otherwise.

func IsMacroKeyword

func IsMacroKeyword(tok Token) bool

IsMacroKeyword returns true for tokens corresponding to macro-related keywords; it returns false otherwise.

func IsOperator

func IsOperator(tok Token) bool

IsOperator returns true for tokens corresponding to operators and delimiters; it returns false otherwise.

func String

func String(tok Token) string

Types

type File

type File struct {
	*token.File
	// contains filtered or unexported fields
}

A File is a handle for a file belonging to a FileSet. A File has a name, size, line offset table and optionally source code.

func (*File) Position

func (f *File) Position(p token.Pos) (pos token.Position)

Position returns the Position value for the given file position p. Calling f.Position(p) is equivalent to calling f.PositionFor(p, true).

func (*File) PositionFor

func (f *File) PositionFor(p token.Pos, adjusted bool) (pos token.Position)

PositionFor returns the Position value for the given file position p. If adjusted is set, the position may be adjusted by position-altering //line comments; otherwise those comments are ignored. p must be a Pos value in f or NoPos.

func (*File) SetSource

func (f *File) SetSource(source []string)

SetSource sets the source code for the given file.

func (*File) SetSourceForContent

func (f *File) SetSourceForContent(content []byte)

SetSourceForContent computes and sets the source code for the given file.

func (*File) Source

func (f *File) Source(p token.Pos) (line string, pos token.Position)

Source returns the source code for the given file position p, if available.

type FileSet

type FileSet struct {
	token.FileSet
	// contains filtered or unexported fields
}

A FileSet represents a set of source files. This is a wrapper for go/token.FileSet that adds a starting line offset to each file in the set

func NewFileSet

func NewFileSet() *FileSet

NewFileSet creates a new file set.

func (*FileSet) AddFile

func (s *FileSet) AddFile(filename string, base, size, line int) *File

AddFile adds a new file with a given filename, base offset, and file size

func (*FileSet) File

func (s *FileSet) File(p token.Pos) (f *File)

File returns the file that contains the position p. If no such file is found (for instance for p == NoPos), the result is nil.

func (*FileSet) Position

func (s *FileSet) Position(p token.Pos) (pos token.Position)

Position converts a Pos p in the fileset into a Position value. Calling s.Position(p) is equivalent to calling s.PositionFor(p, true).

func (*FileSet) PositionFor

func (s *FileSet) PositionFor(p token.Pos, adjusted bool) (pos token.Position)

PositionFor converts a Pos p in the fileset into a Position value. If adjusted is set, the position may be adjusted by position-altering //line comments; otherwise those comments are ignored. p must be a Pos value in s or NoPos.

func (*FileSet) Source

func (s *FileSet) Source(p token.Pos) (line string, pos token.Position)

Source converts a Pos p in the fileset into a line of source code (if available) and a Position value.

type Generics

type Generics int

const (
	GENERICS_NONE Generics = iota
	// enables C++ style generics
	GENERICS_V1_CXX
	// enables generics "contracts are interfaces"
	GENERICS_V2_CTI
)

func (Generics) V1_CXX

func (g Generics) V1_CXX() bool

func (Generics) V2_CTI

func (g Generics) V2_CTI() bool

type Token

type Token = token.Token

const (
	QUOTE Token = (token.VAR+127)&^127 + iota
	QUASIQUOTE
	UNQUOTE
	UNQUOTE_SPLICE
	MACRO
	FUNCTION
	LAMBDA
	TYPECASE
	TEMPLATE // template
	HASH     // #

	// the following are never used by go/scanner
	// they are returned by ast2/Ast.Op() for corresponding AST nodes
	E_ALIASTYPE
	E_CALL
	E_DECLSTMT
	E_FIELDDECL
	E_FUNCDECL
	E_FUNCTYPE
	E_KEYVALUE
	E_LABELEDSTMT
	E_SENDSTMT
	E_TYPEASSERT
	E_TYPESWITCH
)

func Lookup

func Lookup(lit string) Token

Lookup maps a identifier to its keyword token.

func LookupSpecial

func LookupSpecial(lit string) Token

LookupSpecial maps a identifier starting with '~' to its keyword token.