leks

package module
v0.0.0-...-d63147e Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 31, 2015 License: BSL-1.0 Imports: 10 Imported by: 0

Documentation

Index

Constants

View Source
const (
	TEOF = 1 + TokenType(iota)

	TTrue  // true
	TFalse // false
	TNil   // nil

	TDot       // .
	TDoubleDot // ..
	TTripleDot // ...

	THash         // #
	TAt           // @ (word initial)
	TDollar       // $ (word initial)
	TParenOpen    // (
	TParenClose   // )
	TBracketOpen  // [
	TBracketClose // ]
	TCurlOpen     // {
	TCurlClose    // }
	TBackslash    // \
	TComma        // ,
	TSemicolon    // ;
	TNewline      // \n
	TQuote        // '

	TBang     // !
	TNotEqual // !=

	TSlash       // /
	TSlashEquals // /=

	TPercent       // %
	TPercentEquals // %=

	TCaret       // ^
	TCaretEquals // ^=

	TTilde       // ~ (word initial)
	TTildeEquals // ~=

	TGreaterThan  // >
	TShiftRight   // >>
	TGreaterEqual // >=

	TLessThan    // <
	TShiftLeft   // <<
	TLesserEqual // <=
	TInequality  // <>

	TQuestion       // ?
	TDoubleQuestion // ??
	TQuestionEquals // ?=

	TEquals   // =
	TEquality // ==
	TShovel   // =>

	TPipe       // |
	TPipeEquals // |=
	TOr         // ||
	TOrEquals   // ||=

	TAmp       // &
	TAmpEquals // &=
	TAnd       // &&
	TAndEquals // &&=

	TMul       // *
	TDoubleMul // **
	TMulAssign // *=

	TColon       // :
	TDoubleColon // ::
	TColonEqual  // :=
	TColonArrow  // :>

	TMinus       // -
	TDoubleMinus // --
	TMinusEquals // -=
	TArrow       // ->

	TPlus       // +
	TDoublePlus // ++
	TPlusEquals // +=
	TPlusArrow  // +>

	TIdent // any identifier

	TIntLit       // 5
	TIntExpLit    // 5e+8
	TFloatLit     // 0.5
	TFloatExpLit  // 0.5e+8
	TOctLit       // 0655
	THexLit       // 0x1234abcd 0X1234ABCD
	TBinLit       // 0b1101 0B1101
	TRawStringLit // `raw string`
	TRuneLit      // 'c'
	TStringLit    // "string"

	TLineComment  // // comment
	TBlockComment // /* comment */
)
View Source
const TError = TokenType(-1)

TError is the token type for any failed attempt to parse a token. The token's Err field should be set when this is its type.

Variables

View Source
var (
	ErrInvalidKind = errors.New("cannot convert token of this kind")
)

Functions

This section is empty.

Types

type Error

type Error int

func (Error) Error

func (e Error) Error() string

type Lexer

type Lexer struct {
	// contains filtered or unexported fields
}

func New

func New(r io.RuneScanner) *Lexer

New allocates a new Lexer that reads its input from the given io.RuneScanner, r. If r is nil, New panics.

func (*Lexer) ReadToken

func (l *Lexer) ReadToken() (Token, error)

ReadToken reads the next token from the Lexer's rune stream. If an error occurs, it is returned. If a parsing error occurs, it will typically be returned as a token of kind TError with its Err field set. Subsequent reads will attempt to resume from wherever the last token stopped. If the last token was an error, the next token may be in the middle of the previous erroneous token.

type Location

type Location struct {
	Off  int // Offset in bytes from the unit's start.
	Line int // Line number (starting at 1).
	Col  int // Column number (starting at 1).
}

Location is a location in a token's source unit.

func (Location) String

func (l Location) String() string

func (Location) Sub

func (l Location) Sub(r Location) Location

type Token

type Token struct {
	Start, End Location
	Kind       TokenType
	Text       []byte
	Err        error // Only likely to be set if Kind == TError
}

func (Token) After

func (t Token) After(o Token) bool

func (Token) Before

func (t Token) Before(o Token) bool

func (Token) Bytes

func (t Token) Bytes() []byte

func (Token) GoString

func (t Token) GoString() string

func (Token) Int

func (t Token) Int() (int64, error)

func (Token) IsComment

func (t Token) IsComment() bool

IsComment returns whether the token is a comment token (i.e., you might want to discard it).

func (Token) IsError

func (t Token) IsError() bool

IsIdent returns whether the token is an error. This is shorthand for t.Kind == TError.

func (Token) IsID

func (t Token) IsID() bool

IsID returns whether the token is an identifier. This is shorthand for t.Kind == TIdent.

func (Token) IsKeyword

func (t Token) IsKeyword() bool

IsKeyword returns whether the token is a keyword.

func (Token) IsNumeric

func (t Token) IsNumeric() bool

IsNumeric returns whether the token is numeric. Check its Kind field to determine the specific type of numeric.

func (Token) IsPunct

func (t Token) IsPunct() bool

IsPunct returns whether the token is a punctuation token (newlines, parentheses, brackets, symbols, etc.).

func (Token) IsQuoted

func (t Token) IsQuoted() bool

IsQuoted returns whether the token is a quoted string or rune. If true, the rune's Text field does not include the quotation marks.

func (Token) Near

func (t Token) Near(o Token, byLines, byColumns int) bool

func (Token) Rune

func (t Token) Rune() (r rune, size int)

func (Token) String

func (t Token) String() string

func (Token) Uint

func (t Token) Uint() (uint64, error)

type TokenType

type TokenType int

TokenType is the integer identifier for specific types of tokens.

func (TokenType) GoString

func (t TokenType) GoString() string

func (TokenType) String

func (t TokenType) String() string

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL