Documentation
¶
Overview ¶
This file was automatically generated by gen-unicode-table.js. Do not edit.
Index ¶
- Variables
- func ForceValidIdentifier(text string) string
- func IsIdentifier(text string) bool
- func IsIdentifierContinue(codePoint rune) bool
- func IsIdentifierContinueES5AndESNext(codePoint rune) bool
- func IsIdentifierES5AndESNext(text string) bool
- func IsIdentifierES5AndESNextUTF16(text []uint16) bool
- func IsIdentifierStart(codePoint rune) bool
- func IsIdentifierStartES5AndESNext(codePoint rune) bool
- func IsIdentifierUTF16(text []uint16) bool
- func IsWhitespace(codePoint rune) bool
- func RangeOfIdentifier(source logger.Source, loc logger.Loc) logger.Range
- type Lexer
- func (lexer *Lexer) AddRangeErrorWithNotes(r logger.Range, text string, notes []logger.MsgData)
- func (lexer *Lexer) CookedAndRawTemplateContents() ([]uint16, string)
- func (lexer *Lexer) Expect(token T)
- func (lexer *Lexer) ExpectContextualKeyword(text string)
- func (lexer *Lexer) ExpectGreaterThan(isInsideJSXElement bool)
- func (lexer *Lexer) ExpectInsideJSXElement(token T)
- func (lexer *Lexer) ExpectJSXElementChild(token T)
- func (lexer *Lexer) ExpectLessThan(isInsideJSXElement bool)
- func (lexer *Lexer) ExpectOrInsertSemicolon()
- func (lexer *Lexer) Expected(token T)
- func (lexer *Lexer) ExpectedString(text string)
- func (lexer *Lexer) IsContextualKeyword(text string) bool
- func (lexer *Lexer) IsIdentifierOrKeyword() bool
- func (lexer *Lexer) Loc() logger.Loc
- func (lexer *Lexer) Next()
- func (lexer *Lexer) NextInsideJSXElement()
- func (lexer *Lexer) NextJSXElementChild()
- func (lexer *Lexer) Range() logger.Range
- func (lexer *Lexer) Raw() string
- func (lexer *Lexer) RescanCloseBraceAsTemplateToken()
- func (lexer *Lexer) ScanRegExp()
- func (lexer *Lexer) StringLiteral() []uint16
- func (lexer *Lexer) SyntaxError()
- func (lexer *Lexer) Unexpected()
- type LexerPanic
- type MaybeSubstring
- type T
Constants ¶
This section is empty.
Variables ¶
var Keywords = map[string]T{ "break": TBreak, "case": TCase, "catch": TCatch, "class": TClass, "const": TConst, "continue": TContinue, "debugger": TDebugger, "default": TDefault, "delete": TDelete, "do": TDo, "else": TElse, "enum": TEnum, "export": TExport, "extends": TExtends, "false": TFalse, "finally": TFinally, "for": TFor, "function": TFunction, "if": TIf, "import": TImport, "in": TIn, "instanceof": TInstanceof, "new": TNew, "null": TNull, "return": TReturn, "super": TSuper, "switch": TSwitch, "this": TThis, "throw": TThrow, "true": TTrue, "try": TTry, "typeof": TTypeof, "var": TVar, "void": TVoid, "while": TWhile, "with": TWith, }
Functions ¶
func ForceValidIdentifier ¶
func IsIdentifier ¶
func IsIdentifierContinue ¶
func IsIdentifierContinueES5AndESNext ¶ added in v0.12.28
func IsIdentifierES5AndESNext ¶ added in v0.12.28
func IsIdentifierES5AndESNextUTF16 ¶ added in v0.12.28
This does "IsIdentifierES5AndESNext(UTF16ToString(text))" without any allocations
func IsIdentifierStart ¶
func IsIdentifierStartES5AndESNext ¶ added in v0.12.28
func IsIdentifierUTF16 ¶
This does "IsIdentifier(UTF16ToString(text))" without any allocations
func IsWhitespace ¶
See the "White Space Code Points" table in the ECMAScript standard
Types ¶
type Lexer ¶
type Lexer struct {
CommentsToPreserveBefore []js_ast.Comment
AllOriginalComments []js_ast.Comment
Identifier MaybeSubstring
JSXFactoryPragmaComment logger.Span
JSXFragmentPragmaComment logger.Span
SourceMappingURL logger.Span
Number float64
ApproximateNewlineCount int
LegacyOctalLoc logger.Loc
AwaitKeywordLoc logger.Loc
FnOrArrowStartLoc logger.Loc
PreviousBackslashQuoteInJSX logger.Range
LegacyHTMLCommentRange logger.Range
Token T
HasNewlineBefore bool
HasPureCommentBefore bool
PreserveAllCommentsBefore bool
IsLegacyOctalLiteral bool
PrevTokenWasAwaitKeyword bool
// The log is disabled during speculative scans that may backtrack
IsLogDisabled bool
// contains filtered or unexported fields
}
func NewLexerGlobalName ¶ added in v0.8.0
func NewLexerJSON ¶
func (*Lexer) AddRangeErrorWithNotes ¶ added in v0.14.24
func (*Lexer) CookedAndRawTemplateContents ¶ added in v0.12.5
func (*Lexer) ExpectContextualKeyword ¶
func (*Lexer) ExpectGreaterThan ¶
This parses a single ">" token. If that is the first part of a longer token, this function splits off the first ">" and leaves the remainder of the current token as another, smaller token. For example, ">>=" becomes ">=".
func (*Lexer) ExpectInsideJSXElement ¶
func (*Lexer) ExpectJSXElementChild ¶
func (*Lexer) ExpectLessThan ¶
This parses a single "<" token. If that is the first part of a longer token, this function splits off the first "<" and leaves the remainder of the current token as another, smaller token. For example, "<<=" becomes "<=".
func (*Lexer) ExpectOrInsertSemicolon ¶
func (lexer *Lexer) ExpectOrInsertSemicolon()
func (*Lexer) ExpectedString ¶
func (*Lexer) IsContextualKeyword ¶
func (*Lexer) IsIdentifierOrKeyword ¶
func (*Lexer) NextInsideJSXElement ¶
func (lexer *Lexer) NextInsideJSXElement()
func (*Lexer) NextJSXElementChild ¶
func (lexer *Lexer) NextJSXElementChild()
func (*Lexer) RescanCloseBraceAsTemplateToken ¶
func (lexer *Lexer) RescanCloseBraceAsTemplateToken()
func (*Lexer) ScanRegExp ¶
func (lexer *Lexer) ScanRegExp()
func (*Lexer) StringLiteral ¶
func (*Lexer) SyntaxError ¶
func (lexer *Lexer) SyntaxError()
func (*Lexer) Unexpected ¶
func (lexer *Lexer) Unexpected()
type LexerPanic ¶
type LexerPanic struct{}
type MaybeSubstring ¶ added in v0.14.22
This represents a string that is maybe a substring of the current file's "source.Contents" string. The point of doing this is that if it is a substring (the common case), then we can represent it more efficiently.
For compactness and performance, the JS AST represents identifiers as a symbol reference instead of as a string. However, we need to track the string between the first pass and the second pass because the string is only resolved to a symbol in the second pass. To avoid allocating extra memory to store the string, we instead use an index+length slice of the original JS source code. That index is what "Start" represents here. The length is just "len(String)".
Set "Start" to invalid (the zero value) if "String" is not a substring of "source.Contents". This is the case for escaped identifiers. For example, the identifier "fo\u006f" would be "MaybeSubstring{String: "foo"}". It's critical that any code changing the "String" also set "Start" to the zero value, which is best done by just overwriting the whole "MaybeSubstring".
The substring range used to be recovered automatically from the string but that relied on the Go "unsafe" package which can hypothetically break under certain Go compiler optimization passes, so it has been removed and replaced with this more error-prone approach that doesn't use "unsafe".
type T ¶
type T uint8
const ( TEndOfFile T = iota TSyntaxError // "#!/usr/bin/env node" THashbang // Literals TNoSubstitutionTemplateLiteral // Contents are in lexer.StringLiteral ([]uint16) TNumericLiteral // Contents are in lexer.Number (float64) TStringLiteral // Contents are in lexer.StringLiteral ([]uint16) TBigIntegerLiteral // Contents are in lexer.Identifier (string) // Pseudo-literals TTemplateHead // Contents are in lexer.StringLiteral ([]uint16) TTemplateMiddle // Contents are in lexer.StringLiteral ([]uint16) TTemplateTail // Contents are in lexer.StringLiteral ([]uint16) // Punctuation TAmpersand TAmpersandAmpersand TAsterisk TAsteriskAsterisk TAt TBar TBarBar TCaret TCloseBrace TCloseBracket TCloseParen TColon TComma TDot TDotDotDot TEqualsEquals TEqualsEqualsEquals TEqualsGreaterThan TExclamation TExclamationEquals TExclamationEqualsEquals TGreaterThan TGreaterThanEquals TGreaterThanGreaterThan TGreaterThanGreaterThanGreaterThan TLessThan TLessThanEquals TLessThanLessThan TMinus TMinusMinus TOpenBrace TOpenBracket TOpenParen TPercent TPlus TPlusPlus TQuestion TQuestionDot TQuestionQuestion TSemicolon TSlash TTilde // Assignments (keep in sync with IsAssign() below) TAmpersandAmpersandEquals TAmpersandEquals TAsteriskAsteriskEquals TAsteriskEquals TBarBarEquals TBarEquals TCaretEquals TEquals TGreaterThanGreaterThanEquals TGreaterThanGreaterThanGreaterThanEquals TLessThanLessThanEquals TMinusEquals TPercentEquals TPlusEquals TQuestionQuestionEquals TSlashEquals // Class-private fields and methods TPrivateIdentifier // Identifiers TIdentifier // Contents are in lexer.Identifier (string) TEscapedKeyword // A keyword that has been escaped as an identifer // Reserved words TBreak TCase TCatch TClass TConst TContinue TDebugger TDefault TDelete TDo TElse TEnum TExport TExtends TFalse TFinally TFor TFunction TIf TImport TIn TInstanceof TNew TNull TReturn TSuper TSwitch TThis TThrow TTrue TTry TTypeof TVar TVoid TWhile TWith )
If you add a new token, remember to add it to "tokenToString" too
Source Files