Why Go
- Case Studies
  
  Common problems companies solve with Go
- Use Cases
  
  Stories about how and why companies use Go
- Security Policy
  
  How Go can help keep you secure by default
Learn
Docs
- Effective Go
  
  Tips for writing clear, performant, and idiomatic Go code
- Go User Manual
  
  A complete introduction to building software with Go
- Standard library
  
  Reference documentation for Go's standard library
- Release Notes
  
  Learn what's new in each Go release
Packages
Community
- Recorded Talks
  
  Videos from prior events
- Meetups
  
  Meet other local Go developers
- Conferences
  
  Learn and network with Go developers from around the world
- Go blog
  
  The Go project's official blog.
- Go project
  
  Get help and stay informed from Go
- Get connected

token

package

Go to main page

Versions in this module

v0

v0.1.1

Jun 11, 2022

v0.1.0

Jun 11, 2022

v0.0.7

Jun 10, 2022

v0.0.6

Mar 31, 2022

v0.0.5

Mar 31, 2022

v0.0.4

Mar 29, 2022

v0.0.3

Mar 22, 2022

v0.0.2

Mar 16, 2022

Changes in this version

+ var TOKEN_ASTERISK = Token

+ var TOKEN_DOUBLE_NEWLINE = Token

+ var TOKEN_EOF = Token

+ var TOKEN_EOS = Token

+ var TOKEN_NEWLINE = Token

+ var TokenTypeMap = map[TokenType][]rune

+ func EndsWith(ts []Token, pat []Token) bool

+ func HasPattern(ts []Token, pat []TokenType) bool

+ func Literals(ts []Token) string

+ func Split(ts []Token, tt TokenType) [][]Token

+ func SplitOnFirst(ts []Token, tt TokenType) ([]Token, []Token)

+ func StartsWith(ts []Token, pat []Token) bool

+ type Lexer struct

+ func NewLexer(in io.Reader) *Lexer

+ func (lex *Lexer) Error() error

+ func (lex *Lexer) Out() chan Token

+ type Token struct

+ Literal string

+ Type TokenType

+ func (t Token) Equal(wanted ...Token) bool

+ func (t Token) Len() int

+ type TokenReader struct

+ func NewTokenReader(in chan Token) *TokenReader

+ func (tr *TokenReader) Discard()

+ func (tr *TokenReader) Read(n int) ([]Token, bool)

+ func (tr *TokenReader) Unread(n int) bool

+ type TokenStream struct

+ func NewTokenStream(toks []Token) *TokenStream

+ func (s *TokenStream) Out() chan Token

+ type TokenType int

+ const TYPE_ASTERISK

+ const TYPE_BACKTICK

+ const TYPE_BRACKETCLOSE

+ const TYPE_BRACKETOPEN

+ const TYPE_COLON

+ const TYPE_DASH

+ const TYPE_EOF

+ const TYPE_EOS

+ const TYPE_EQUALSIGN

+ const TYPE_NEWLINE

+ const TYPE_UNDERSCORE

+ const TYPE_WHITESPACE

+ const TYPE_WORD

+ func FindTokenType(ch rune) TokenType