Versions in this module Expand all Collapse all v0 v0.1.0 Nov 12, 2022 v0.0.1 Nov 10, 2022 Changes in this version + func Split(s string) ([]string, error) + type Lexer Tokenizer + func NewLexer(r io.Reader) *Lexer + func (l *Lexer) Next() (string, error) + type Token struct + func (a *Token) Equal(b *Token) bool + type TokenType int + const CommentToken + const SpaceToken + const UnknownToken + const WordToken + type Tokenizer struct + func NewTokenizer(r io.Reader) *Tokenizer + func (t *Tokenizer) Next() (*Token, error)