token

Documentation

Overview

Package token implements tokenization of BIT code.

Index

• Variables
• type ErrInvalidChar
  • func (err ErrInvalidChar) Error() string
• type Parser
  • func (p *Parser) Done() error
  • func (p *Parser) Write(b []byte) (n int, err error)
  • func (p *Parser) WriteByte(c byte) error
  • func (p *Parser) WriteRune(c rune) error
  • func (p *Parser) WriteString(str string) (n int, err error)
• type Token
  • func (t Token) String() string

Variables

var ErrSyntax = errors.New("bit/token: syntax error")

ErrSyntax is returned by Parser methods when a token is truncated or invalid.

Types

type ErrInvalidChar

type ErrInvalidChar struct {
	Char rune
}

ErrInvalidChar is returned by Parser methods when a character that is not an uppercase ASCII letter or a space is written.

func (ErrInvalidChar) Error

func (err ErrInvalidChar) Error() string

Error implements the error interface.

type Parser

type Parser struct {
	// Tokens are appended to the slice by writing to the parser. The
	// Parser does not change or access this slice in any other way.
	Tokens []Token
	// contains filtered or unexported fields
}

Parser converts a stream of bytes or runes into a []Token.

It is not safe for concurrent use across multiple goroutines; however, Parser does not modify any global or shared state, so multiple independent instances of Parser are safe to use concurrently.

Methods that return error can return ErrSyntax or ErrInvalidChar. If ErrSyntax is returned by a Write* method, the current token is discarded to assist with tokenization error recovery.

func (*Parser) Done

func (p *Parser) Done() error

Done returns ErrSyntax if a partial token is buffered. Otherwise, it returns nil.

func (*Parser) Write

func (p *Parser) Write(b []byte) (n int, err error)

Write wraps the WriteByte method of Parser.

func (*Parser) WriteByte

func (p *Parser) WriteByte(c byte) error

WriteByte adds a byte to the Parser input. If the byte is not an uppercase ASCII letter or a space character as defined by Unicode, this returns ErrInvalidChar. If an invalid token would be created by adding this byte, ErrSyntax is returned and the current token is discarded.

func (*Parser) WriteRune

func (p *Parser) WriteRune(c rune) error

WriteRune supports Unicode but has the same restrictions and behavior as WriteByte. This method only differs in that it accepts non-ASCII whitespace and the ErrInvalidChar it returns will contain a full unicode code point.

func (*Parser) WriteString

func (p *Parser) WriteString(str string) (n int, err error)

WriteString wraps the WriteRune method of Parser.

type Token

type Token uint8

Token represents a word or pair of words in the BIT language.

const (
	AddressOf Token
	At
	Beyond
	Close
	Code
	Equals
	Goto
	If
	Is
	JumpRegister
	LineNumber
	Nand
	One
	Open
	Parenthesis
	Print
	Read
	The
	Value
	Variable
	Zero
)

Possible values for Token.

func (Token) String

func (t Token) String() string

String returns the token's source code representation, or "INVALID" if the token is not a valid Token constant from this package.