token

Documentation

Overview

Package token provides tokens for parsing the regular expressions.

Index

• type Token
  • func NewToken(value rune, k Type) Token
  • func (t Token) String() string
• type Type
  • func (k Type) String() string

Types

type Token

type Token struct {
	V  rune // token value
	Ty Type // token type
}

Token represents a token.

func NewToken

func NewToken(value rune, k Type) Token

NewToken returns a new Token.

func (Token) String

func (t Token) String() string

type Type

type Type int

Type is integer to identify the type of token.

const (
	CHARACTER Type = iota
	UNION
	STAR
	PLUS
	LPAREN
	RPAREN
	EOF
)

Each token is identified by a unique integer.

func (Type) String

func (k Type) String() string