grammar

Documentation

Index

• type Enumer
• type Token
  • func NewTerminalToken[T Enumer](type_ T, data string) *Token[T]
  • func NewToken[T Enumer](type_ T, data string, children []*Token[T]) (*Token[T], error)
  • func (tk Token[T]) GetData() string
  • func (tk Token[T]) GetPos() int
  • func (tk Token[T]) GetType() T

Types

type Enumer

type Enumer interface {
	~int

	// String returns the literal name of the token type.
	// This is used for debugging and error messages.
	//
	// Returns:
	// 	- string: The literal name of the token type.
	String() string
}

Enumer is an interface for all token types. The 0th value is reserved for the EOF token.

type Token

type Token[T Enumer] struct {
	// Type is the type of the token.
	Type T

	// Data is the value of the token.
	Data string

	// Pos is the position of the token in the input stream.
	Pos int

	// Lookahead is the next token in the input stream.
	Lookahead *Token[T]

	// Children are the children of the token.
	Children []*Token[T]
}

Token represents a token in the grammar.

func NewTerminalToken

func NewTerminalToken[T Enumer](type_ T, data string) *Token[T]

NewTerminalToken creates a new terminal token with the given type, data, and lookahead.

Parameters:

• type_: The type of the token.
• data: The value of the token.

Returns:

• *Token: The new token. Never returns nil.

func NewToken

func NewToken[T Enumer](type_ T, data string, children []*Token[T]) (*Token[T], error)

NewToken creates a new non-terminal token with the given type, data, and children.

Keep in mind that the last children must be the furthest in the input stream.

Parameters:

• type_: The type of the token.
• data: The value of the token.
• children: The children of the token.

Returns:

• *Token: The new token.
• error: An error of type *errors.ErrInvalidParameter if there is an empty list of children.

func (Token[T]) GetData

func (tk Token[T]) GetData() string

GetData returns the value of the token.

Returns:

• string: The value of the token.

func (Token[T]) GetPos

func (tk Token[T]) GetPos() int

GetPos returns the position of the token in the input stream.

Returns:

• int: The position of the token in the input stream.

func (Token[T]) GetType

func (tk Token[T]) GetType() T

GetType returns the type of the token.

Returns:

• T: The type of the token.