tweelexer

Documentation

Index

• type Item
  • func (i Item) String() string
• type ItemType
• type Tweelexer
  • func NewTweelexer(input []byte) *Tweelexer
  • func (l *Tweelexer) Drain()
  • func (l *Tweelexer) GetItems() chan Item
  • func (l *Tweelexer) NextItem() (Item, bool)

Types

type Item

type Item struct {
	Type ItemType // Type of the item.
	Line int      // Line within the input (1-base) of the item.
	Pos  int      // Starting position within the input, in bytes, of the item.
	Val  []byte   // Value of the item.
}

Item represents a lexed item, a lexeme.

func (Item) String

func (i Item) String() string

String returns a formatted debugging string for the item.

type ItemType

type ItemType int

ItemType identifies the type of the items.

const (
	ItemError    ItemType = iota // Error.  Its value is the error message.
	ItemEOF                      // End of input.
	ItemHeader                   // '::', but only when starting a line.
	ItemName                     // Text w/ backslash escaped characters.
	ItemTags                     // '[tag1 tag2 tagN]'.
	ItemMetadata                 // JSON chunk, '{…}'.
	ItemContent                  // Plain text.
)

Item type constants.

type Tweelexer

type Tweelexer struct {
	// contains filtered or unexported fields
}

Tweelexer holds the state of the scanner.

func NewTweelexer

func NewTweelexer(input []byte) *Tweelexer

NewTweelexer creates a new scanner for the input text.

func (*Tweelexer) Drain

func (l *Tweelexer) Drain()

Drain drains the item channel so the lexing goroutine will close the item channel and exit. Called by the parser, not tweelexer.

func (*Tweelexer) GetItems

func (l *Tweelexer) GetItems() chan Item

GetItems returns the item channel. Called by the parser, not tweelexer.

func (*Tweelexer) NextItem

func (l *Tweelexer) NextItem() (Item, bool)

NextItem returns the next item and its ok status from the item channel. Called by the parser, not tweelexer.