tok

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 20, 2022 License: MIT Imports: 7 Imported by: 0

README 

= tok

Go package to parse strings.
Inpired by the cScanner implementation from link:https://clingo.aiq.dk/[clingo].

== Scanner

Scanner is the main type to parse a string.
It has basic functions to parse a text forward or backward.
The basic functions exist for the following types:

* string
* fold
* rune
* anyrune
* between
* match

Basic functions return just a bool.
Additional has the scanner read functions to read values.
Read functions return the value and an error if it was not possible to read the value.
The package has the following read functions:

* ReadBool
* ReadInt
* ReadUint

== Reader

Readers can be used by the scanner to read from the scanner.
tok has the following build-in Reader:
Any, AnyFold, AnyRune, At, Between, BetweenAny, Body, Bool, Digit, Fold, Hex, Holey, Int, Janus, Lit, Many, Map, Match, Named, Not, Opt, Past, Rune, Seq, Set, SkipSeq, SkipWSSeq, Times, To, Uint, Wrap, WS, Zom


== Mark Types

Marker::
can be used to mark a scanner position

Token::
can be used to mark a range of the scanner

Segment::
can be used tag a Token with addition information

== Grammar

A grammar is a Reader that has connected Rules.
Check the grammar package with different grammars, like JSOM, MXT and Lua.

== Graph

A graph allows to arrange the picked values hierarchically via Nodes.
With the FlameStack function is it possible to generate a string that can be used to produce a FlameGraph:

[source,shell]
----
$ flamegraph.pl flame.stack > graph.svg 
----

== Tracker

A Tracker can be coupled with a Scannar and track the movemend.
Basket is a Tracker und can be used to Pick-Up Elements that where read by the Parser.

== Log

A Log can be used to monitor and log the movemend in a Reader graph.

Documentation

Index

Constants

View Source 
const InvalidReaderMarker = "::INVALID-READER::"

InvalidReaderMarker is the value that can be used to identify invalid reader in a Reader, Rule or Grammar.

Variables

This section is empty.

Functions

func AnyHasInvalidReader 

func AnyHasInvalidReader(strs []string) bool

AnyHasInvalidReader returns true if any of the strings in strs contains the :INVALID-READER: marker.

func CheckRuleName 

func CheckRuleName(name string) error

CheckRuleName validates the Name field of a RuleReader.

func CheckRules 

func CheckRules(g Grammar) error

CheckRules checks if the Rules in a Grammar have a Name and a Reader set.

func GrammarLines 

func GrammarLines(g []*Rule) []string

GrammarRules calls the Rule function on all rules and returns the result.

func HasInvalidReader 

func HasInvalidReader(str string) bool

HasInvalidReader returns true if str contains the :INVALID-READER: marker.

func InvalidReader 

func InvalidReader(format string, a ...interface{}) invalidReader

InvalidReader is a Reader that allways fails. The arguments will be passed to fmt.Errorf.

func Janus 

func Janus(name string, r Reader) (Reader, Reader)

Janus creates two Reader. The first one tries to match with r. If the first matches expects the second the matched sub string.

func MustCheckRules 

func MustCheckRules(g Grammar)

MustCheckRules panics if an error occurs during CheckRules.

func MustSetRuleNames 

func MustSetRuleNames(g Grammar)

MustSetRuleNames panics if an error occurs during SetRuleNames.

func SetRuleNames 

func SetRuleNames(g Grammar) error

SetRuleNames sets the rule names via the associated Field-Tag.

func SortSegments 

func SortSegments(values []Segment)

SortSegments sorts the segments in a slice. Segments that cover other segments will appear before the covered values.

func SortSegmentsByOrder 

func SortSegmentsByOrder(values []Segment, order []string)

SortSegmentsByOrder sorts like SortSegments with an order as additional tiebreaker. The appearence of an information in the order slice determines the order for segments with equal tokens.

Types

type Basket 

type Basket struct {
	// contains filtered or unexported fields
}

Basket can be used to Pick readed Segments.

func (*Basket) Add 

func (b *Basket) Add(seg Segment)

Add adds a Segment to the Basket.

func (*Basket) PickWith 

func (b *Basket) PickWith(rules ...*Rule)

PickWith calls Pick on all rules with the Basket as paramter.

func (*Basket) Picked 

func (b *Basket) Picked() []Segment

Picked returns the picked Segments.

func (*Basket) String 

func (b *Basket) String() string

func (*Basket) Update 

func (b *Basket) Update(m Marker)

type BoolReader 

type BoolReader struct {
	Value  bool
	Format string
}

------------------------------------------------------------------------------ BoolReader is a Reader that stores the readed bool value in the field Value.

func Bool 

func Bool(format string) *BoolReader

Bool creates a Reader to Read bool values from the scanner. Valid format values are - "l" for true and false - "U" for TRUE and FALSE - "Cc" for True and False - "*" for all cases An empty format string will be interpreted as "*".

func (*BoolReader) Read 

func (r *BoolReader) Read(s *Scanner) error

func (*BoolReader) What 

func (r *BoolReader) What() string

type Grammar 

type Grammar interface {
	Reader
	Grammar() []*Rule
}

Grammer is a Reader that has a Grammar.

type Graph 

type Graph struct {
	Root *Node
}

Graph allows to arrange the picked values hierarchically.

func BuildGraph 

func BuildGraph(name string, segments []Segment) *Graph

BuildGraph creates a graph with the segments. The name argument will be used as the info of the root node.

func NewGraph 

func NewGraph(name string) *Graph

NewGraph creates an empty graph with name as the info of the root node.

func (*Graph) Append 

func (g *Graph) Append(seg Segment) (*Graph, bool)

Append appends a Segment to a graph.

func (*Graph) Equal 

func (g *Graph) Equal(oth *Graph) bool

Equal

func (*Graph) FlameStack 

func (g *Graph) FlameStack() string

FlameStack

func (*Graph) Leafs 

func (g *Graph) Leafs() []Segment

Leafs returns the values of a graph that hang on the leafs.

type IntReader 

type IntReader struct {
	Value   int64
	Base    int
	BitSize int
}

------------------------------------------------------------------------------ IntReader is a Reader that stores the readed int value in the field Value.

func Int 

func Int(base int, bitSize int) *IntReader

Int creates a Reader to Read int values from the scanner. Valid base values are 8. 10 and 16. Valid bitSize values are 8, 16, 32 and 64.

func (*IntReader) Read 

func (r *IntReader) Read(s *Scanner) error

func (*IntReader) What 

func (r *IntReader) What() string

type Log 

type Log struct {
	Entries []LogEntry
	// contains filtered or unexported fields
}

func MonitorGrammar 

func MonitorGrammar(g Grammar) *Log

func (*Log) Enter 

func (l *Log) Enter(info string, pos int) *LogEntry

func (*Log) Exit 

func (l *Log) Exit(e *LogEntry, pos int, err error)

func (*Log) Monitor 

func (l *Log) Monitor(readers ...*Rule)

func (*Log) Print 

func (l *Log) Print()

func (*Log) PrintWithPreview 

func (l *Log) PrintWithPreview(str string, n int)

func (*Log) Reset 

func (l *Log) Reset()

type LogEntry 

type LogEntry struct {
	EnterAt int
	Info    string
	Level   int
	ExitAt  int
	Error   error
}

func (LogEntry) String 

func (e LogEntry) String() string

type MapFunc 

type MapFunc func(Token)

------------------------------------------------------------------------------

type Marker 

type Marker int

---------------------------------------------------------------------- Marker Marker represents a position in the text that will be scanned.

type MatchFunc 

type MatchFunc func(rune) bool

---------------------------------------------------------------------- match Type of functions to check a rune.

type Node 

type Node struct {
	Segment
	Nodes []*Node
}

Node

func (*Node) Equal 

func (n *Node) Equal(oth *Node) bool

Equal

type ReadError 

type ReadError struct {
	Marker
	What string
}

Error type that ReadFunc and the Reader here return.

func (ReadError) Error 

func (e ReadError) Error() string

Error function to match the error interface.

func (ReadError) Later 

func (e ReadError) Later(oth ReadError) bool

Later checks if e occurred later as oth.

type ReadFunc 

type ReadFunc func(*Scanner) error

ReadFunc represents the prototype of a read function.

type Reader 

type Reader interface {
	Read(s *Scanner) error
	What() string
}

Reader can be used by the scanner to read from the scanner.

func Any 

func Any(list ...interface{}) Reader

Any creates a Reader that tries to Read with any of the given Reader. The type of the list values can be rune, string or Reader. The first Reader that reads without an error will be used.

func AnyFold 

func AnyFold(list ...string) Reader

AnyFold creates a Reader that tries to Read any of the strings in list. The first string that matches under Unicode case-folding will be used.

func AnyRune 

func AnyRune(str string) Reader

AnyRune creates a Reader that tries to Read any of the runes in list.

func At 

func At(r Reader) Reader

At creates a Reader that checks the current postion of the scanner. The Reader does not move the scanner.

func AtEnd 

func AtEnd() Reader

At creates a Reader that checks the scanner reaches the end.

func Between 

func Between(min rune, max rune) Reader

Between creates a Reader that tries to Read a rune that is >= min and <= max.

func BetweenAny 

func BetweenAny(str string) Reader

BetweenAny creates a Reader that tries to Read a rune that is between any of the ranges that str describes. A range can look like "a-z", multible ranges can look like "a-zA-Z0-9". An invalid str value that can't be interpreted lead to an invalid reader.

func Body 

func Body(body, tail Reader) Reader

Body creates a Reader that ends before something that matches tail and all runes inbetween can be read with body.

func BodyTail 

func BodyTail(body, tail Reader) Reader

BodyTail creates a Reader that ends with something that matches tail and all runes inbetween can be read with body.

func BuildBetweenAny 

func BuildBetweenAny(minMax ...rune) Reader

BuildBetweenAny creates a Reader that tries to Read a rune that is between any of the ranges that minMax descibe. The number minMax arguments must be even, an invalid number of minMax values lead to an invalid reader.

func Digit 

func Digit() Reader

------------------------------------------------------------------------------ Digit creates a Reader that reads the digit runes '0'-'9'

func Fold 

func Fold(str string) Reader

AnyFold creates a Reader that tries to Read a string under Unicode case-folding.

func HexDigit 

func HexDigit() Reader

------------------------------------------------------------------------------ HexDigit creates a Reader that reads the hex digit runes '0'-'9'/'a'-'f'/'A'-'F'.

func Holey 

func Holey(min rune, max rune, holes string) Reader

Holey creates a Reader that tries to Read a rune that is >= min and <= max without the runes in holes.

func Lit 

func Lit(str string) Reader

Lit creates a Reader that tries to read the string str.

func Many 

func Many(i interface{}) Reader

Many creates a Reader that expects that i matches one or more times. The type of i can be rune, string or Reader. See Zom for a Reader that expects zero or more.

func Map 

func Map(r Reader, f MapFunc) Reader

Map creates a Reader that passed the Token that r reads forward to f.

func Match 

func Match(what string, f MatchFunc) Reader

Match creates a Reader that tries to read a rune that matches by f.

func Monitor 

func Monitor(r Reader, l *Log, info string) Reader

func NL 

func NL() Reader

------------------------------------------------------------------------------ NL creates a Reader to read new lines.

func Named 

func Named(name string, r Reader) Reader

Named creates a Reader with a custom name that the function What returns.

func Not 

func Not(r Reader) Reader

Not creates a Reader that moves 1 Rune forward if r does not match.

func Opt 

func Opt(i interface{}) Reader

Opt creates Reader that catches the error that i can produce and returns allways nil. The type of i can be rune, string or Reader.

func Past 

func Past(i interface{}) Reader

Past creates a Reader that reads until i matches, with the matched part. The type of i can be rune, string or Reader.

func Pick 

func Pick(r Reader, b *Basket, info string) Reader

Pick creates a Reader that appends the Segments that r reads forward to the Basket with info as Info value.

func RuleName 

func RuleName() Reader

RuleName returns a Reader to validate a Name field of a RuleReader.

func Rune 

func Rune(r rune) Reader

Rune creates a Reader that tries to read r.

func Seq 

func Seq(list ...interface{}) Reader

Seq creates a Reader that tries to Read with all readers in list sequential. The type of the list values can be rune, string or Reader.

func Set 

func Set(ranges string, singles string) Reader

Set creates a Reader that tries to Read a rune that is between any of the ranges and singles describe. A range can look like "a-z", multible ranges can look like "a-zA-Z0-9". A single value is a range that covers a single value. An invalid ranges value that can't be interpreted lead to an invalid reader.

func SkipSeq 

func SkipSeq(skip Reader, list ...interface{}) Reader

SkipSeq creates a Reader that tries to Read with all readers in list sequential and skip add the beginning, between and the end everything that matches skip. The type of the list values can be rune, string or Reader.

func SkipWSSeq 

func SkipWSSeq(list ...interface{}) Reader

SkipSeq creates a Reader that tries to Read with all readers in list sequential and skip whitespaces add the beginning, between and the end. The type of the list values can be rune, string or Reader.

func Times 

func Times(n int, r Reader) Reader

Times creates a Reader that tries to Read n times with r.

func To 

func To(i interface{}) Reader

To creates a Reader that reads until i matches, without the matched part. The type of i can be rune, string or Reader.

func WS 

func WS() Reader

------------------------------------------------------------------------------ WS creates a Reader to read one whitespace character(" \r\n\t").

func Wrap 

func Wrap(what string, f ReadFunc) Reader

Wrap creates a Reader that wraps f.

func Zom 

func Zom(i interface{}) Reader

Zom creates a Reader that expects that i matches zero or more times. The type of i can be rune, string or Reader. See Many for a Reader that expects one or more.

type Rule 

type Rule struct {
	Name   string
	Reader Reader
}

------------------------------------------------------------------------------ RuleReader can be used to set the rules of a grammar.

func CollectRules 

func CollectRules(g Grammar) []*Rule

CollectRules collects the Rules in a Grammar.

func (*Rule) Map 

func (r *Rule) Map(f MapFunc)

Map connects the move of a Reader to the function f.

func (*Rule) Monitor 

func (r *Rule) Monitor(l *Log)

func (*Rule) Pick 

func (r *Rule) Pick(basket *Basket)

Pick collects the Segments if a Reader was moven and sets the Info field with the Reader Name.

func (*Rule) Read 

func (r *Rule) Read(s *Scanner) error

func (*Rule) Rule 

func (r *Rule) Rule() string

func (*Rule) What 

func (r *Rule) What() string

type Scanner 

type Scanner struct {
	Tracker Tracker
	// contains filtered or unexported fields
}

func NewRevScanner 

func NewRevScanner(str string) *Scanner

----------------------------------------------------------------------------- Creates a new Scanner to scan the str string and moves the scanner to the end.

func NewScanner 

func NewScanner(str string) *Scanner

Creates a new Scanner to scan the str string.

func (*Scanner) AtEnd 

func (s *Scanner) AtEnd() bool

Returns true if s is at the end, otherwise false.

func (*Scanner) AtStart 

func (s *Scanner) AtStart() bool

Returns true if s is at the start, otherwise false.

func (*Scanner) BoolErrorFor 

func (s *Scanner) BoolErrorFor(ok bool, name string) error

Generates a ErrorFor if ok is false, otherwise returns the function nil.

func (*Scanner) Capture 

func (s *Scanner) Capture(f ScopeFunc) (string, bool)

Capture returns the sub string that was scanned by f.

func (*Scanner) CaptureUse 

func (s *Scanner) CaptureUse(r Reader) (string, error)

CaptureUse returns the sub string that was scanned by r.

func (*Scanner) ErrorFor 

func (s *Scanner) ErrorFor(name string) error

Generates a ReadError for name.

func (*Scanner) Get 

func (s *Scanner) Get(t Token) string

Returns the sub string that t represents.

func (*Scanner) Head 

func (s *Scanner) Head() string

Returns the left side from the current position in s.

func (*Scanner) If 

func (s *Scanner) If(str string) bool

---------------------------------------------------------------------- string Moves s the length of str forward if Tail() has str as the prefix. Returns true if s was moved, otherwise false.

func (*Scanner) IfAny 

func (s *Scanner) IfAny(strs ...string) bool

Moves s the length of the value in strs forward if Tail() that is the prefix. Returns true if s was moved, otherwise false.

func (*Scanner) IfAnyRune 

func (s *Scanner) IfAnyRune(str string) bool

--------------------------------------------------------------------- anyrune Moves s one rune forward if the first rune in Tail() is any of the runes in str. Returns true if s was moved, otherwise false.

func (*Scanner) IfBetween 

func (s *Scanner) IfBetween(min, max rune) bool

Moves s one rune forward if the first rune in Tail() is >= min and <= max. Returns true if s was moved, otherwise false.

func (*Scanner) IfFold 

func (s *Scanner) IfFold(str string) bool

------------------------------------------------------------------------ fold Moves s the length of str forward if Tail() has str as the prefix under Unicode case-folding. Returns true if s was moved, otherwise false.

func (*Scanner) IfMatch 

func (s *Scanner) IfMatch(check MatchFunc) bool

Moves s one rune value forward if the first rune in Tail() passes the check. Returns true if s was moved, otherwise false.

func (*Scanner) IfRune 

func (s *Scanner) IfRune(r rune) bool

------------------------------------------------------------------------ rune Moves s one rune value forward if the first rune in Tail() equals r. Returns true if s was moved, otherwise false.

func (*Scanner) LineCol 

func (s *Scanner) LineCol(tab int) (int, int)

Returns the current line and column in the full string.

func (*Scanner) Mark 

func (s *Scanner) Mark() Marker

Returns a Marker to mark the current positon in the text.

func (*Scanner) Move 

func (s *Scanner) Move(n int) bool

A positive value moves s n bytes to the right, a negative value moves s n bytes to the left.

func (*Scanner) MoveRunes 

func (s *Scanner) MoveRunes(n int) bool

A positve value moves s n runes to the right, a negative value moves s n runes to the left.

func (*Scanner) NewBasket 

func (s *Scanner) NewBasket() *Basket

Returns a new empty Basket that is coupled as Tracker on the scanner.

func (*Scanner) NewBasketFor 

func (s *Scanner) NewBasketFor(g Grammar) *Basket

Returns a new empty Basket that is coupled as Tracker on the scanner. Each Rule that matches picks the Segment to the Basket.

func (*Scanner) ReadBool 

func (s *Scanner) ReadBool(format string) (bool, error)

ReadBool reads bool value from the scanner. Valid format values are - "l" for true and false - "U" for TRUE and FALSE - "Cc" for True and False - "*" for all cases An empty format string will be interpreted as "*".

func (*Scanner) ReadInt 

func (s *Scanner) ReadInt(base int, bitSize int) (int64, error)

ReadInt reads a integer value from the scanner. Valid base values are 8. 10 and 16. Valid bitSize values are 8, 16, 32 and 64.

func (*Scanner) ReadRune 

func (s *Scanner) ReadRune() (rune, error)

ReadRune reads one rune value from the scanner.

func (*Scanner) ReadUint 

func (s *Scanner) ReadUint(base int, bitSize int) (uint64, error)

ReadUint reads a unsigned integer value from the scanner. Valid base values are 8, 10 and 16. Valid bitSize values are 8, 16, 32 and 64.

func (*Scanner) RevIf 

func (s *Scanner) RevIf(str string) bool

---------------------------------------------------------------------- string Moves s the length of str backward if Head() has str as the prefix. Returns true if s was moved, otherwise false.

func (*Scanner) RevIfAny 

func (s *Scanner) RevIfAny(strs []string) bool

Moves s the length fo the value in strs backward if Head() is the suffix. Returns true if s was moved, otherwise false.

func (*Scanner) RevIfAnyRune 

func (s *Scanner) RevIfAnyRune(str string) bool

--------------------------------------------------------------------- anyrune Moves s one rune backward if the last rune in Head() is any of the runes in str. Returns true if s was moved, otherwise false.

func (*Scanner) RevIfBetween 

func (s *Scanner) RevIfBetween(min, max rune) bool

--------------------------------------------------------------------- between Moves s one rune backward if the last rune in Head() is >= min and <= max. Returns true if s was moved, otherwise false.

func (*Scanner) RevIfFold 

func (s *Scanner) RevIfFold(str string) bool

------------------------------------------------------------------------ fold Moves s the length of str backward if Head() has str as the suffix under Unicode case-folding. Returns true if s was moved, otherwise false.

func (*Scanner) RevIfMatch 

func (s *Scanner) RevIfMatch(check MatchFunc) bool

---------------------------------------------------------------------- match Moves s one rune backward if the last rune in Head() passes the check. Returns true if s was moved, otherwise false.

func (*Scanner) RevIfRune 

func (s *Scanner) RevIfRune(r rune) bool

------------------------------------------------------------------------ rune Moves s one rune value backward if the last rune in Head() equals r. Returns true if s was moved, otherwise false.

func (*Scanner) RevReadRune 

func (s *Scanner) RevReadRune() (rune, error)

RevReadRune reads one rune value from the scanner the reverse way.

func (*Scanner) RevTo 

func (s *Scanner) RevTo(str string) bool

Moves s to the last appearance of str in Head(). Returns true if s was moved, otherwise false.

func (*Scanner) RevToAnyRune 

func (s *Scanner) RevToAnyRune(str string) bool

Moves s to the last rune in Head() that matches any of the runes in str. Returns true if a match was found, otherwise false.

func (*Scanner) RevToBetween 

func (s *Scanner) RevToBetween(min, max rune) bool

Moves s to the last rune in Head() that is >= min and <= max. Returns true if s was moved, otherwise false.

func (*Scanner) RevToFold 

func (s *Scanner) RevToFold(str string) bool

Moves s to the last appearance of str in Head() under Unicode case-folding. Returns true if s was moved, otherwise false.

func (*Scanner) RevToMatch 

func (s *Scanner) RevToMatch(check MatchFunc) bool

Moves s to the last rune in Head() that passes the ckeck. Returns true if s was moved, otherwise false.

func (*Scanner) RevToRune 

func (s *Scanner) RevToRune(r rune) bool

Moves s to the last rune in Head() that matches with r. The function does not move s if not values matches with r. Returns true if a match was found, otherwise false.

func (*Scanner) RevWhileAnyRune 

func (s *Scanner) RevWhileAnyRune(str string) bool

Moves s to the last rune in Head() that does not match any of the runes in str. Returns true if s was moved, otherwise false.

func (*Scanner) RevWhileBetween 

func (s *Scanner) RevWhileBetween(min, max rune) bool

Moves s to the last rune in Head() that is < min or > max. Returns true if s was moved, otherwise false.

func (*Scanner) RevWhileMatch 

func (s *Scanner) RevWhileMatch(check MatchFunc) bool

Moves s to the last rune in Head() that does not pass the ckeck. Returns true if s was moved, otherwise false.

func (*Scanner) RevWhileRune 

func (s *Scanner) RevWhileRune(r rune) bool

Moves s to the last rune in Head() that does not match with r. Returns true if s was moved, otherwise false.

func (*Scanner) ScanString 

func (s *Scanner) ScanString(n int) (string, bool)

ScanString reads n runes as string from the scanner. The scanner will only be moved if all n runes can be read from the scanner. Returns true if s was moved, otherwise false.

func (*Scanner) Scope 

func (s *Scanner) Scope(f ScopeFunc) bool

Scopes a lambda functions and moves s back if f does not success.

func (*Scanner) Segmentate 

func (s *Scanner) Segmentate(segments []Segment) ([]Segment, error)

Segmentate splits the full string of a Scanner into segments.

func (*Scanner) Since 

func (s *Scanner) Since(m Marker) string

Returns a sub string from the text that will be scanned.

func (*Scanner) Tail 

func (s *Scanner) Tail() string

----------------------------------------------------------------------- state Returns the right side from the current position in s.

func (*Scanner) To 

func (s *Scanner) To(str string) bool

Moves s to the first appearance of str in Tail(). Returns true if s was moved, otherwise false.

func (*Scanner) ToAnyRune 

func (s *Scanner) ToAnyRune(str string) bool

Moves s to the first rune in Tail() that matches any of the runes in str. Returns true if a match was found, otherwise false.

func (*Scanner) ToBetween 

func (s *Scanner) ToBetween(min, max rune) bool

Moves s to the first rune in Tail() that is >= min and <= max. Returns true if a match was found, otherwise false.

func (*Scanner) ToEnd 

func (s *Scanner) ToEnd() bool

Moves s to the end of the text that should be scanned.

func (*Scanner) ToFold 

func (s *Scanner) ToFold(str string) bool

Moves s to the first appearance of str in Tail() under Unicode case-folding. Returns true if s was moved, otherwise false.

func (*Scanner) ToMarker 

func (s *Scanner) ToMarker(m Marker) bool

Moves s to the marked position. Returns true if s was moved, otherwise false.

func (*Scanner) ToMatch 

func (s *Scanner) ToMatch(check MatchFunc) bool

Moves s to the first rune in Tail() that passes the check. Returns true if s was moved, otherwise false.

func (*Scanner) ToRune 

func (s *Scanner) ToRune(r rune) bool

Moves s to the first rune in Tail() that matches with r. The function does not move s if no value matches with r. Returns true if a match was found, otherwise false.

func (*Scanner) ToStart 

func (s *Scanner) ToStart() bool

Moves s to the start of the text that should be scanned.

func (*Scanner) Tokenize 

func (s *Scanner) Tokenize(f ScopeFunc) (Token, bool)

------------------------------------------------------------------------------ Tokenize marks the sub string that was scanned by f.

func (*Scanner) TokenizeUse 

func (s *Scanner) TokenizeUse(r Reader) (Token, error)

TokenizeUse marks the sub string that was read by r.

func (*Scanner) Trace 

func (s *Scanner) Trace(f ScopeFunc) (string, bool)

Returns the sub string that was scanned by f.

func (*Scanner) TraceUse 

func (s *Scanner) TraceUse(r Reader) (string, error)

TraceUse traces the readed sub string.

func (*Scanner) TraceUseFunc 

func (s *Scanner) TraceUseFunc(f ReadFunc) (string, error)

TraceUseFunc traces the via f traced sub string.

func (*Scanner) Use 

func (s *Scanner) Use(r Reader) error

Use uses r on the scanner. The scanner is only moved if no error occurs.

func (*Scanner) UseFunc 

func (s *Scanner) UseFunc(f ReadFunc) error

UseFunc uses f on the scanner. The scanner is only moved if no error occurs.

func (*Scanner) While 

func (s *Scanner) While(f ScopeFunc) bool

Repeats f until it returns false.

func (*Scanner) WhileAnyRune 

func (s *Scanner) WhileAnyRune(str string) bool

Moves s to the first rune in Tail() that does not match any of the runes in str. Returns true if s was moved, otherwise false.

func (*Scanner) WhileBetween 

func (s *Scanner) WhileBetween(min, max rune) bool

Moves s to the first rune in Tail() that is < min or > max. Returns true if s was moved, otherwise false.

func (*Scanner) WhileMatch 

func (s *Scanner) WhileMatch(check MatchFunc) bool

Moves s to the first rune in Tail() that does not pass the check. Returns true if s was moved, otherwise false.

func (*Scanner) WhileRune 

func (s *Scanner) WhileRune(r rune) bool

Moves s to the first rune in Tail() that does not match with r. Returns true if s was moved, otherwise false.

type ScopeFunc 

type ScopeFunc func() bool

Type of functions to mark the use of the scanner. The main use case are lambda functions.

type Segment 

type Segment struct {
	Info string
	Token
}

------------------------------------------------------------------------------ Segment is a Token with additional Meta-Information that can be stored in Info.

func (Segment) Known 

func (seg Segment) Known() bool

Known reports if information about this segment exist.

func (Segment) Segmentate 

func (s Segment) Segmentate(segments []Segment) ([]Segment, error)

Segmentate splits a segment into subsegments.

func (Segment) Split 

func (v Segment) Split(sep Segment) (Segment, Segment)

Split splits a Segment into two parts via sep.

func (Segment) String 

func (v Segment) String() string

String returns a readable representation of a Segment.

type Token 

type Token struct {
	// contains filtered or unexported fields
}

Token marks a sub string in a Scanner.

func MakeToken 

func MakeToken(a, b Marker) Token

Generates a Token from two Marker.

func (Token) After 

func (t Token) After(oth Token) bool

After reports whether t is after oth.

func (Token) Before 

func (t Token) Before(oth Token) bool

Before reports whether t is before oth.

func (Token) Clashes 

func (t Token) Clashes(oth Token) bool

Clash returns true if both tokens overlap, but no one covers the other.

func (Token) Covers 

func (t Token) Covers(sub Token) bool

Covers returns true if sub is a sub Token of t.

func (Token) Len 

func (t Token) Len() int

func (Token) Merge 

func (t Token) Merge(oth Token) Token

Merge creates a Token that convers both Tokens t and oth.

func (Token) Split 

func (t Token) Split(sep Token) (Token, Token)

Split splits a Token into two parts via sep.

func (Token) String 

func (t Token) String() string

String returns a readable representation of a Token.

type Tracker 

type Tracker interface {
	Update(m Marker)
}

Tracker is an interface that can be coupled with a Scannar and track the movemend.

type UintReader 

type UintReader struct {
	Value   uint64
	Base    int
	BitSize int
}

------------------------------------------------------------------------------ UintReader is a Reader that stores the readed uint value in the field Value.

func Uint 

func Uint(base int, bitSize int) *UintReader

Uint creates a Reader to Read uint values from the scanner. Valid base values are 8, 10 and 16. Valid bitSize values are 8, 16, 32 and 64.

func (*UintReader) Read 

func (r *UintReader) Read(s *Scanner) error

func (*UintReader) What 

func (r *UintReader) What() string

Source Files

View all Source files

Directories

Path Synopsis
demo
full-lua-stack
json-stack

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.