token

package
v0.0.0-...-9f34108 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 31, 2018 License: MIT Imports: 8 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ListTyped 

func ListTyped() []string

ListTyped returns a list of all registered typed token names.

func LoopExists 

func LoopExists(root Token) bool

LoopExists determines if a cycle exists in the internal token graph

func PrettyPrintInternalTree 

func PrettyPrintInternalTree(w io.Writer, root Token)

PrettyPrintInternalTree prints the internal token represenation as a text tree

func PrettyPrintTree 

func PrettyPrintTree(w io.Writer, root Token)

PrettyPrintTree prints the token represenation as a text tree

func RegisterTyped 

func RegisterTyped(name string, ct CreateTypedFunc)

RegisterTyped registers a typed token creation function with the given name.

func ReleaseTokens 

func ReleaseTokens(root Token)

ReleaseTokens traverses the token graph and calls Release for every release token

func ResetCombinedScope 

func ResetCombinedScope(root Token)

ResetCombinedScope resets all scopes of external and interal tokens in the token graph that fullfill the ScopeToken interface

func ResetInternalScope 

func ResetInternalScope(root Token)

ResetInternalScope resets all scopes of interal tokens in the token graph that fullfill the ScopeToken interface

func ResetResetTokens 

func ResetResetTokens(root Token) (err error)

ResetResetTokens resets all tokens in the token graph that fullfill the ResetToken interface

func ResetScope 

func ResetScope(root Token)

ResetScope resets all scopes of tokens in the token graph that fullfill the ScopeToken interface

func SetCombinedScope 

func SetCombinedScope(root Token, scope *VariableScope)

SetCombinedScope sets all scopes of external and internal tokens in the token graph that fullfill the ScopeToken interface

func SetInternalScope 

func SetInternalScope(root Token, scope *VariableScope)

SetInternalScope sets all scopes of internal tokens in the token graph that fullfill the ScopeToken interface

func SetScope 

func SetScope(root Token, scope *VariableScope)

SetScope sets all scopes of tokens in the token graph that fullfill the ScopeToken interface

func Walk 

func Walk(root Token, walkFunc func(tok Token) error) error

Walk traverses a token graph beginning from the given token and calls for every newly visited token the given function. A depth-first algorithm is used to traverse the graph. If the given walk function returns an error, the whole walk process ends by returning the error back to the caller

func WalkInternal 

func WalkInternal(root Token, walkFunc func(tok Token) error) error

WalkInternal traverses a internal token graph beginning from the given token and calls for every newly visited token the given function. A depth-first algorithm is used to traverse the graph. If the given walk function returns an error, the whole walk process ends by returning the error back to the caller

func WalkInternalTail 

func WalkInternalTail(root Token, walkFunc func(tok Token) error) error

WalkInternalTail traverses a internal token graph beginning from the given token and calls for every newly visited token the given function after it has traversed all children. A depth-first algorithm is used to traverse the graph. If the given walk function returns an error, the whole walk process ends by returning the error back to the caller

Types

type ArgumentsTypedParser 

type ArgumentsTypedParser interface {
	// GetInt tries to parse the argument name and returns its integer value or defaultValue if the argument is not found.
	// The return value is valid only if Err returns nil.
	GetInt(name string, defaultValue int) int
	// Err returns the first error encountered by the ArgumentsTypedParser.
	Err() error
}

ArgumentsTypedParser defines a parser for the arguments of a typed token. Parsing stops unrecoverably at the first error. The return value of Err must be checked before using the values returned by precedings calls to GetInt.

type CreateTypedFunc 

type CreateTypedFunc func(argParser ArgumentsTypedParser) (Token, error)

CreateTypedFunc defines a function to create a typed token

type Follow 

type Follow interface {
	// Follow returns if the children of the token should be traversed
	Follow() bool
}

Follow defines if the children of a token should be traversed

type Forward 

type Forward interface {
	// Get returns the current referenced token
	Get() Token

	// InternalGet returns the current referenced internal token
	InternalGet() Token
	// InternalLogicalRemove removes the referenced internal token and returns the replacement for the current token or nil if the current token should be removed.
	InternalLogicalRemove(tok Token) Token
}

Forward defines a forward token which can reference another token

type ForwardToken 

type ForwardToken interface {
	Token
	Forward
	InternalReplace
}

ForwardToken combines the Token and Forward interface

type Index 

type Index interface {
	// Index returns the index of this token in its parent token
	Index() int
}

Index defines an index token which provides the index in its parent token

type IndexToken 

type IndexToken interface {
	Token
	Index
}

IndexToken combines the Token and Index interface

type InternalParser 

type InternalParser struct {
	Data    string
	DataLen int
}

InternalParser holds the data information for an internal parser

func (InternalParser) GetPosition 

func (p InternalParser) GetPosition(i int) scanner.Position

GetPosition returns a text position in the data given an index of the data

type InternalReplace 

type InternalReplace interface {
	// InternalReplace replaces an old with a new internal token if it is referenced by this token. The error return argument is not nil, if the replacement is not suitable.. The error return argument is not nil, if the replacement is not suitable.
	InternalReplace(oldToken, newToken Token) error
}

InternalReplace defines if a token has methods to replace internal tokens

type Len 

type Len interface {
	// Len returns the number of the current referenced tokens
	Len() int
}

Len defines a general len token

type LenToken 

type LenToken interface {
	Token
	Len
}

LenToken combines the Token and Len interface

type List 

type List interface {
	// Get returns the current referenced token at the given index. The error return argument is not nil, if the index is out of bound.
	Get(i int) (Token, error)
	// Len returns the number of the current referenced tokens
	Len() int

	// InternalGet returns the current referenced internal token at the given index. The error return argument is not nil, if the index is out of bound.
	InternalGet(i int) (Token, error)
	// InternalLen returns the number of referenced internal tokens
	InternalLen() int
	// InternalLogicalRemove removes the referenced internal token and returns the replacement for the current token or nil if the current token should be removed.
	InternalLogicalRemove(tok Token) Token
}

List defines a general list token

type ListToken 

type ListToken interface {
	Token
	List
	InternalReplace
}

ListToken combines the Token and List interface

type Minimize 

type Minimize interface {
	// Minimize tries to minimize itself and returns a token if it was successful, or nil if there was nothing to minimize
	Minimize() Token
}

Minimize defines a minimize token which has methods to reduce itself to easier constructs

type MinimizeToken 

type MinimizeToken interface {
	Token
	Minimize
}

MinimizeToken combines the Token and Minimize interface

type Optional 

type Optional interface {
	// IsOptional checks dynamically if this token is in the current state optional
	IsOptional() bool
	// Activate activates this token
	Activate()
	// Deactivate deactivates this token
	Deactivate()
}

Optional defines an optional token which can be (de)activated

type OptionalToken 

type OptionalToken interface {
	Token
	Optional
}

OptionalToken combines the Token and Optional interface

type ParserError 

type ParserError struct {
	Message string
	Type    ParserErrorType

	Position scanner.Position
}

ParserError holds a parser error

func (*ParserError) Error 

func (err *ParserError) Error() string

type ParserErrorType 

type ParserErrorType int

ParserErrorType the parser error type 

const (
	// ParseErrorNoStart no Start token was defined
	ParseErrorNoStart ParserErrorType = iota
	// ParseErrorNewLineNeeded a new line is needed
	ParseErrorNewLineNeeded
	// ParseErrorEarlyNewLine an unexpected new line was encountered
	ParseErrorEarlyNewLine
	// ParseErrorEmptyExpressionIsInvalid empty expressions are not allowed
	ParseErrorEmptyExpressionIsInvalid
	// ParseErrorEmptyString an empty string was detected which is not allowed
	ParseErrorEmptyString
	// ParseErrorEmptyTokenDefinition empty token definitions are not allowed
	ParseErrorEmptyTokenDefinition
	// ParseErrorInvalidArgumentValue invalid argument value
	ParseErrorInvalidArgumentValue
	// ParseErrorInvalidTokenName invalid token name
	ParseErrorInvalidTokenName
	// ParseErrorInvalidTokenType invalid token type
	ParseErrorInvalidTokenType
	// ParseErrorUnusedToken token is unused
	ParseErrorUnusedToken
	// ParseErrorMissingTypedTokenArgument a typed token argument is missing
	ParseErrorMissingTypedTokenArgument
	// ParseErrorNonTerminatedString string is not properly terminated
	ParseErrorNonTerminatedString
	// ParseErrorNoTokenForVariable variable is not assigned to a token
	ParseErrorNoTokenForVariable
	// ParseErrorNotAlwaysUsedAsAVariable token is not always used as a variable but at least sometimes in a variable context
	ParseErrorNotAlwaysUsedAsAVariable
	// ParseErrorRepeatWithOptionalTerm a repeat with an optional term was detected, which is forbidden
	ParseErrorRepeatWithOptionalTerm
	// ParseErrorTokenAlreadyDefined token name is already in use
	ParseErrorTokenAlreadyDefined
	// ParseErrorTokenNotDefined there is no token with this name
	ParseErrorTokenNotDefined
	// ParseErrorTypeNotDefinedForTypedToken type is not defined for this typed token
	ParseErrorTypeNotDefinedForTypedToken
	// ParseErrorExpectRune the given rune would be expected
	ParseErrorExpectRune
	// ParseErrorExpectOperator the given operator would be expected
	ParseErrorExpectOperator
	// ParseErrorUnknownBooleanOperator the boolean operator is unknown
	ParseErrorUnknownBooleanOperator
	// ParseErrorUnknownCondition the condition is unknown
	ParseErrorUnknownCondition
	// ParseErrorUnkownOperator the operator is unknown
	ParseErrorUnkownOperator
	// ParseErrorUnknownTypedTokenArgument the typed token argument is unknown
	ParseErrorUnknownTypedTokenArgument
	// ParseErrorUnknownTypedTokenType the typed token type is unknown
	ParseErrorUnknownTypedTokenType
	// ParseErrorUnknownTokenAttribute the token attribute is unknown
	ParseErrorUnknownTokenAttribute
	// ParseErrorUnexpectedTokenDefinitionTermination token definition was unexpectedly terminated
	ParseErrorUnexpectedTokenDefinitionTermination
	// ParseErrorExpectedExpressionTerm expression term is expected
	ParseErrorExpectedExpressionTerm
	// ParseErrEndlessLoopDetected an invalid loop was detected
	ParseErrEndlessLoopDetected

	// ParseErrorExpectedEOF expected EOF
	ParseErrorExpectedEOF
	// ParseErrorRootIsNil root token is nil
	ParseErrorRootIsNil
	// ParseErrorUnexpectedEOF EOF was not expected
	ParseErrorUnexpectedEOF
	// ParseErrorUnexpectedData additional data was not expected
	ParseErrorUnexpectedData
)

func (ParserErrorType) String 

func (i ParserErrorType) String() string

type PermutationError 

type PermutationError struct {
	Type PermutationErrorType
}

PermutationError holds a permutation error

func (*PermutationError) Error 

func (err *PermutationError) Error() string

type PermutationErrorType 

type PermutationErrorType int

PermutationErrorType the permutation error type 

const (
	// PermutationErrorIndexOutOfBound an index not in the bound of available permutations was used.
	PermutationErrorIndexOutOfBound PermutationErrorType = iota
)

type Pointer 

type Pointer interface {
	Forward

	// Set sets the referenced token which must conform to the pointers token reference type
	Set(o Token) error
}

Pointer defines a pointer token which can reference another token and can be reset to reference another token

type PointerToken 

type PointerToken interface {
	Token
	Pointer
}

PointerToken combines the Token and Pointer interface

type Reduce 

type Reduce interface {
	// Reduce sets a specific reduction for this token
	Reduce(i uint) error
	// Reduces returns the number of reductions for this token
	Reduces() uint
}

Reduce defines a reduce token which provides methods to reduce itself and its children

type ReduceError 

type ReduceError struct {
	Type ReduceErrorType
}

ReduceError holds a reduce error

func (*ReduceError) Error 

func (err *ReduceError) Error() string

type ReduceErrorType 

type ReduceErrorType int

ReduceErrorType the reduce error type 

const (
	// ReduceErrorIndexOutOfBound an index not in the bound of available reductions was used.
	ReduceErrorIndexOutOfBound ReduceErrorType = iota
)

type ReduceToken 

type ReduceToken interface {
	Token
	Reduce
}

ReduceToken combines the Token and Reduce interface

type Release 

type Release interface {
	// Release gives the token a chance to remove resources
	Release()
}

Release defines a release token which provides methods to release resources on removal

type ReleaseToken 

type ReleaseToken interface {
	Token
	Release
}

ReleaseToken combines the Token and Release interface

type Reset 

type Reset interface {
	// Reset resets the (internal) state of this token and its dependences, returns an error if the reseted state should not be used for a generation.
	Reset() error
}

Reset defines a reset token which can reset its (internal) state

type ResetToken 

type ResetToken interface {
	Token
	Reset
}

ResetToken combines the Token and Index interface

type Resolve 

type Resolve interface {
	// Resolve returns the token which is referenced by the token, or a path of tokens
	Resolve() Token
}

Resolve defines if a token has methods to resolve its token path

type Scope 

type Scope interface {
	// SetScope sets the scope of the token
	SetScope(variableScope *VariableScope)
}

Scope defines a scope token which holds a scope

type ScopeToken 

type ScopeToken interface {
	Token
	Scope
}

ScopeToken combines the Token and Scope interface

type Scoping 

type Scoping interface {
	// Scoping returns if the token holds a new scope
	Scoping() bool
}

Scoping defines a scoping token which holds a new scope

type Token 

type Token interface {
	fmt.Stringer

	// Clone returns a copy of the token and all its children
	Clone() Token

	// Permutation sets a specific permutation for this token
	Permutation(i uint) error
	// Permutations returns the number of permutations for this token
	Permutations() uint
	// PermutationsAll returns the number of all possible permutations for this token including its children
	PermutationsAll() uint

	// Parse tries to parse the token beginning from the current position in the parser data.
	// If the parsing is successful the error argument is nil and the next current position after the token is returned.
	Parse(pars *InternalParser, cur int) (int, []error)
}

Token defines a general token

func MinimizeTokens 

func MinimizeTokens(root Token) (Token, error)

MinimizeTokens traverses the token graph and replaces unnecessary complicated constructs with their simpler form One good example is an All list token with one token which can be replaced by this one token. The minimize checks and operation is done by the token itself which has to implement the MinimizeToken interface, since it is not always predictable if a token with one child is doing something special,

func NewTyped 

func NewTyped(name string, argParser ArgumentsTypedParser, pos scanner.Position) (Token, error)

NewTyped creates a typed token by calling the function registered with the given name. The error return argument is not nil if the name does not exist in the registered typed token list or if the token creation failed.

func UnrollPointers 

func UnrollPointers(root Token) (Token, error)

UnrollPointers unrolls pointer tokens by copying their referenced graphs. Pointers that lead to themselves are unrolled at maximum tavor.MaxRepeat times.

type Variable 

type Variable interface {
	Forward
	Index
	Scope

	// Name returns the name of the variable
	Name() string

	// Len returns the number of the current referenced tokens
	Len() int
}

Variable defines a variable token which holds a variable

type VariableScope 

type VariableScope struct {
	// contains filtered or unexported fields
}

VariableScope holds a variable scope and a reference to its parent scope

func NewVariableScope 

func NewVariableScope() *VariableScope

NewVariableScope returns a new instance of a variable scope

func NewVariableScopeFrom 

func NewVariableScopeFrom(s map[string]Token) *VariableScope

NewVariableScopeFrom returns a new instance of a variable scope initializing the scope with the given map

func (*VariableScope) Combine 

func (s *VariableScope) Combine() map[string]Token

Combine returns a map which holds the combination of all variable scopes

func (*VariableScope) Get 

func (s *VariableScope) Get(name string) Token

Get searches the variable scope for a variable with the given name and returns the token, or nil if there is no variable with the given name

func (*VariableScope) Pop 

func (s *VariableScope) Pop() *VariableScope

Pop returns the parent scope, or panics if there is no parent scope

func (*VariableScope) Push 

func (s *VariableScope) Push() *VariableScope

Push creates a new variable scope and returns it

func (*VariableScope) Set 

func (s *VariableScope) Set(name string, tok Token)

Set sets a variable with the given name

type VariableToken 

type VariableToken interface {
	Token
	Variable
}

VariableToken combines the Token and Variable interface

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.