goprep

Documentation

Overview

This package provides a framework for creating powerful lexical preprocessors for go code.

Index

• func False(Token) bool
• func Ignore(f func(Token) bool) func(*Pipe)
• func IgnoreToken(str string) func(*Pipe)
• func IgnoreType(tok token.Token) func(*Pipe)
• func Lines(pipeline *Pipe)
• func Link(f func(chan Token, chan Token, chan string, chan interface{})) func(*Pipe)
• func Pass(f func(Token) bool) func(*Pipe)
• func PassToken(str string) func(*Pipe)
• func PassType(tok token.Token) func(*Pipe)
• func PipeEnd(p *Pipe, oWriter io.Writer)
• func True(Token) bool
• type Pipe
  • func PipeInit(iReader io.Reader) *Pipe
• type Token

Functions

func False

func False(Token) bool

False takes a Token and always returns false.

func Ignore

func Ignore(f func(Token) bool) func(*Pipe)

Ignore produces a modified input stream that does not include any tokens for which f evaluates to true, thus discarding a certain class of tokens.

func IgnoreToken

func IgnoreToken(str string) func(*Pipe)

IgnoreToken is like Ignore, discarding all tokens whose string content is equal to the given string.

func IgnoreType

func IgnoreType(tok token.Token) func(*Pipe)

IgnoreType is like Ignore, discarding all tokens of a certain type.

func Lines

func Lines(pipeline *Pipe)

Lines passes line pragma information as needed to the output channel, thus ensuring that line numbers in the output match line numbers in the input.

func Link

func Link(f func(chan Token, chan Token, chan string, chan interface{})) func(*Pipe)

Link adds the given routine to the pipeline.

func Pass

func Pass(f func(Token) bool) func(*Pipe)

Pass redirects all tokens for which f evaluates to true to the output channel, returning the altered input channel.

func PassToken

func PassToken(str string) func(*Pipe)

PassToken is like Pass, passing all tokens whose string content is equal to the given string.

func PassType

func PassType(tok token.Token) func(*Pipe)

PassType is like Pass, passing all tokens of a certain type.

func PipeEnd

func PipeEnd(p *Pipe, oWriter io.Writer)

PipeEnd implements the closing (writing) portion of a pipeline.

func True

func True(Token) bool

True takes a Token and always returns true.

Types

type Pipe

type Pipe struct {
	Input  chan Token
	Output chan string
	Sync   chan interface{}
}

Pipe represents a unit of the pipeline from Token inputs to string outputs. This would probably be a monad in haskell.

We're using channels to serve as an abstracted for-loop, not because we want parallelism. All processing is synchronized with Sync. A message to Sync signals that the token has been 'used' (sent to Output or discarded), and the sender is ready for the next. Since adding a read to sync (which is 'always' being read from by the frontend goroutine) would result in undeterministic behaviour, routines originating artificial tokens must create a new synchronization channel.

func PipeInit

func PipeInit(iReader io.Reader) *Pipe

PipeInit initializes a pipeline; input will be read from iReader.

type Token

type Token struct {
	Pos   token.Position
	Token token.Token
	Str   string
}

Represents a token as returned by scanner.Scanner.Scan(), with the position, token type, and string representation.