pipeline

Documentation

Overview

Package pipeline in Gozerian allows clients to to define and create sequential processing pipes for http requests and their associated responses. There are a couple of key concepts to understand and use this effectively:

Pipe

A Pipe represents an intermediary (proxy) during the round trip of a HTTP request from a client to a target and any functional activity or translations that may need to be made during processing of the request or response.

Unlike most proxies that have a single request handler pipeline (including the built-in handlers in Go), Gozerian takes a contrarian view that the request and response pipes should be separated in order to provide the greatest level of control and compatibility with external systems. (For example, a primary use case of Gozerian is to work as a plugin to the ngxin server.)

In addition to the request and response pipes, a pipe maintains a context and control system. This is accessible by type asserting the passed http.Writer to a pipeline.ControlHolder and calling its Control() function like so:

return func(w http.ResponseWriter, r *http.Request) {
	control := pipeline.ControlFromContext(r.Context())

Once you have a pipeline.Control, your fitting now has a variety of options available to it including logging, error handling, flow variables, etc. See the pipeline.Control documentation for more information.

Fitting

A Fitting is simply an interface that allows for custom behavior for a single step (either request or response or both) in a Pipe. The interface is:

RequestHandlerFunc() http.HandlerFunc
ResponseHandlerFunc() ResponseHandlerFunc

Either of these methods may return nil. Note that the request handler is the standard Go handler (http.RequestHandlerFunc) while a response handler is defined as a pipeline.ResponseHandlerFunc as there is no preexisting standard function. It is defined as:

func(w http.ResponseWriter, r *http.Request, res *http.Response)

Die

A Die is nothing more than a factory function to create a Fitting. A Die is registered with the system by name and is called during the initialization of a Pipe. The function signature is:

func(config interface{}) (Fitting, error)

To register a Die with the system, you need only ensure that the Die (and Fitting, of course) is compiled into the Go executable and call RegisterDie with a unique ID and the Die function like so:

pipeline.RegisterDie("dump", test_util.CreateDumpFitting)

Setup & Execution:

Configuration can easily done via a YAML file. Example:

port: 8080
target: http://httpbin.org
pipes:                      # pipe definitions
  main:                     # pipe id (as registered)
    request:                # request pipeline
    - dump:                 # name of plugin
	dumpBody: true      # plugin-specific configuration
    response:               # response pipeline
    - dump:                 # name of plugin
	dumpBody: true      # plugin-specific configuration
proxies:                    # maps host & path -> pipe
  - host: localhost         # host
    path: /                 # path
    pipe: main              # pipe to use

Then, your main file need only register your Dies, open the configuration file and execute the gateway:

pipeline.RegisterDie("dump", test_util.CreateDumpFitting)

yamlReader, err := os.Open("main.yaml")
if err != nil {
	fmt.Print(err)
}

err = go_gateway.ListenAndServe(yamlReader)
if err != nil {
	fmt.Print(err)
}

Index

• Constants
• func DefaultErrorHanderFunc(writer http.ResponseWriter, err error) error
• func NewControlContext(ctx context.Context, ctl Control) context.Context
• func RegisterDie(id string, die Die)
• func RegisterDies(m map[string]Die)
• type Control
  • func ControlFromContext(ctx context.Context) Control
• type Definition
  • func DefinePipe(r io.Reader) (Definition, error)
  • func DefinePipeDirectly(pipeConfig PipeDef) (Definition, error)
  • func NewDefinition(reqFittings []FittingWithID, resFittings []FittingWithID) Definition
• type Die
• type ErrorHandlerFunc
• type Fitting
• type FittingDef
• type FittingWithID
  • func NewFitting(dieID string, config interface{}) (FittingWithID, error)
• type FlowData
• type Logger
• type Pipe
• type PipeDef
• type ResponseHandlerFunc

Constants

const (
	ConfigTimeout  = "timeout"
	ConfigLogLevel = "logLevel"
)

Functions

func DefaultErrorHanderFunc

func DefaultErrorHanderFunc(writer http.ResponseWriter, err error) error

DefaultErrorHanderFunc for now just sends the error to the client

func NewControlContext

func NewControlContext(ctx context.Context, ctl Control) context.Context

func RegisterDie

func RegisterDie(id string, die Die)

RegisterDie associates a Die with an id

func RegisterDies

func RegisterDies(m map[string]Die)

RegisterDies associates multiple Dies with ids

Types

type Control

type Control interface {
	RequestID() string

	SetErrorHandler(eh ErrorHandlerFunc)
	ErrorHandler() ErrorHandlerFunc
	HandleError(w http.ResponseWriter, err interface{}) error

	Error() error
	Cancel()
	Cancelled() bool

	Log() Logger

	Config() config

	FlowData() FlowData
}

Control contains control and context for a Pipe instance

func ControlFromContext

func ControlFromContext(ctx context.Context) Control

type Definition

type Definition interface {
	CreatePipe() Pipe
}

Definition of a pipe

func DefinePipe

func DefinePipe(r io.Reader) (Definition, error)

DefinePipeFromReader returns a Pipe Definition as defined in the yaml Reader

func DefinePipeDirectly

func DefinePipeDirectly(pipeConfig PipeDef) (Definition, error)

func NewDefinition

func NewDefinition(reqFittings []FittingWithID, resFittings []FittingWithID) Definition

DefinePipe returns a Pipe Definition defined by the passed handlers

type Die

type Die func(config interface{}) (Fitting, error)

Die is a factory to create Fittings

type ErrorHandlerFunc

type ErrorHandlerFunc func(writer http.ResponseWriter, err error) error

ErrorHandlerFunc is a function called to handle an error in a Pipe

type Fitting

type Fitting interface {
	RequestHandlerFunc() http.HandlerFunc
	ResponseHandlerFunc() ResponseHandlerFunc
}

Fitting is a function in a Pipe

type FittingDef

type FittingDef map[string]interface{}

type FittingWithID

type FittingWithID interface {
	Fitting
	ID() string
}

func NewFitting

func NewFitting(dieID string, config interface{}) (FittingWithID, error)

NewFitting creates a fitting from a registered Die

type FlowData

type FlowData map[string]interface{}

ExtraData is just an open map for storing stuff

type Logger

type Logger interface {
	Debugf(format string, args ...interface{})
	Infof(format string, args ...interface{})
	Printf(format string, args ...interface{})
	Warnf(format string, args ...interface{})
	Warningf(format string, args ...interface{})
	Errorf(format string, args ...interface{})
	Fatalf(format string, args ...interface{})
	Panicf(format string, args ...interface{})

	Debug(args ...interface{})
	Info(args ...interface{})
	Print(args ...interface{})
	Warn(args ...interface{})
	Warning(args ...interface{})
	Error(args ...interface{})
	Fatal(args ...interface{})
	Panic(args ...interface{})

	Debugln(args ...interface{})
	Infoln(args ...interface{})
	Println(args ...interface{})
	Warnln(args ...interface{})
	Warningln(args ...interface{})
	Errorln(args ...interface{})
	Fatalln(args ...interface{})
	Panicln(args ...interface{})

	WithField(key string, value interface{}) *logrus.Entry
	WithFields(fields logrus.Fields) *logrus.Entry
}

Logger represents the logging interface

type Pipe

type Pipe interface {
	PrepareRequest(reqID string, req *http.Request) *http.Request

	// Be sure to call this with a prepared http.Request!
	RequestHandlerFunc() http.HandlerFunc

	// Be sure to call this with a prepared http.Request!
	ResponseHandlerFunc() ResponseHandlerFunc
}

Pipe runs a series of request and response handlers

type PipeDef

type PipeDef struct {
	Request  []FittingDef `yaml:"request"`
	Response []FittingDef `yaml:"response"`
}

type ResponseHandlerFunc

type ResponseHandlerFunc func(w http.ResponseWriter, r *http.Request, res *http.Response)

ResponseHandlerFunc is the part of the Pipe that handles responses