middleware

package
v0.33.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 24, 2021 License: GPL-2.0, GPL-3.0 Imports: 19 Imported by: 0

README

Middleware

Each middleware implements a function with the following signature

func (middleware CatchMiddleware) Run(
	invocation *models.PipelineInvocation,
	result *models.RunResult,
	next func(*models.PipelineInvocation, *models.RunResult),
	logger *logging.MiddlewareLogger,
	stack *middleware.Stack) {}

Built-in middleware

catch - Error Handler
dir - Directory Navigator
docker - Docker Executor
each - Input Duplicator
timer - Directory Timer Middleware

Writing your own middleware

Documentation

Overview

Package middleware provides middlewares that together implement pipedream's core functionalities

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func LookUpPipelineDefinition 

func LookUpPipelineDefinition(definitionsLookup pipeline.DefinitionsLookup, identifier string, rootFileName string) (*pipeline.Definition, bool)

LookUpPipelineDefinition looks for a particular pipeline within the already parsed deinitions

Types

type ExecutionContext 

type ExecutionContext struct {
	// SelectableFiles is a list of filenames of all pipeline files in the current directory
	//
	// Only one file can be selected for execution, but all files will be parsed.
	SelectableFiles []string
	// PipelineFiles is a list of pipeline files in the current directory, including recursive imports
	PipelineFiles []pipeline.File
	// Definitions contains all pipeline definitions in the PipelineFiles, as well as built-in pipes
	Definitions pipeline.DefinitionsLookup
	// MiddlewareStack is a list of middleware items that will be executed in turn
	MiddlewareStack []Middleware
	// Defaults contains some execution options that can be set at file level
	Defaults pipeline.DefaultSettings
	// Hooks can execute certain functions before or after pipeline execution
	//
	// Currently not implemented or used.
	Hooks pipeline.HookDefinitions

	// Log is the execution context's logger
	Log *logrus.Logger

	// ProjectPath is the directory in which the tool is currently executing
	ProjectPath string
	// RootFileName is the name of the file selected for execution
	RootFileName string

	Runs        []*pipeline.Run
	Connections []*pipeline.DataConnection

	// UserPromptImplementation by default shows an interactive prompt to the user
	//
	// Can be overwritten if you need a different implementation e.g. for tests.
	UserPromptImplementation func(
		label string,
		items []string,
		initialSelection int,
		size int,
		input io.ReadCloser,
		output io.WriteCloser,
	) (int, string, error)
	// contains filtered or unexported fields
}

ExecutionContext is the data model keeping track of everything required to execute a pipeline file

func NewExecutionContext 

func NewExecutionContext(options ...ExecutionContextOption) *ExecutionContext

NewExecutionContext creates a new ExecutionContext with the specified options

func (*ExecutionContext) CancelAll 

func (executionContext *ExecutionContext) CancelAll() error

CancelAll cancels all active runs

func (*ExecutionContext) Execute 

func (executionContext *ExecutionContext) Execute(pipelineIdentifier string, stdoutWriter io.Writer, stderrWriter io.Writer)

Execute runs a pipeline and outputs the result

func (*ExecutionContext) FullRun 

func (executionContext *ExecutionContext) FullRun(options ...FullRunOption) *pipeline.Run

FullRun starts the complete execution procedure for a nested pipeline, unwinding the entire middleware stack again

func (*ExecutionContext) PipelineFileAtPath 

func (executionContext *ExecutionContext) PipelineFileAtPath(path string) (*pipeline.File, error)

PipelineFileAtPath returns a *pipeline.File corresponding to the parsed pipeline file at the given path, if any

func (*ExecutionContext) SetUpCancelHandler added in v0.33.3 

func (executionContext *ExecutionContext) SetUpCancelHandler(stdoutWriter io.Writer, stderrWriter io.Writer, handler func())

SetUpCancelHandler registers a handler for interrupt signals

func (*ExecutionContext) SetUpPipelines 

func (executionContext *ExecutionContext) SetUpPipelines(fileFlag string) error

SetUpPipelines collects and parses all relevant pipeline files

func (*ExecutionContext) UserRuns added in v0.32.32 

func (executionContext *ExecutionContext) UserRuns() []*pipeline.Run

UserRuns lists all runs of pipes that are not built-in

func (*ExecutionContext) WaitForRun added in v0.32.32 

func (executionContext *ExecutionContext) WaitForRun(identifier string) *pipeline.Run

WaitForRun blocks until a run with the specified identifier is found and completes

type ExecutionContextOption 

type ExecutionContextOption func(*ExecutionContext)

ExecutionContextOption is an option that can be applied to an ExecutionContext or provided to the NewExecutionContext constructor

func WithDefinitionsLookup 

func WithDefinitionsLookup(definitions pipeline.DefinitionsLookup) ExecutionContextOption

WithDefinitionsLookup sets the execution context's definitions lookup

func WithExecutionFunction 

func WithExecutionFunction(executionFunction func(run *pipeline.Run)) ExecutionContextOption

WithExecutionFunction set the execution context's execution function

You don't need to replace the standard implementation, which unwinds a middleware stack, unless you want to specify a specific behaviour e.g. for tests.

func WithLogger 

func WithLogger(logger *logrus.Logger) ExecutionContextOption

WithLogger sets the execution context's logger

func WithMiddlewareStack 

func WithMiddlewareStack(stack []Middleware) ExecutionContextOption

WithMiddlewareStack sets the execution context's middleware stack

func WithParser 

func WithParser(parser *parsing.Parser) ExecutionContextOption

WithParser sets the execution context's parser

func WithProjectPath 

func WithProjectPath(projectPath string) ExecutionContextOption

WithProjectPath sets the execution context's project path

func WithUserPromptImplementation 

func WithUserPromptImplementation(implementation func(
	label string,
	items []string,
	initialSelection int,
	size int,
	input io.ReadCloser,
	output io.WriteCloser,
) (int, string, error)) ExecutionContextOption

WithUserPromptImplementation sets the execution context's implementation of a user prompt

By default, this will use promptui to show an interactive prompt to the user, but you may want to override it for tests.

type FullRunOption 

type FullRunOption func(*FullRunOptions)

FullRunOption represents an option provided to an executions context's FullRun

func WithArguments 

func WithArguments(arguments map[string]interface{}) FullRunOption

WithArguments sets the full run's arguments

func WithIdentifier 

func WithIdentifier(identifier *string) FullRunOption

WithIdentifier sets the identifier of the pipeline to be run

It will be used to look up the matching definition, if any

func WithLogWriter 

func WithLogWriter(logWriter io.WriteCloser) FullRunOption

WithLogWriter sets an io.WriteCloser to which logs will be written

func WithParentRun 

func WithParentRun(parentRun *pipeline.Run) FullRunOption

WithParentRun sets the parent run, i.e. the enclosing run that triggered the full run in question

func WithSetupFunc 

func WithSetupFunc(preCallback func(*pipeline.Run)) FullRunOption

WithSetupFunc sets a function that will be executed between the full run's setup and main execution function

func WithTearDownFunc 

func WithTearDownFunc(postCallback func(*pipeline.Run)) FullRunOption

WithTearDownFunc sets a function that will be executed after the full run's main execution function

type FullRunOptions 

type FullRunOptions struct {
	// contains filtered or unexported fields
}

FullRunOptions collects a number of different FullRunOption options into a single structure

type Middleware 

type Middleware interface {
	String() string
	Apply(
		run *pipeline.Run,
		next func(*pipeline.Run),
		executionContext *ExecutionContext,
	)
}

Middleware slices the implementation into middlewares, each performing side effects and adapting the run based on provided arguments

each interpret a slice of the run's arguments and transform provide transforms the run during execution

Source Files

View all Source files

Directories

Path Synopsis
Package catch provides a middleware for graceful handling of stderr output
 Package catch provides a middleware for graceful handling of stderr output
Package dir provides a middleware for changing the current working directory
 Package dir provides a middleware for changing the current working directory
Package docker provides a middleware for execution within a Docker (Compose) container
 Package docker provides a middleware for execution within a Docker (Compose) container
Package each provides a middleware that copies some input into several child pipes running simultaneously
 Package each provides a middleware that copies some input into several child pipes running simultaneously
Package env provides a middleware managing environment variables
 Package env provides a middleware managing environment variables
Package inherit provides a middleware that passes arguments from a parent to its children
 Package inherit provides a middleware that passes arguments from a parent to its children
Package _input provides a middleware to overwrite a pipe's input directly
 Package _input provides a middleware to overwrite a pipe's input directly
Package interpolate provides a middleware to substitute arguments or inputs into other arguments
 Package interpolate provides a middleware to substitute arguments or inputs into other arguments
Package _output provides a middleware to overwrite a pipe's output directly
 Package _output provides a middleware to overwrite a pipe's output directly
Package pipe provides a middleware to execute several commands in sequence
 Package pipe provides a middleware to execute several commands in sequence
Package _select provides a middleware that shows a selection prompt to the user
 Package _select provides a middleware that shows a selection prompt to the user
Package shell provides a middleware to execute commands in a shell
 Package shell provides a middleware to execute commands in a shell
Package ssh provides a middleware enabling remote command execution via SSH
 Package ssh provides a middleware enabling remote command execution via SSH
Package stack defines a list of middleware items to be executed when a pipeline is run
 Package stack defines a list of middleware items to be executed when a pipeline is run
Package _switch provides a middleware that replaces the output using the first match from a list of regexes
 Package _switch provides a middleware that replaces the output using the first match from a list of regexes
Package timer provides a middleware that records execution time
 Package timer provides a middleware that records execution time
Package when provides a middleware that enables conditional execution
 Package when provides a middleware that enables conditional execution
Package with provides a middleware that extracts and processes input patterns
 Package with provides a middleware that extracts and processes input patterns

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.