plugins

Documentation

Overview

Package plugins defines interfaces to be implemented by feature plugins.

A plugin is an object that gets called for each step in the task execution flow. At each step the plugin will get access to engine and runtime object, such as the SandboxBuilder, the Sandbox, the SandboxContext, etc. A plugin is then supposed to implement a specific feature using these object, this could be live logging, artifact uploading, attachment of proxies, mouting of caches, archival of caches, and many other things.

A plugin does not get any plugin specific task.payload space, instead the options required by a plugin must be part of the task.payload. Hence, all plugins must be available on all platforms. To faciliate this plugins should not use platform specific APIs, instead they should rely on interface offered by the engine objects to do so. And fail gracefully if an engine method is not supported in a given configuration and returns ErrFeatureNotSupported.

When a plugin encounters an unsupported feature that it needs, it may either return a MalformedPayloadError, or simply ignore the error and workaround it.

Plugin packages should provide a method:

NewXXXPluginFactory(engine.Engine,*runtime.EngineContext) PluginFactory

and have it registered in pluginmanager.go

At high-level PluginFactory stores the global state owned by the plugin, and Plugin stores the task-specific state owned by the plugin. A new Plugin instance will be created for each task.

In summery, plugins are not really "plugins", they are merely abstractions that allows us to implement features in complete isolation. Maybe they will become more flexible in the future, but there is no need to design for this at this point.

Index

• type ExceptionReason
• type Plugin
• type PluginBase
  • func (PluginBase) NewTaskPlugin(TaskPluginOptions) (TaskPlugin, error)
  • func (PluginBase) PayloadSchema() (runtime.CompositeSchema, error)
• type TaskPlugin
• type TaskPluginBase
  • func (TaskPluginBase) BuildSandbox(engines.SandboxBuilder) error
  • func (TaskPluginBase) Dispose() error
  • func (TaskPluginBase) Exception(reason ExceptionReason) error
  • func (TaskPluginBase) Finished(success bool) error
  • func (TaskPluginBase) Prepare(*runtime.TaskContext) error
  • func (TaskPluginBase) Started(engines.Sandbox) error
  • func (TaskPluginBase) Stopped(engines.ResultSet) (bool, error)
• type TaskPluginOptions

Types

type ExceptionReason

type ExceptionReason int

An ExceptionReason specifies the reason a task reached an exception state.

const (
	Cancelled ExceptionReason = iota
	MalformedPayload
	WorkerShutdown
)

Reasons why a task can reach an exception state. Implementors should be warned that additional entries may be added in the future.

type Plugin

type Plugin interface {
	// PayloadSchema returns the CompositeSchema that represents the payload.
	//
	// The Payload property on TaskPluginOptions given to NewTaskPlugin will be
	// the result from CompositeSchema.Parse() on the CompositeSchema returned
	// from this method.
	PayloadSchema() (runtime.CompositeSchema, error)

	// NewTaskPlugin method will be called once for each task. The TaskPlugin
	// instance returned will be called for each stage in the task execution.
	//
	// This is a poor place to do any processing, and not a great place to start
	// long-running operations as you don't have a place to write log messages.
	// Consider waiting until Prepare() is called with TaskContext that you can
	// write log messages to.
	//
	// Plugins implementing logging should not return an error here, as it
	// naturally follows that such an error can't be logged if no logging plugin
	// is created.
	//
	// Implementors may return nil, if the plugin doesn't have any hooks for the
	// given tasks.
	//
	// Non-fatal errors: MalformedPayloadError
	NewTaskPlugin(options TaskPluginOptions) (TaskPlugin, error)
}

Plugin is a plugin to the worker, for each task NewTaskPlugin is created. The Plugin instance is responsible for creating these objects and managing data shared between TaskPlugin instances.

All methods on this interface must be thread-safe.

type PluginBase

type PluginBase struct{}

PluginBase is a base implementation of the Plugin interface, it just handles all methods and does nothing.

Implementors should embed this to ensure forward compatibility when we add new optional methods.

func (PluginBase) NewTaskPlugin

func (PluginBase) NewTaskPlugin(TaskPluginOptions) (TaskPlugin, error)

NewTaskPlugin returns nil ignoring the request to create a TaskPlugin for the given task.

func (PluginBase) PayloadSchema

func (PluginBase) PayloadSchema() (runtime.CompositeSchema, error)

PayloadSchema returns an empty composite schema for plugins that doesn't take any payload.

type TaskPlugin

type TaskPlugin interface {
	// Prepare will be called in parallel with NewSandboxBuilder().
	//
	// Notice that this method is a good place to start long-running operations,
	// you then have to take care to clean-up in Dispose() if they are still
	// running. You should wait for your long-running operations to finish in
	// BuildSandbox() or whatever hook you need them in.
	//
	// Non-fatal errors: MalformedPayloadError
	Prepare(context *runtime.TaskContext) error

	// BuildSandbox is called once NewSandboxBuilder() has returned.
	//
	// This is the place to wait for downloads and other expensive operations to
	// finished, before mounting caches, proxies, etc. and returning.
	//
	// Non-fatal errors: MalformedPayloadError
	BuildSandbox(SandboxBuilder engines.SandboxBuilder) error

	// Started is called once the sandbox has started execution. This is a good
	// place to hook if you want to do interactive things.
	//
	// Non-fatal errors: MalformedPayloadError
	Started(sandbox engines.Sandbox) error

	// Stopped is called once the sandbox has terminated. Returns true, if the
	// task execution was successful.
	//
	// This is a good place to upload artifacts, logs, check exit code, and start
	// to clean-up resources if such clean-up is expected to take a while.
	//
	// Non-fatal errors: MalformedPayloadError
	Stopped(result engines.ResultSet) (bool, error)

	// Finished is called once the sandbox has terminated and Stopped() have been
	// called.
	//
	// At this stage the task-specific log is closed, and attempts to log data
	// using the previous TaskContext will fail. That makes this a good place to
	// upload logs and any processing of the logs. In fact logging is the primary
	// motivation for this stage.
	//
	// As there is no logging in this method, it's not recommend to do anything
	// that may fail here.
	Finished(success bool) error

	// Exception is called once the task is resolved exception. This may happen
	// instead of calls to Prepare(), BuildSandbox(), Started(), Stopped(), or
	// Finished().
	//
	// This is a good place for best-effort to upload artifacts and logs that you
	// wish to persist. Naturally, log messages written at this stage will be
	// dropped and all error messages will be fatal.
	//
	// Implementors should be ware that additional reasons may be added in the
	// future. Therefore they must handle the default case, if switching on the
	// reason parameter.
	Exception(reason ExceptionReason) error

	// Dispose is called once everything is done and it's time for clean-up.
	//
	// This method will invoked following Stopped() or Exception(). It is then
	// the responsibility of the implementor to abort or wait for any long-running
	// processes and clean-up any resources held.
	Dispose() error
}

TaskPlugin holds the task-specific state for a plugin

Each method on this interface represents stage in the task execution and will be called when this stage is reached. The some methods are allowed to take significant amounts of time, as they will run asynchronously.

These methods does not have to be thread-safe, we will never call the next method, before the previous method has returned.

Implementors of this interface should be sure to embed TaskPluginBase. This will do absolutely nothing, but provide empty implementations for any current and future methods that isn't implemented.

The methods are called in the order listed here with the exception of Exception() which may be called following any method, and Dispose() which will always be called as a final step allowing you to clean up.

type TaskPluginBase

type TaskPluginBase struct{}

TaskPluginBase is a base implementation of the TaskPlugin interface, it just handles all methods and does nothing.

Implementors should embed this to ensure forward compatibility when we add new optional methods.

func (TaskPluginBase) BuildSandbox

func (TaskPluginBase) BuildSandbox(engines.SandboxBuilder) error

BuildSandbox ignores the sandbox building stage.

func (TaskPluginBase) Dispose

func (TaskPluginBase) Dispose() error

Dispose ignores the stage where resources are disposed.

func (TaskPluginBase) Exception

func (TaskPluginBase) Exception(reason ExceptionReason) error

Exception ignores the stage where a task is resolved exception

func (TaskPluginBase) Finished

func (TaskPluginBase) Finished(success bool) error

Finished ignores the stage where a task has been finished

func (TaskPluginBase) Prepare

func (TaskPluginBase) Prepare(*runtime.TaskContext) error

Prepare ignores the sandbox preparation stage.

func (TaskPluginBase) Started

func (TaskPluginBase) Started(engines.Sandbox) error

Started ignores the stage where the sandbox has started

func (TaskPluginBase) Stopped

func (TaskPluginBase) Stopped(engines.ResultSet) (bool, error)

Stopped ignores the stage where the sandbox has returned a ResultSet, and returns true saying the task was successful, as not to poison the water.

type TaskPluginOptions

type TaskPluginOptions struct {
	TaskInfo *runtime.TaskInfo
	Payload  interface{}
}

The TaskPluginOptions is a wrapper for the set of arguments given to NewTaskPlugin.

We wrap the arguments in a single argument to maintain source compatibility when introducing additional arguments.