plugins

Documentation

Overview

Package plugins defines interfaces to be implemented by feature plugins.

A plugin is an object that gets called for each step in the task execution flow. At each step the plugin will get access to engine and runtime object, such as the SandboxBuilder, the Sandbox, the SandboxContext, etc. A plugin is then supposed to implement a specific feature using these objects, this could be live logging, artifact uploading, attachment of proxies, mouting of caches, archival of caches, and many other things.

A plugin does not get any plugin specific task.payload space, instead the options required by a plugin must be part of the task.payload. Hence, all plugins must be available on all platforms. To facilitate this, plugins should not use platform specific APIs, instead they should rely on interface offered by the engine objects to do so, fail gracefully if an engine method is not supported in a given configuration and returns ErrFeatureNotSupported.

When a plugin encounters an unsupported feature that it needs, it may either return a MalformedPayloadError, or simply ignore the error and workaround it.

Plugin packages should provide a method:

NewXXXPluginFactory(engine.Engine,*runtime.EngineContext) PluginFactory

and have it registered in pluginmanager.go

At high-level, PluginFactory stores the global state owned by the plugin, and Plugin stores the task-specific state owned by the plugin. A new Plugin instance will be created for each task.

In summary, plugins are not really "plugins", they are merely abstractions that allows us to implement features in complete isolation. Maybe they will become more flexible in the future, but there is no need to design for this at this point.

Index

• func PluginManagerConfigSchema() schematypes.Object
• func Plugins() map[string]PluginProvider
• func Register(name string, provider PluginProvider)
• type Plugin
  • func NewPluginManager(options PluginOptions) (Plugin, error)
• type PluginBase
  • func (PluginBase) NewTaskPlugin(TaskPluginOptions) (TaskPlugin, error)
  • func (PluginBase) PayloadSchema() schematypes.Object
• type PluginOptions
• type PluginProvider
• type PluginProviderBase
  • func (PluginProviderBase) ConfigSchema() schematypes.Schema
• type TaskPlugin
• type TaskPluginBase
  • func (TaskPluginBase) BuildSandbox(engines.SandboxBuilder) error
  • func (TaskPluginBase) Dispose() error
  • func (TaskPluginBase) Exception(reason runtime.ExceptionReason) error
  • func (TaskPluginBase) Finished(success bool) error
  • func (TaskPluginBase) Prepare(*runtime.TaskContext) error
  • func (TaskPluginBase) Started(engines.Sandbox) error
  • func (TaskPluginBase) Stopped(engines.ResultSet) (bool, error)
• type TaskPluginOptions

Functions

func PluginManagerConfigSchema

func PluginManagerConfigSchema() schematypes.Object

PluginManagerConfigSchema returns configuration for PluginOptions.Config for NewPluginManager.

func Plugins

func Plugins() map[string]PluginProvider

Plugins returns map from plugin name to PluginProviders.

func Register

func Register(name string, provider PluginProvider)

Register will register a PluginProvider. This is meant to be called from init(), and will panic if name is already used by another plugin.

Types

type Plugin

type Plugin interface {
	// PayloadSchema returns a schematypes.Object with the properties for
	// for the TaskPluginOptions.Payload property.
	//
	// Note: this will be merged with payload schemas from engine and other
	// plugins, thus, it cannot contain conflicting properties. Furthermore the
	// metadata will be discarded and additionalProperties will not be allowed.
	PayloadSchema() schematypes.Object

	// NewTaskPlugin method will be called once for each task. The TaskPlugin
	// instance returned will be called for each stage in the task execution.
	//
	// This is a poor place to do any processing, and not a great place to start
	// long-running operations as you don't have a place to write log messages.
	// Consider waiting until Prepare() is called with TaskContext that you can
	// write log messages to.
	//
	// Plugins implementing logging should not return an error here, as it
	// naturally follows that such an error can't be logged if no logging plugin
	// is created.
	//
	// Implementors may return nil, if the plugin doesn't have any hooks for the
	// given tasks.
	//
	// Non-fatal errors: MalformedPayloadError
	NewTaskPlugin(options TaskPluginOptions) (TaskPlugin, error)
}

Plugin is a plugin to the worker, for each task NewTaskPlugin is created. The Plugin instance is responsible for creating these objects and managing data shared between TaskPlugin instances.

All methods on this interface must be thread-safe.

func NewPluginManager

func NewPluginManager(options PluginOptions) (Plugin, error)

NewPluginManager loads all plugins not disabled in configuration and returns a Plugin implementation that wraps all of the plugins.

This expects options.Config satisfying schema from PluginManagerConfigSchema().

type PluginBase

type PluginBase struct{}

PluginBase is a base implementation of the Plugin interface, it just handles all methods and does nothing.

Implementors should embed this to ensure forward compatibility when we add new optional methods.

func (PluginBase) NewTaskPlugin

func (PluginBase) NewTaskPlugin(TaskPluginOptions) (TaskPlugin, error)

NewTaskPlugin returns nil ignoring the request to create a TaskPlugin for the given task.

func (PluginBase) PayloadSchema

func (PluginBase) PayloadSchema() schematypes.Object

PayloadSchema returns a schema for an empty object for plugins that doesn't take any payload.

type PluginOptions

type PluginOptions struct {
	Environment *runtime.Environment
	Engine      engines.Engine
	Log         *logrus.Entry
	Config      interface{}
}

PluginOptions is a wrapper for the arguments/options given when instantiating a Plugin using PluginProvider.

We wrap all arguments so that we can add additional properties without breaking source compatibility with older plugins.

type PluginProvider

type PluginProvider interface {
	NewPlugin(options PluginOptions) (Plugin, error)

	// ConfigSchema returns schema for the PluginOptions.Config property.
	// May return nil, if no configuration should be given.
	ConfigSchema() schematypes.Schema
}

The PluginProvider interface must be implemented and registered by anyone implementing a Plugin.

If an implementor can determine that a plugin isn't available at compile-time it is preferred not to register the plugin.

type PluginProviderBase

type PluginProviderBase struct{}

PluginProviderBase is a base struct that provides empty implementations of some methods for PluginProvider

Implementors of PluginProvider should embed this struct to ensure forward compatibility when we add new optional method to PluginProvider.

func (PluginProviderBase) ConfigSchema

func (PluginProviderBase) ConfigSchema() schematypes.Schema

ConfigSchema returns a nil schema.

type TaskPlugin

type TaskPlugin interface {
	// Prepare will be called in parallel with NewSandboxBuilder().
	//
	// Notice that this method is a good place to start long-running operations,
	// you then have to take care to clean-up in Dispose() if they are still
	// running. You should wait for your long-running operations to finish in
	// BuildSandbox() or whatever hook you need them in.
	//
	// Non-fatal errors: MalformedPayloadError
	Prepare(context *runtime.TaskContext) error

	// BuildSandbox is called once NewSandboxBuilder() has returned.
	//
	// This is the place to wait for downloads and other expensive operations to
	// finished, before mounting caches, proxies, etc. and returning.
	//
	// Non-fatal errors: MalformedPayloadError
	BuildSandbox(SandboxBuilder engines.SandboxBuilder) error

	// Started is called once the sandbox has started execution. This is a good
	// place to hook if you want to do interactive things.
	//
	// Non-fatal errors: MalformedPayloadError
	Started(sandbox engines.Sandbox) error

	// Stopped is called once the sandbox has terminated.
	//
	// This is a good place to upload artifacts, logs, check exit code, and start
	// to clean-up resources if such clean-up is expected to take a while.
	//
	// This will return false if the operation could not be completed successfully.
	// Such as artifact upload failure or files not existing.  If this returns false
	// it shall be assumed that the task has failed and should be reported as a failure.
	//
	// Non-fatal errors: MalformedPayloadError
	Stopped(result engines.ResultSet) (bool, error)

	// Finished is called once the sandbox has terminated and Stopped() have been
	// called.
	//
	// At this stage the task-specific log is closed, and attempts to log data
	// using the previous TaskContext will fail. That makes this a good place to
	// upload logs and any processing of the logs. In fact logging is the primary
	// motivation for this stage.
	//
	// As there is no logging in this method, it's not recommend to do anything
	// that may fail here.
	Finished(success bool) error

	// Exception is called once the task is resolved exception. This may happen
	// instead of calls to Prepare(), BuildSandbox(), Started(), Stopped(), or
	// Finished().
	//
	// This is a good place for best-effort to upload artifacts and logs that you
	// wish to persist. Naturally, log messages written at this stage will be
	// dropped and all error messages will be fatal.
	//
	// Implementors should be aware that additional reasons may be added in the
	// future. Therefore they must handle the default case, if switching on the
	// reason parameter.
	Exception(reason runtime.ExceptionReason) error

	// Dispose is called once everything is done and it's time for clean-up.
	//
	// This method will be invoked following Stopped() or Exception(). It is then
	// the responsibility of the implementor to abort or wait for any long-running
	// processes and clean-up any resources held.
	Dispose() error
}

TaskPlugin holds the task-specific state for a plugin

Each method on this interface represents stage in the task execution and will be called when this stage is reached. The methods are allowed to take significant amounts of time, as they will run asynchronously.

These methods does not have to be thread-safe, we will never call the next method, before the previous method has returned.

Implementors of this interface should be sure to embed TaskPluginBase. This will do absolutely nothing, but provide empty implementations for any current and future methods that isn't implemented.

The methods are called in the order listed here with the exception of Exception() which may be called following any method, and Dispose() which will always be called as a final step allowing you to clean up.

type TaskPluginBase

type TaskPluginBase struct{}

TaskPluginBase is a base implementation of the TaskPlugin interface, it just handles all methods and does nothing.

Implementors should embed this to ensure forward compatibility when we add new optional methods.

func (TaskPluginBase) BuildSandbox

func (TaskPluginBase) BuildSandbox(engines.SandboxBuilder) error

BuildSandbox ignores the sandbox building stage.

func (TaskPluginBase) Dispose

func (TaskPluginBase) Dispose() error

Dispose ignores the stage where resources are disposed.

func (TaskPluginBase) Exception

func (TaskPluginBase) Exception(reason runtime.ExceptionReason) error

Exception ignores the stage where a task is resolved exception

func (TaskPluginBase) Finished

func (TaskPluginBase) Finished(success bool) error

Finished ignores the stage where a task has been finished

func (TaskPluginBase) Prepare

func (TaskPluginBase) Prepare(*runtime.TaskContext) error

Prepare ignores the sandbox preparation stage.

func (TaskPluginBase) Started

func (TaskPluginBase) Started(engines.Sandbox) error

Started ignores the stage where the sandbox has started

func (TaskPluginBase) Stopped

func (TaskPluginBase) Stopped(engines.ResultSet) (bool, error)

Stopped ignores the stage where the sandbox has returned a ResultSet, and returns true saying the task was successful, as not to poison the water.

type TaskPluginOptions

type TaskPluginOptions struct {
	TaskInfo *runtime.TaskInfo
	Payload  map[string]interface{}
	Log      *logrus.Entry
}

The TaskPluginOptions is a wrapper for the set of arguments given to NewTaskPlugin.

We wrap the arguments in a single argument to maintain source compatibility when introducing additional arguments.