Documentation
¶
Overview ¶
Package engine specfies the interfaces that each engine must implement.
This is all rooted at the EngineProvider interface specified in the extpoints package, where implementors should register their engine.
Consumers of this package should import the extpoints package and use the 'EngineProvider.Lookup(name) EngineProvider' function to load the desired engine.
Notice that many of the features specified are optional, and it's often possible to return an ErrFeatureNotSupported error rather than implementing the feature.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrContractViolation = errors.New("Engine has detected a contract violation")
ErrContractViolation is returned when a contract with the engine has been violated.
var ErrEngineIsSingleton = errors.New("Engine cannot run multiple sandboxes in parallel")
ErrEngineIsSingleton is returned when attempts to start multiple sandboxes of a singleton engine.
var ErrEngineNotSupported = errors.New("Engine is not available in the current configuration")
ErrEngineNotSupported is used to indicate that the engine isn't supported in the current configuration.
var ErrFeatureNotSupported = errors.New("Feature not support by current engine")
ErrFeatureNotSupported is a common error that may be returned from optional Engine methods to indicate the engine implementation doesn't support the given feature.
Note, all methods are allowed to return this error, some methods are required, and may not return this error.
When the worker encounters this error from an optional method, it should workaround if possible, but most likely resolve the task as "exception" with reason "malformed-payload".
var ErrImmutableMountNotSupported = errors.New("The engine doesn't support immutable volume attachements")
ErrImmutableMountNotSupported is returned when volume attachements are supported, but immutable mounts aren't supported.
var ErrMutableMountNotSupported = errors.New("The engine doesn't support mutable volume attachments")
ErrMutableMountNotSupported is returned when volume attachments are supported, but mutable mounts aren't supported.
var ErrNonFatalInternalError = errors.New("Engine encountered a non-fatal internal error")
ErrNonFatalInternalError is used to indicate that the operation failed because of internal error that isn't expected to affect other tasks.
var ErrResourceNotFound = errors.New("The referenced resource wasn't found")
ErrResourceNotFound is returned when trying to extract a file or folder that doesn't exist.
var ErrSandboxAborted = errors.New("Exection of sandbox was aborted")
ErrSandboxAborted is used to indicate that a Sandbox has been aborted.
var ErrSandboxTerminated = errors.New("The Sandbox has terminated")
ErrSandboxTerminated is used to indicate that a SandBox has already terminated and can't be aborted.
Functions ¶
This section is empty.
Types ¶
type Capabilities ¶
type Capabilities struct {
IsSingletonEngine bool
}
The Capabilities structure defines the set of features supported by an engine.
Some plugins will use this for feature detection, most plugins will call the methods in question and handle the ErrFeatureNotSupported error. For this reason it's essential to also return ErrFeatureNotSupported from methods related to unsupported features (see docs of individual methods).
Plugin implementors are advised to call methods and handling unsupported features by handling ErrFeatureNotSupported errors. But in some cases it might be necessary to adjust behavior in case of unsupported methods, for this up-front feature checking using Capabilities is necessary.
To encourage the try and handle errors pattern, the Capabilities shall only list features for which we critically need up-front feature testing.
type Engine ¶
type Engine interface {
// PayloadSchema returns the CompositeSchema that represents the payload.
//
// The Payload property on SandboxOptions given to NewSandboxBuilder will be
// the result from CompositeSchema.Parse() on the CompositeSchema returned
// from this method.
PayloadSchema() runtime.CompositeSchema
// Capabilities returns a structure declaring which features are supported,
// this is used for up-front feature checking. Unsupport methods must also
// return ErrFeatureNotSupported when called.
//
// This property is strictly for plugins that needs to do up-front feature
// checking. Consumers are encouraged to just try then and handle errors
// rather than testing for supported features up-front. Granted this is not
// always possible, hence, the presence of this property.
//
// Implementors must return a constant that is always the same.
Capabilities() Capabilities
// NewSandboxBuilder returns a new instance of the SandboxBuilder interface.
//
// We'll create a SandboxBuilder for each task run. This is really a setup
// step where the implementor may acquire resources referenced in the
// SandboxOptions.
//
// Example: An engine implementation based on docker, may start downloading
// the docker image in before returning from NewSandboxBuilder(). The
// SandboxBuilder instance returned will then reference the docker image
// downloading process, and be ready to start a new docker container once
// StartSandbox() is called, Obviously blocking that call until docker image
// download is completed.
//
// This operation should parse the engine-specific payload parts given in
// SandboxOptions and return a MalformedPayloadError error if the payload
// is invalid.
//
// Non-fatal errors: MalformedPayloadError, ErrEngineIsSingleton.
NewSandboxBuilder(options SandboxOptions) (SandboxBuilder, error)
// NewCacheFolder returns a new Volume backed by a file system folder
// if cache-folders folders are supported, otherwise it must return
// ErrFeatureNotSupported.
//
// Non-fatal errors: ErrFeatureNotSupported
NewCacheFolder() (Volume, error)
// NewMemoryDisk returns a new Volume backed by a ramdisk, if ramdisks are
// supported, otherwise it must return ErrFeatureNotSupported.
//
// Non-fatal errors: ErrFeatureNotSupported
NewMemoryDisk() (Volume, error)
}
An Engine implementation provides and backend upon which tasks can be executed.
We do not intend for a worker to use multiple engines at the same time, whilst it might be fun to try some day, implementors need not design with this use-case in mind. This means that you can safely assume that your engine is the only engine that is instantiated.
While we do not intend to use multiple engines at the same time, implementors must design engines to support running multiple sandboxes in parallel. If the engine can't run multiple sandboxes in parallel, it should return set IsSingletonEngine to false in its FeatureSet(). Additionally, it must return ErrEngineIsSingleton if a second sandbox is created, before the previous sandbox is disposed.
Obviously not all engines are available on all platforms and not all features can be implemented on all platforms. See individual methods to see which are required and which can be implemented by returning ErrFeatureNotSupported.
type EngineBase ¶
type EngineBase struct{}
EngineBase is a base implemenation of Engine. It will implement all optional methods such that they return ErrFeatureNotSupported.
Note: This will not implement NewSandboxBuilder() and other required methods.
Implementors of Engine should embed this struct to ensure source compatibility when we add more optional methods to Engine.
func (EngineBase) Capabilities ¶
func (EngineBase) Capabilities() Capabilities
Capabilities returns an zero value Capabilities struct indicating that most features aren't supported.
func (EngineBase) NewCacheFolder ¶
func (EngineBase) NewCacheFolder() (Volume, error)
NewCacheFolder returns ErrFeatureNotSupported indicating that the feature isn't supported.
func (EngineBase) NewMemoryDisk ¶
func (EngineBase) NewMemoryDisk() (Volume, error)
NewMemoryDisk returns ErrFeatureNotSupported indicating that the feature isn't supported.
func (EngineBase) PayloadSchema ¶
func (EngineBase) PayloadSchema() runtime.CompositeSchema
PayloadSchema returns an empty CompositeSchema indicating that a nil payload is sufficient.
type FileHandler ¶
type FileHandler func(path string, stream io.ReadCloser) error
FileHandler is given as callback when iterating through a list of files.
ResultSet.ExtractFolder(path, handler) takes a FileHandler as the handler parameter. This function maybe called sequentially or concurrently, but if it returns an the ResultSet should stop calling it and pass the error through as return value from ResultSet.ExtractFolder.
type MalformedPayloadError ¶
type MalformedPayloadError struct {
// contains filtered or unexported fields
}
The MalformedPayloadError error type is used to indicate that some operation failed because of malformed-payload. For example a string expected to be path contained invalid characters, a required property was missing, or an integer was outside the permitted range.
func NewMalformedPayloadError ¶
func NewMalformedPayloadError(message string) MalformedPayloadError
NewMalformedPayloadError creates a MalformedPayloadError object, please make sure to include a detailed description of the error, preferably using multiple lines and with examples.
These will be printed in the logs and end-users will rely on them to debug their tasks.
func (MalformedPayloadError) Error ¶
func (e MalformedPayloadError) Error() string
type ResultSet ¶
type ResultSet interface {
// Success, returns true if the execution was successful, typically implying
// that the process exited zero.
Success() bool
// Extract a file from the sandbox.
//
// Interpretation of the string path format is engine specific and must be
// documented by the engine implementor. The engine may impose restrictions on
// the string, if these restrictions are violated the engine should return a
// MalformedPayloadError.
//
// If the file requested doesn't exist the engine should return
// ErrResourceNotFound. Further more the engine may return
// ErrFeatureNotSupported rather than implementing this method.
//
// Non-fatal erorrs: ErrFeatureNotSupported, ErrResourceNotFound,
// MalformedPayloadError
ExtractFile(path string) (io.ReadCloser, error)
// Extract a folder from the sandbox.
//
// Interpretation of the string path format is engine specific and must be
// documented by the engine implementor. The engine may impose restrictions on
// the string, if these restrictions are violated the engine should return a
// MalformedPayloadError.
//
// If the folder requested doesn't exist the engine should return
// ErrResourceNotFound. Further more the engine may return
// ErrFeatureNotSupported rather than implementing this method.
//
// For each file found under the given path the handler(path, stream) is
// called. Implementor may call this function sequentially or in parallel.
// If a handler(path, stream) call returns an error the error should passed
// through as return value from the ExtractFolder call.
//
// If an error occurs during iteration iteration is halted, and when all
// calls to handler(path, stream) has returned ExtractFolder may with return
// with the error.
//
// The only non-fatal error is ErrNonFatalInternalError, indicating that
// something went wrong while streaming out artfacts and all artifacts may not
// have been extracted, or they may not have been streamed out completely.
//
// The ErrNonFatalInternalError may only be returned if the engine expected
// further request to be successful. And attempts to call other methods or
// extract other paths might work out fine.
//
// Non-fatal erorrs: ErrFeatureNotSupported, ErrResourceNotFound,
// MalformedPayloadError, ErrNonFatalInternalError
ExtractFolder(path string, handler FileHandler) error
// ArchiveSandbox streams out the entire sandbox (or as much as possible)
// as a tar-stream. Ideally this also includes cache folders.
ArchiveSandbox() (io.ReadCloser, error)
// Dispose shall release all resources.
//
// CacheFolders given to the sandbox shall not be disposed, instead they are
// just no longer owned by the engine.
//
// Implementors should only return an error if cleaning up fails and the
// worker therefor needs to stop operation.
Dispose() error
}
The ResultSet interface represents the results of a sandbox that has finished execution, but is hanging around while results are being extracted.
When returned from Sandbox this takes ownership of all resources. If the engine uses docker then the ResultSet would have ownership of cache folders as well as the terminated docker container.
All methods on this interface must be thread-safe.
type ResultSetBase ¶
type ResultSetBase struct{}
ResultSetBase is a base implemenation of ResultSet. It will implement all optional methods such that they return ErrFeatureNotSupported.
Note: This will not implement Success() and other required methods.
Implementors of ResultSet should embed this struct to ensure source compatibility when we add more optional methods to ResultSet.
func (ResultSetBase) ArchiveSandbox ¶
func (ResultSetBase) ArchiveSandbox() (io.ReadCloser, error)
ArchiveSandbox returns ErrFeatureNotSupported indicating that the feature isn't supported.
func (ResultSetBase) Dispose ¶
func (ResultSetBase) Dispose() error
Dispose returns nil indicating that resources have been released.
func (ResultSetBase) ExtractFile ¶
func (ResultSetBase) ExtractFile(string) (io.ReadCloser, error)
ExtractFile returns ErrFeatureNotSupported indicating that the feature isn't supported.
func (ResultSetBase) ExtractFolder ¶
func (ResultSetBase) ExtractFolder(string, FileHandler) error
ExtractFolder returns ErrFeatureNotSupported indicating that the feature isn't supported.
type Sandbox ¶
type Sandbox interface {
// Wait for task execution and termination of all associated shells, and
// return immediately if sandbox execution has finished.
//
// When this method returns all resources held by the Sandbox instance must
// have been released or transferred to the ResultSet instance returned. If an
// internal error occured resources may be freed and WaitForResult() may
// return ErrNonFatalInternalError if the error didn't leak resources and we
// don't expect the error to be persistent.
//
// When this method has returned any calls to Abort() or NewShell() should
// return ErrSandboxTerminated. If Abort() is called before WaitForResult()
// returns, WaitForResult() should return ErrSandboxAborted and release all
// resources held.
//
// Notice that this method may be invoked more than once. In all cases it
// should return the same value when it decides to return. In particular, it
// must keep a reference to the ResultSet instance created and return the same
// instance, so that any resources held aren't transferred to multiple
// different ResultSet instances.
//
// Non-fatal errors: ErrNonFatalInternalError, ErrSandboxAborted.
WaitForResult() (ResultSet, error)
// NewShell creates a new Shell for interaction with the sandbox. This is
// useful for debugging and other purposes.
//
// If the engine doesn't support interactive shells it may return
// ErrFeatureNotSupported. This should not interrupt/abort the execution of
// the task which should proceed as normal.
//
// If the WaitForResult() method has returned and the sandbox isn't running
// anymore this method must return ErrSandboxTerminated, signaling that you
// can't interact with the sandbox anymore.
//
// Non-fatal errors: ErrFeatureNotSupported, ErrSandboxTerminated.
NewShell() (Shell, error)
// Abort the sandbox, this means killing the task execution as well as all
// associated shells and releasing all resources held.
//
// If called before the sandbox execution finished, then WaitForResult() must
// return ErrSandboxAborted. If sandbox execution has finished when Abort() is
// called Abort() should return ErrSandboxTerminated and not release any
// resources as they should have been released by WaitForResult() or
// transferred to the ResultSet instance returned.
//
// Non-fatal errors: ErrSandboxTerminated
Abort() error
}
The Sandbox interface represents an active sandbox.
All methods on this interface must be thread-safe.
type SandboxBase ¶
type SandboxBase struct{}
SandboxBase is a base implemenation of Sandbox. It will implement all optional methods such that they return ErrFeatureNotSupported.
Note: This will not implement WaitForResult() and other required methods.
Implementors of SandBox should embed this struct to ensure source compatibility when we add more optional methods to SandBox.
func (SandboxBase) Abort ¶
func (SandboxBase) Abort() error
Abort returns nil indicating that resources have been released.
func (SandboxBase) NewShell ¶
func (SandboxBase) NewShell() (Shell, error)
NewShell returns ErrFeatureNotSupported indicating that the feature isn't supported.
type SandboxBuilder ¶
type SandboxBuilder interface {
// Attach a volume at given mountpoint.
//
// The volume given must have been created by this engine, using a method like
// engine.NewVolume(). Implementors are free to make such a type assertion.
//
// The mountpoint is a string in engine-specific format. If the given
// mountpoint violates the engine-specific format, a MalformedPayloadError
// should be returned. For example a docker engine may expect the mountpoint
// to be a path, where as a different engine might expect it to be a folder
// name, or the name of an environement variable pointing to the folder.
//
// If the mountpoint is invalid because it's already in use a
// MalformedPayloadError is also appropriate.
//
// If the engine doesn't support mutable or immutable volume attachments, it
// should return ErrMutableMountNotSupported or ErrImmutableMountNotSupported,
// respectively.
//
// Non-fatal errors: MalformedPayloadError, ErrMutableMountNotSupported,
// ErrImmutableMountNotSupported, ErrFeatureNotSupported.
AttachVolume(mountpoint string, volume Volume, readOnly bool) error
// Attach a proxy to the sandbox.
//
// The name is a engine-specific format. If the given name violates
// engine-specific format, a MalformedPayloadError should be returned.
// For example a docker engine may expect the name to be a hostname, where as
// a different engine could have ot being the path of a unix-domain socket,
// a port on localhost, or the prefix of a URL path.
//
// It is the engines responsbility to ensure that requests aimed at the given
// name is forwarded to the handler. And to ensure that no other processes are
// able to forward requests to the handler.
//
// If the engine doesn't support proxy attachments, it should return
// ErrFeatureNotSupported.
//
// Non-fatal errors: MalformedPayloadError, ErrFeatureNotSupported,
AttachProxy(name string, handler http.Handler) error
// Start execution of task in sandbox. After a call to this method resources
// held by the SandboxBuilder instance should be released or transferred to
// the Sandbox implementation.
//
// Non-fatal errors: MalformedPayloadError
StartSandbox() (Sandbox, error)
// Discard must free all resources held by the SandboxBuilder interface.
// Any error returned is fatal, so do not return an error unless there is
// something very wrong.
Discard() error
}
The SandboxBuilder interface wraps the state required to start a Sandbox.
Before returning a SandboxBuilder engine implementors should start downloading and setting up all the resources needed to start execution. A docker based engine may wish to ensure the docker image is downloaded, or lay a claim on it so the GarbageCollector won't remove it. A naive Windows engine may wish to create a new user account and setup a folder for the sandbox.
Implementors can be sure that any instance of this interface will only be used to create a single Sandbox, that is StartSandbox() will atmost be called once. If StartSandbox() is called twice a sane implementor should return ErrContractViolation, and feel free to exhibit undefined behavior.
All methods of this interface must be thread-safe.
type SandboxBuilderBase ¶
type SandboxBuilderBase struct{}
SandboxBuilderBase is a base implemenation of SandboxBuilder. It will implement all optional methods such that they return ErrFeatureNotSupported.
Note: This will not implement StartSandbox() and other required methods.
Implementors of SandBoxBuilder should embed this struct to ensure source compatibility when we add more optional methods to SandBoxBuilder.
func (SandboxBuilderBase) AttachProxy ¶
func (SandboxBuilderBase) AttachProxy(string, http.Handler) error
AttachProxy returns ErrFeatureNotSupported indicating that the feature isn't supported.
func (SandboxBuilderBase) AttachVolume ¶
func (SandboxBuilderBase) AttachVolume(string, Volume, bool) error
AttachVolume returns ErrFeatureNotSupported indicating that the feature isn't supported.
func (SandboxBuilderBase) Discard ¶
func (SandboxBuilderBase) Discard() error
Discard returns nil, indicating that resources have been released.
type SandboxOptions ¶
type SandboxOptions struct {
// Task contains information about the task we're starting a sandbox for.
TaskContext *runtime.TaskContext
// Result from PayloadSchema().Parse() implementors should feel safe in
// type asserting this back to their target type.
Payload interface{}
}
The SandboxOptions structure is a wrapper around the options/arguments for creating a NewSandboxBuilder. This allows us to added new arguments without source compatibility with older Engine implementations.
type Shell ¶
type Shell interface {
StdinPipe() (io.WriteCloser, error)
StdoutPipe() (io.ReadCloser, error)
StderrPipe() (io.ReadCloser, error)
Terminate() error
Wait() (bool, error)
}
The Shell interface is not fully specified yet, but the idea is that it pipes data to a shell inside a Sandbox.
type Volume ¶
type Volume interface {
// Dispose deletes all resources used by the Volume.
Dispose() error
}
Volume that we can modify and mount on a Sandbox.
Note, that engine implementations are not responsible for tracking the Volume, deletion and/or if it's mounted on more than one Sandbox at the same time.
The engine is responsible for creating it, mounting it in sandboxes, loading data through the defined interface, extracting data through the defined interface and deleting the underlying storage when Dispose is called.
type VolumeBase ¶
type VolumeBase struct{}
VolumeBase is a base implemenation of Volume. It will implement all optional methods such that they return ErrFeatureNotSupported.
Implementors of Volume should embed this struct to ensure source compatibility when we add more optional methods to Volume.
func (VolumeBase) Dispose ¶
func (VolumeBase) Dispose() error
Dispose returns nil indicating that resources were released.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package enginetest provides utilities for testing generic engine implementations.
|
Package enginetest provides utilities for testing generic engine implementations. |
|
generated by go-extpoints -- DO NOT EDIT Package extpoints provides methods that engine plugins can register their implements with as an import side-effect.
|
generated by go-extpoints -- DO NOT EDIT Package extpoints provides methods that engine plugins can register their implements with as an import side-effect. |
|
Package mockengine implements a MockEngine that doesn't really do anything, but allows us to test plugins without having to run a real engine.
|
Package mockengine implements a MockEngine that doesn't really do anything, but allows us to test plugins without having to run a real engine. |