runtime

Documentation

Overview

Package runtime contains the generic functionality that an engine and plugins use.

Index

• Variables
• func CreateErrorArtifact(artifact ErrorArtifact, context *TaskContext) error
• func CreateLogger(level string) (*logrus.Logger, error)
• func CreateRedirectArtifact(artifact RedirectArtifact, context *TaskContext) error
• func Gunzip(filename string) (string, error)
• func NewTaskContext(tempLogFile string, task TaskInfo, server webhookserver.WebHookServer) (*TaskContext, *TaskContextController, error)
• func Untar(filename string) error
• func Unzip(filename string) error
• func UploadS3Artifact(artifact S3Artifact, context *TaskContext) error
• type Environment
• type ErrorArtifact
• type ExceptionReason
• type RedirectArtifact
• type S3Artifact
• type ShutdownManager
  • func NewShutdownManager(host string) ShutdownManager
• type TaskContext
  • func (c *TaskContext) Abort()
  • func (c *TaskContext) AttachWebHook(handler http.Handler) string
  • func (c *TaskContext) Cancel()
  • func (c *TaskContext) Deadline() (deadline time.Time, ok bool)
  • func (c *TaskContext) Done() <-chan struct{}
  • func (c *TaskContext) Err() error
  • func (c *TaskContext) ExtractLog() (ioext.ReadSeekCloser, error)
  • func (c *TaskContext) IsAborted() bool
  • func (c *TaskContext) IsCancelled() bool
  • func (c *TaskContext) Log(a ...interface{})
  • func (c *TaskContext) LogDrain() io.Writer
  • func (c *TaskContext) LogError(a ...interface{})
  • func (c *TaskContext) NewLogReader() (io.ReadCloser, error)
  • func (c *TaskContext) Queue() client.Queue
  • func (c *TaskContext) Value(key interface{}) interface{}
• type TaskContextController
  • func (c *TaskContextController) CloseLog() error
  • func (c *TaskContextController) Dispose() error
  • func (c *TaskContextController) SetQueueClient(client client.Queue)
• type TaskInfo
• type TaskStatus
• type TemporaryFile
• type TemporaryFolder
• type TemporaryStorage
  • func NewTemporaryStorage(path string) (TemporaryStorage, error)

Variables

var ErrLogClosed = errors.New("Log already closed")

ErrLogClosed raises when there is a double call to CloseLog.

var ErrLogNotClosed = errors.New("Log is still open")

ErrLogNotClosed represents an invalid attempt to extract a log while it is still open.

Functions

func CreateErrorArtifact

func CreateErrorArtifact(artifact ErrorArtifact, context *TaskContext) error

CreateErrorArtifact is responsible for inserting error artifacts into the queue.

func CreateLogger

func CreateLogger(level string) (*logrus.Logger, error)

CreateLogger returns a new logger that can be passed around through the environment. Loggers can be created based on the one returned from this method by calling WithField or WithFields and specifying additional fields that the package would like.

func CreateRedirectArtifact

func CreateRedirectArtifact(artifact RedirectArtifact, context *TaskContext) error

CreateRedirectArtifact is responsible for inserting redirect artifacts into the queue.

func Gunzip

func Gunzip(filename string) (string, error)

Gunzip gunzips a zipped file and returns its name

func NewTaskContext

func NewTaskContext(tempLogFile string,
	task TaskInfo,
	server webhookserver.WebHookServer) (*TaskContext, *TaskContextController, error)

NewTaskContext creates a TaskContext and associated TaskContextController

func Untar

func Untar(filename string) error

Untar unpacks a tar file

func Unzip

func Unzip(filename string) error

Unzip unzips a zipped file

func UploadS3Artifact

func UploadS3Artifact(artifact S3Artifact, context *TaskContext) error

UploadS3Artifact is responsible for creating new artifacts in the queue and then performing the upload to s3.

Types

type Environment

type Environment struct {
	GarbageCollector gc.ResourceTracker
	//TODO: Add some sort of interface to the system logger
	//TODO: Add some interface to submit statistics for influxdb/signalfx
	//TODO: Add some interface to attach a http.Handler to public facing server
	TemporaryStorage TemporaryStorage
	Log              *logrus.Logger
	Sentry           *raven.Client
	WebHookServer    webhookserver.WebHookServer
}

Environment is a collection of objects that makes up a runtime environment.

type ErrorArtifact

type ErrorArtifact struct {
	Name    string
	Message string
	Reason  string
	Expires tcclient.Time
}

ErrorArtifact wraps all of the needed fields to upload an error artifact

type ExceptionReason

type ExceptionReason string

An ExceptionReason specifies the reason a task reached an exception state.

const (
	MalformedPayload ExceptionReason = "malformed-payload"
	WorkerShutdown   ExceptionReason = "worker-shutdown"
	InternalError    ExceptionReason = "internal-error"
)

Reasons why a task can reach an exception state. Implementors should be warned that additional entries may be added in the future.

type RedirectArtifact

type RedirectArtifact struct {
	Name     string
	Mimetype string
	URL      string
	Expires  tcclient.Time
}

RedirectArtifact wraps all of the needed fields to upload a redirect artifact

type S3Artifact

type S3Artifact struct {
	Name              string
	Mimetype          string
	Expires           tcclient.Time
	Stream            ioext.ReadSeekCloser
	AdditionalHeaders map[string]string
}

S3Artifact wraps all of the needed fields to upload an s3 artifact

type ShutdownManager

type ShutdownManager interface {
	WaitForShutdown() <-chan struct{}
}

ShutdownManager implements a method for listening for shutdown events. Consumers

func NewShutdownManager

func NewShutdownManager(host string) ShutdownManager

NewShutdownManager will return a shutdown manager appropriate for the host that the worker is being run on.

Shutdown events are triggered different ways depending on where the worker is running. When running in AWS, then notifications are sent on their meta-data api, but running locally could cause the worker to represent to different kind of shutdown events.

type TaskContext

type TaskContext struct {
	TaskInfo
	// contains filtered or unexported fields
}

The TaskContext exposes generic properties and functionality related to a task that is currently being executed.

This context is used to ensure that every component both engines and plugins that operates on a task have access to some common information about the task. This includes log drains, per-task credentials, generic task properties, and abortion notifications.

func (*TaskContext) Abort

func (c *TaskContext) Abort()

Abort sets the status to aborted

func (*TaskContext) AttachWebHook

func (c *TaskContext) AttachWebHook(handler http.Handler) string

AttachWebHook will take an http.Handler and expose it to the internet such that requests to any sub-resource of url returned will be forwarded to the handler.

Additionally, we promise that the URL contains a cryptographically random sequence of characters rendering it unpredictable. This can be used as a cheap form of access-control, and it is safe as task-specific web hooks are short-lived by nature.

Example use-cases:

• livelog plugin
• plugins for interactive shell/display/debugger, etc.
• engines that send an email and await user confirmation ...

Implementors attaching a hook should take care to ensure that the handler is able to respond with a non-2xx response, if the data it is accessing isn't available anymore. All webhooks will be detached at the end of the task-cycle, but not until the very end.

func (*TaskContext) Cancel

func (c *TaskContext) Cancel()

Cancel sets the status to cancelled

func (*TaskContext) Deadline

func (c *TaskContext) Deadline() (deadline time.Time, ok bool)

Deadline returns empty time and false, this is implemented to satisfy context.Context.

func (*TaskContext) Done

func (c *TaskContext) Done() <-chan struct{}

Done returns a channel that is closed when to TaskContext is aborted or canceled.

Implemented in compliance with context.Context.

func (*TaskContext) Err

func (c *TaskContext) Err() error

Err returns context.Canceled, if task as canceled or aborted.

Implemented in compliance with context.Context.

func (*TaskContext) ExtractLog

func (c *TaskContext) ExtractLog() (ioext.ReadSeekCloser, error)

ExtractLog returns an IO object to read the log.

func (*TaskContext) IsAborted

func (c *TaskContext) IsAborted() bool

IsAborted returns true if the current status is Aborted

func (*TaskContext) IsCancelled

func (c *TaskContext) IsCancelled() bool

IsCancelled returns true if the current status is Cancelled

func (*TaskContext) Log

func (c *TaskContext) Log(a ...interface{})

Log writes a log message from the worker

These log messages will be prefixed "taskcluster" so it's easy to see to that they are worker logs.

func (*TaskContext) LogDrain

func (c *TaskContext) LogDrain() io.Writer

LogDrain returns a drain to which log message can be written.

Users should note that multiple writers are writing to this drain concurrently, and it is recommend that writers write in chunks of one line.

func (*TaskContext) LogError

func (c *TaskContext) LogError(a ...interface{})

LogError writes a log error message from the worker

These log messages will be prefixed "[taskcluster:error]" so it's easy to see to that they are worker logs. These errors are also easy to grep from the logs in case of failure.

func (*TaskContext) NewLogReader

func (c *TaskContext) NewLogReader() (io.ReadCloser, error)

NewLogReader returns a ReadCloser that reads the log from the start as the log is written.

Calls to Read() on the resulting ReadCloser are blocking. They will return when data is written or EOF is reached.

Consumers should ensure the ReadCloser is closed before discarding it.

func (*TaskContext) Queue

func (c *TaskContext) Queue() client.Queue

Queue will return a client for the TaskCluster Queue. This client is useful for plugins that require interactions with the queue, such as creating artifacts.

func (*TaskContext) Value

func (c *TaskContext) Value(key interface{}) interface{}

Value returns nil, this is implemented to satisfy context.Context

type TaskContextController

type TaskContextController struct {
	*TaskContext
}

TaskContextController exposes logic for controlling the TaskContext.

Spliting this out from TaskContext ensures that engines and plugins doesn't accidentally Dispose() the TaskContext.

func (*TaskContextController) CloseLog

func (c *TaskContextController) CloseLog() error

CloseLog will close the log so no more messages can be written.

func (*TaskContextController) Dispose

func (c *TaskContextController) Dispose() error

Dispose will clean-up all resources held by the TaskContext

func (*TaskContextController) SetQueueClient

func (c *TaskContextController) SetQueueClient(client client.Queue)

SetQueueClient will set a client for the TaskCluster Queue. This client can then be used by others that have access to the task context and require interaction with the queue.

type TaskInfo

type TaskInfo struct {
	TaskID   string
	RunID    int
	Created  tcclient.Time
	Deadline tcclient.Time
	Expires  tcclient.Time
}

The TaskInfo struct exposes generic properties from a task definition.

Note, do not be tempted to add task definition or status here in its entirety as it can encourage undesired behaviors. Instead only the data necessary should be exposed and nothing more. One such anti-pattern could be for a plugin to look at task.extra instead of adding data to task.payload.

type TaskStatus

type TaskStatus string // TODO: (jonasfj) TaskContext shouldn't track status

TaskStatus represents the current status of the task.

const (
	Aborted   TaskStatus = "Aborted" // TODO: (jonasfj) Don't distinguish between cancel/abort
	Cancelled TaskStatus = "Cancelled"
	Succeeded TaskStatus = "Succeeded"
	Failed    TaskStatus = "Failed"
	Errored   TaskStatus = "Errored"
	Claimed   TaskStatus = "Claimed"
	Reclaimed TaskStatus = "Reclaimed"
)

Enumerate task status to aid life-cycle decision making Use strings for benefit of simple logging/reporting

type TemporaryFile

type TemporaryFile interface {
	io.ReadWriteSeeker
	io.Closer
	Path() string
}

TemporaryFile is a temporary file that will be removed when closed.

type TemporaryFolder

type TemporaryFolder interface {
	TemporaryStorage
	Path() string
	Remove() error
}

TemporaryFolder is a temporary folder that is backed by the filesystem. User are nicely asked to stay with the folder they've been issued.

We don't really mock the file system interface as we need to integrate with other applications like docker, so we have to expose real file paths.

type TemporaryStorage

type TemporaryStorage interface {
	NewFolder() (TemporaryFolder, error)
	NewFile() (TemporaryFile, error)
	NewFilePath() string
}

TemporaryStorage can create temporary folders and files.

func NewTemporaryStorage

func NewTemporaryStorage(path string) (TemporaryStorage, error)

NewTemporaryStorage TemporaryStorage rooted in the given path.