taskcluster-multistage-docker-worker

module

v0.0.0-...-30d97dc Latest Latest Go to latest Published: May 14, 2021 License: Apache-2.0

README ¶

Multi-Stage Docker Worker

This project is a spin on the Taskcluster Docker worker. The key difference is that the Multi-Stage worker supports having multiple steps which are executed in the same 'pod'. Each pod is run in a dedicated Docker instance (using docker-in-docker), where they can share volumes without them having to be transferred over the network.

Usage

The worker can run regular tasks from Taskcluster with its own payload format.

Task Example

provisionerId: shared
workerType: multistage-docker
created: {$fromNow: ''}
deadline: {$fromNow: '3 hours'}
expires: {$fromNow: '10 days'}
payload:
  steps:
    - image: my/ci/bootstrap
    - image: bitnami/git
      args: ["clone", "https://github.com/my/repo", "."]
    - image: docker
      args: ["build", ".", "-t", "my/image/name"]
    - image: docker
      args: ["push", "my/image/name"]
metadata:
  name: example-task
  description: An **example** task
  owner: name@example.com

Deployment

The worker is compatible with worker-runner using the generic-worker implementation. It has a few configuration variables but the most common ones match generic-worker:

worker-runner Config Example

provider:
  providerType: google  
worker:
  implementation: generic-worker
  path: ./local/multistage-docker-worker
  configPath: local/generic-worker-config.yml
workerConfig:
  wstAudience: taskcluster
  wstServerURL: https://my.taskcluster
  shutdownOnIdleSecs: 30

License

This project is licensed under the Apache 2.0 license.

Directories ¶

Path	Synopsis
cmd
multistage-docker-worker
internal
config This package contains the global worker configuration and task configuration including the task payload.	This package contains the global worker configuration and task configuration including the task payload.
cri Package cri contains the container runtime interface abstraction.	Package cri contains the container runtime interface abstraction.
exception This package provides helper methods for producing errors which will mark tasks as exception-ing rather than failing.	This package provides helper methods for producing errors which will mark tasks as exception-ing rather than failing.
livelog This package implements the livelog transport, optionally using websocktunnel or just listening on a port.	This package implements the livelog transport, optionally using websocktunnel or just listening on a port.
log This package contains helpers to make the logging awesome.	This package contains helpers to make the logging awesome.
pubsubbuffer This package implements a wrapper around bytes.Buffer which can be subscribed to and will only return EOF when the source buffer is closed.	This package implements a wrapper around bytes.Buffer which can be subscribed to and will only return EOF when the source buffer is closed.
worker This is the core implementation of the worker.	This is the core implementation of the worker.
workerproto This package contains an implementation of worker-runner's worker protocol.	This package contains an implementation of worker-runner's worker protocol.

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL