dockertracker

README

Docker Tracker

Introduction

Docker Tracker implements the JobTracker interface used by the Go DRMAA2 implementation in order to use Docker as a backend for managing jobs as containers from the DRMAA2 interface.

Functionality

Basic Usage

A JobTemplate requires:

• JobCategory -> which maps to an installed Docker image
• RemoteCommand -> which is the command executed within the given Docker image

Job Control Mapping

DRMAA2 Job Control	Docker
Suspend	Signal: SIGSTOP
Resume	Signal: SIGCONT
Terminate	Signal: SIGKILL
Hold	Unsupported
Release	Unsupported

State Mapping

DRMAA2 State	Docker State
Failed	OOMKilled
Failed or Done depending on exit code	Exited
Failed or Done depending on exit code	Dead
Suspended	Paused
Running	Running
Queued	Restarting
Undetermined	other

DeleteJob

DeleteJob equals docker rm and is removing an installed container. It must be terminated / finished before.

Job Template Mapping

Mapping between the job template and the Docker container config request:

DRMAA2 JobTemplate	Docker Container Config Request
RemoteCommand	Cmd[0]
Args	Cmd[1:]
CandidateMachines[0]	Hostname
JobCategory	Image
WorkingDir	WorkingDir
JobEnvironment (k: v)	Env ("k=v")
StageInFiles	-v localPath:containerPath
ErrorPath	Writes stderr into a local file (not a file in container)
OutputPath	Writes stdout into a local file.
Extension: "user"	User / must exist in container if set
Extension: "exposedPorts"	-p / multiple entries are splitted with ","

If more extensions needed just open an issue.

Note that the image must be available (pulled already)!

Job Info Mapping

Job Arrays

Since Array Jobs are not supported by Docker the job array functionality is implemented by creating n tasks sequentially in a loop. The array job ID contains all IDs of the created Docker containers.

Documentation

Index

• type DockerTracker
  • func New(jobsession string) (*DockerTracker, error)
  • func (dt *DockerTracker) AddArrayJob(jt drmaa2interface.JobTemplate, begin int, end int, step int, maxParallel int) (string, error)
  • func (dt *DockerTracker) AddJob(jt drmaa2interface.JobTemplate) (string, error)
  • func (dt *DockerTracker) DeleteJob(jobid string) error
  • func (dt *DockerTracker) JobControl(jobid, state string) error
  • func (dt *DockerTracker) JobInfo(jobid string) (ji drmaa2interface.JobInfo, err error)
  • func (dt *DockerTracker) JobState(jobid string) drmaa2interface.JobState
  • func (dt *DockerTracker) ListArrayJobs(id string) ([]string, error)
  • func (dt *DockerTracker) ListJobCategories() ([]string, error)
  • func (dt *DockerTracker) ListJobs() ([]string, error)
  • func (dt *DockerTracker) Wait(jobid string, timeout time.Duration, states ...drmaa2interface.JobState) error

Types

type DockerTracker

type DockerTracker struct {
	// contains filtered or unexported fields
}

func New

func New(jobsession string) (*DockerTracker, error)

New creates a new DockerTracker. How the Docker client is configured can be influenced by (from the Docker Documentation (https://github.com/moby/moby/blob/master/client/client.go)): "Use DOCKER_HOST to set the url to the docker server.

Use DOCKER_API_VERSION to set the version of the API to reach, leave empty for latest.
Use DOCKER_CERT_PATH to load the TLS certificates from.
Use DOCKER_TLS_VERIFY to enable or disable TLS verification, off by default."

func (*DockerTracker) AddArrayJob

func (dt *DockerTracker) AddArrayJob(jt drmaa2interface.JobTemplate, begin int, end int, step int, maxParallel int) (string, error)

func (*DockerTracker) AddJob

func (dt *DockerTracker) AddJob(jt drmaa2interface.JobTemplate) (string, error)

func (*DockerTracker) DeleteJob

func (dt *DockerTracker) DeleteJob(jobid string) error

DeleteJob removes a container so it is no longer in docker ps -a (and therefore not in the job list).

func (*DockerTracker) JobControl

func (dt *DockerTracker) JobControl(jobid, state string) error

func (*DockerTracker) JobInfo

func (dt *DockerTracker) JobInfo(jobid string) (ji drmaa2interface.JobInfo, err error)

func (*DockerTracker) JobState

func (dt *DockerTracker) JobState(jobid string) drmaa2interface.JobState

func (*DockerTracker) ListArrayJobs

func (dt *DockerTracker) ListArrayJobs(id string) ([]string, error)

func (*DockerTracker) ListJobCategories

func (dt *DockerTracker) ListJobCategories() ([]string, error)

ListJobCategories lists all containers available to run commands on.

func (*DockerTracker) ListJobs

func (dt *DockerTracker) ListJobs() ([]string, error)

func (*DockerTracker) Wait

func (dt *DockerTracker) Wait(jobid string, timeout time.Duration, states ...drmaa2interface.JobState) error