kubernetestracker

README

Kubernetes Tracker

Implements the JobTracker interface for kubernetes batch jobs.

Introduction

The kubernetes tracker provides methods for managing sets of grouped batch jobs (JobSessions). JobSessions are implemented by using labels attached to batch job objects ("drmaa2jobsession") refering to the JobSession name.

Functionality

Notes

At this point in time Kubernetes batch jobs don't play very well with sidecars. So when using things like istio you might run in state issues (sidecar container is still running after batch job finished).

Job Control Mapping

DRMAA2 Job Control	Kubernetes
Suspend	Unsupported
Resume	Unsupported
Terminate	Delete() - leads to Undetermined state
Hold	Unsupported
Release	Unsupported

State Mapping

Based on JobStatus

DRMAA2 State.	Kubernetes Job State
Done	status.Succeeded >= 1
Failed	status.Failed >= 1
Suspended	-
Running	status.Active >= 1
Queued	-
Undetermined	other / Terminate()

Job Template Mapping

DRMAA2 JobTemplate	Kubernetes Batch Job
RemoteCommand	v1.Container.Command[0]
Args	v1.Container.Args
CandidateMachines[0]	v1.Container.Hostname
JobCategory	v1.Container.Image
WorkingDir	v1.Container.WorkingDir
JobName	Note: If set and a job with the same name exists in history submission will fail. metadata: Name
DeadlineTime	AbsoluteTime converted to relative time (v1.Container.ActiveDeadlineSeconds)

Job Template extensions:

Extension key	Extension value
namespace	v1.Namespace
labels	"key=value,key2=value2" v1.Labels

Required:

• RemoteCommand
• JobCategory as it specifies the image

Other implicit settings:

• Parallelism: 1
• Completions: 1
• BackoffLimit: 1

Job Info Mapping

DRMAA2 JobInfo.	Kubernetes
ExitStatus	0 or 1 (1 if between 1 and 255 / not supported in Status)
SubmissionTime	job.CreationTimestamp.Time
DispatchTime	job.Status.StartTime.Time
FinishTime	job.Status.CompletionTime.Time
State	see above
JobID	v1.Job.UID

Documentation

Index

• Constants
• func DRMAA2State(jc batchv1.JobInterface, jobid string) drmaa2interface.JobState
• func JobToJobInfo(jc batchv1.JobInterface, jobid string) (drmaa2interface.JobInfo, error)
• func NewClientSet() (*kubernetes.Clientset, error)
• type KubernetesTracker
  • func New(jobsession string, cs *kubernetes.Clientset) (*KubernetesTracker, error)
  • func (kt *KubernetesTracker) AddArrayJob(jt drmaa2interface.JobTemplate, begin int, end int, step int, maxParallel int) (string, error)
  • func (kt *KubernetesTracker) AddJob(jt drmaa2interface.JobTemplate) (string, error)
  • func (kt *KubernetesTracker) DeleteJob(jobid string) error
  • func (kt *KubernetesTracker) JobControl(jobid, state string) error
  • func (kt *KubernetesTracker) JobInfo(jobid string) (drmaa2interface.JobInfo, error)
  • func (kt *KubernetesTracker) JobState(jobid string) drmaa2interface.JobState
  • func (kt *KubernetesTracker) ListArrayJobs(id string) ([]string, error)
  • func (kt *KubernetesTracker) ListJobCategories() ([]string, error)
  • func (kt *KubernetesTracker) ListJobs() ([]string, error)
  • func (kt *KubernetesTracker) Wait(jobid string, timeout time.Duration, states ...drmaa2interface.JobState) error

Constants

const K8S_JT_EXTENSION_LABELS = "labels"

const K8S_JT_EXTENSION_NAMESPACE = "namespace"

Functions

func DRMAA2State

func DRMAA2State(jc batchv1.JobInterface, jobid string) drmaa2interface.JobState

func JobToJobInfo

func JobToJobInfo(jc batchv1.JobInterface, jobid string) (drmaa2interface.JobInfo, error)

func NewClientSet

func NewClientSet() (*kubernetes.Clientset, error)

NewClientSet create a new clientset by parsing the .kube/config file in the home directory.

Types

type KubernetesTracker

type KubernetesTracker struct {
	// contains filtered or unexported fields
}

func New

func New(jobsession string, cs *kubernetes.Clientset) (*KubernetesTracker, error)

New creates a new KubernetesTracker either by using a given kubernetes Clientset or by allocating a new one (if the parameter is zero).

func (*KubernetesTracker) AddArrayJob

func (kt *KubernetesTracker) AddArrayJob(jt drmaa2interface.JobTemplate, begin int, end int, step int, maxParallel int) (string, error)

func (*KubernetesTracker) AddJob

func (kt *KubernetesTracker) AddJob(jt drmaa2interface.JobTemplate) (string, error)

AddJob converts the given DRMAA2 job template into a batchv1.Job and creates the job within Kubernetes.

func (*KubernetesTracker) DeleteJob

func (kt *KubernetesTracker) DeleteJob(jobid string) error

DeleteJob removes a job from kubernetes.

func (*KubernetesTracker) JobControl

func (kt *KubernetesTracker) JobControl(jobid, state string) error

JobControl changes the state of the given job by execution the given action (suspend, resume, hold, release, terminate).

func (*KubernetesTracker) JobInfo

func (kt *KubernetesTracker) JobInfo(jobid string) (drmaa2interface.JobInfo, error)

func (*KubernetesTracker) JobState

func (kt *KubernetesTracker) JobState(jobid string) drmaa2interface.JobState

func (*KubernetesTracker) ListArrayJobs

func (kt *KubernetesTracker) ListArrayJobs(id string) ([]string, error)

func (*KubernetesTracker) ListJobCategories

func (kt *KubernetesTracker) ListJobCategories() ([]string, error)

func (*KubernetesTracker) ListJobs

func (kt *KubernetesTracker) ListJobs() ([]string, error)

func (*KubernetesTracker) Wait

func (kt *KubernetesTracker) Wait(jobid string, timeout time.Duration, states ...drmaa2interface.JobState) error

Wait returns when the job is in one of the given states or when a timeout occurs (errors then).