jobdb

package
v0.4.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 1, 2024 License: Apache-2.0 Imports: 21 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func SchedulingOrderCompare added in v0.4.0 

func SchedulingOrderCompare(job, other *Job) int

SchedulingOrderCompare defines the order in which jobs in a queue should be scheduled (both when scheduling new jobs and when re-scheduling evicted jobs). Specifically, compare returns

  • 0 if the jobs have equal job id,
  • -1 if job should be scheduled before other,
  • +1 if other should be scheduled before other.

Types

type Job 

type Job struct {
	// contains filtered or unexported fields
}

Job is the scheduler-internal representation of a job.

func (*Job) ActiveRunTimestamp added in v0.4.12 

func (job *Job) ActiveRunTimestamp() int64

ActiveRunTimestamp returns the creation time of the most recent run associated with this job.

func (*Job) AllRuns added in v0.3.63 

func (job *Job) AllRuns() []*JobRun

AllRuns returns all runs associated with job.

func (*Job) Assert added in v0.4.12 

func (job *Job) Assert() error

Assert makes the assertions outlined below and returns - nil if the job is valid and - an error explaining why not otherwise.

Assertions: Required fields must be set.

All associated runs are valid. The state of LatestRun is compatible with that of the job.

The states {Queued, Running, Cancelled, Failed, Succeeded} are mutually exclusive.

Only these job state transitions are valid: - Queued -> {Running, Cancelled, Failed} - Running -> {Queued, Cancelled, Failed, Succeeded} - Succeeded, Cancelled, Failed -> {}

QueuedVersion is 0 initially and is incremented by 1 every time the queued status of the job changes. Hence: - If the job is queued, the queuedVersion must be 2x the number of runs. E.g.:

  • queued (qv = 0)
  • queued -> running -> queued: qv = 2

- If the job is running or succeeded, the queuedVersion must be one less than 2x number of runs. E.g.:

  • queued -> running -> succeeded: qv = 1
  • queued -> running -> queued -> running -> succeeded: qv = 3

- If the job is failed or cancelled, the queuedVersion may be either equal to or 1 less than 2x the number of runs. E.g.:

  • queued -> failed/cancelled: qv = 0
  • queued -> running -> failed/cancelled: qv = 1
  • queued -> running -> queued -> failed/cancelled: qv = 2

func (*Job) CancelByJobsetRequested 

func (job *Job) CancelByJobsetRequested() bool

CancelByJobsetRequested returns true if the user has requested this job's jobSet be cancelled.

func (*Job) CancelRequested 

func (job *Job) CancelRequested() bool

CancelRequested returns true if the user has requested this job be cancelled.

func (*Job) Cancelled 

func (job *Job) Cancelled() bool

Cancelled Returns true if the scheduler has cancelled the job

func (*Job) Created 

func (job *Job) Created() int64

Created Returns the creation time of the job

func (*Job) DeepCopy added in v0.3.89 

func (job *Job) DeepCopy() *Job

func (*Job) Equal added in v0.4.0 

func (job *Job) Equal(other *Job) bool

Equal returns true if job is equal to other and false otherwise. Scheduling requirements are assumed to be equal if both jobs have equal schedulingKey.

func (*Job) Failed 

func (job *Job) Failed() bool

Failed Returns true if the scheduler has marked the job as failed

func (*Job) GetAffinity added in v0.3.71 

func (job *Job) GetAffinity() *v1.Affinity

Needed for compatibility with interfaces.LegacySchedulerJob

func (*Job) GetAnnotations 

func (job *Job) GetAnnotations() map[string]string

GetAnnotations returns the annotations on the job. This is needed for compatibility with interfaces.LegacySchedulerJob

func (*Job) GetId 

func (job *Job) GetId() string

GetId returns the id of the Job. This is needed for the LegacyJob interface.

func (*Job) GetJobSet 

func (job *Job) GetJobSet() string

GetJobSet returns the jobSet the job belongs to. This is needed for compatibility with legacyJob

func (*Job) GetNodeSelector added in v0.3.71 

func (job *Job) GetNodeSelector() map[string]string

Needed for compatibility with interfaces.LegacySchedulerJob

func (*Job) GetPerQueuePriority added in v0.3.77 

func (job *Job) GetPerQueuePriority() uint32

GetPerQueuePriority exists for compatibility with the LegacyJob interface.

func (*Job) GetPodRequirements added in v0.3.78 

func (job *Job) GetPodRequirements(_ map[string]types.PriorityClass) *schedulerobjects.PodRequirements

GetPodRequirements is needed for compatibility with interfaces.LegacySchedulerJob.

func (*Job) GetPriorityClassName added in v0.3.70 

func (job *Job) GetPriorityClassName() string

Needed for compatibility with interfaces.LegacySchedulerJob

func (*Job) GetQueue 

func (job *Job) GetQueue() string

GetQueue returns the queue this job belongs to. This is needed for the LegacyJob interface.

func (*Job) GetQueueTtlSeconds added in v0.3.95 

func (job *Job) GetQueueTtlSeconds() int64

Needed for compatibility with interfaces.LegacySchedulerJob

func (*Job) GetResourceRequirements added in v0.3.71 

func (job *Job) GetResourceRequirements() v1.ResourceRequirements

Needed for compatibility with interfaces.LegacySchedulerJob

func (*Job) GetScheduledAtPriority added in v0.4.8 

func (job *Job) GetScheduledAtPriority() (int32, bool)

func (*Job) GetSchedulingKey added in v0.3.100 

func (job *Job) GetSchedulingKey() (schedulerobjects.SchedulingKey, bool)

GetSchedulingKey returns the scheduling key associated with a job. The second return value is always true since scheduling keys are computed at job creation time. This is needed for compatibility with interfaces.LegacySchedulerJob.

func (*Job) GetSubmitTime added in v0.3.77 

func (job *Job) GetSubmitTime() time.Time

GetSubmitTime exists for compatibility with the LegacyJob interface.

func (*Job) GetTolerations added in v0.3.71 

func (job *Job) GetTolerations() []v1.Toleration

Needed for compatibility with interfaces.LegacySchedulerJob

func (*Job) HasQueueTtlExpired added in v0.3.95 

func (job *Job) HasQueueTtlExpired() bool

HasQueueTtlExpired returns true if the given job has reached its queueTtl expiry. Invariants:

  • job.created < `t`

func (*Job) HasQueueTtlSet added in v0.3.95 

func (job *Job) HasQueueTtlSet() bool

HasQueueTtlSet returns true if the given job has a queueTtl set.

func (*Job) HasRuns 

func (job *Job) HasRuns() bool

HasRuns returns true if the job has been run If this is returns true then LatestRun is guaranteed to return a non-nil value.

func (*Job) Id 

func (job *Job) Id() string

Id returns the id of the Job.

func (*Job) InTerminalState 

func (job *Job) InTerminalState() bool

InTerminalState returns true if the job is in a terminal state

func (*Job) JobSchedulingInfo 

func (job *Job) JobSchedulingInfo() *schedulerobjects.JobSchedulingInfo

JobSchedulingInfo returns the scheduling requirements associated with the job

func (*Job) Jobset 

func (job *Job) Jobset() string

Jobset returns the jobSet the job belongs to.

func (*Job) LatestRun 

func (job *Job) LatestRun() *JobRun

LatestRun returns the currently active job run or nil if there are no runs yet. Callers should either guard against nil values explicitly or via HasRuns.

func (*Job) NumAttempts added in v0.3.63 

func (job *Job) NumAttempts() uint

NumAttempts returns the number of times the executors tried to run this job. Note that this is O(N) on Runs, but this should be fine as the number of runs should be small.

func (*Job) NumReturned 

func (job *Job) NumReturned() uint

NumReturned returns the number of times this job has been returned by executors Note that this is O(N) on Runs, but this should be fine as the number of runs should be small.

func (*Job) PodRequirements added in v0.3.78 

func (job *Job) PodRequirements() *schedulerobjects.PodRequirements

func (*Job) Priority 

func (job *Job) Priority() uint32

Priority returns the priority of the job.

func (*Job) Queue 

func (job *Job) Queue() string

Queue returns the queue this job belongs to.

func (*Job) Queued 

func (job *Job) Queued() bool

Queued returns true if the job should be considered by the scheduler for assignment or false otherwise.

func (*Job) QueuedVersion added in v0.3.63 

func (job *Job) QueuedVersion() int32

QueuedVersion returns current queued state version.

func (*Job) RequestedPriority 

func (job *Job) RequestedPriority() uint32

RequestedPriority returns the requested priority of the job.

func (*Job) RunById 

func (job *Job) RunById(id uuid.UUID) *JobRun

RunById returns the Run corresponding to the provided run id or nil if no such Run exists.

func (*Job) SchedulingOrderCompare added in v0.4.0 

func (job *Job) SchedulingOrderCompare(other interfaces.LegacySchedulerJob) int

SchedulingOrderCompare defines the order in which jobs in a particular queue should be scheduled,

func (*Job) String added in v0.4.12 

func (job *Job) String() string

func (*Job) Succeeded 

func (job *Job) Succeeded() bool

Succeeded Returns true if the scheduler has marked the job as succeeded

func (*Job) WithCancelByJobsetRequested 

func (job *Job) WithCancelByJobsetRequested(cancelByJobsetRequested bool) *Job

WithCancelByJobsetRequested returns a copy of the job with the cancelByJobSetRequested status updated.

func (*Job) WithCancelRequested 

func (job *Job) WithCancelRequested(cancelRequested bool) *Job

WithCancelRequested returns a copy of the job with the cancelRequested status updated.

func (*Job) WithCancelled 

func (job *Job) WithCancelled(cancelled bool) *Job

WithCancelled returns a copy of the job with the cancelled status updated

func (*Job) WithCreated added in v0.3.53 

func (job *Job) WithCreated(created int64) *Job

WithCreated returns a copy of the job with the creation time updated.

func (*Job) WithFailed 

func (job *Job) WithFailed(failed bool) *Job

WithFailed returns a copy of the job with the failed status updated.

func (*Job) WithJobSchedulingInfo added in v0.3.54 

func (job *Job) WithJobSchedulingInfo(jobSchedulingInfo *schedulerobjects.JobSchedulingInfo) *Job

WithJobSchedulingInfo returns a copy of the job with the job scheduling info updated.

func (*Job) WithJobset added in v0.3.54 

func (job *Job) WithJobset(jobset string) *Job

WithJobset returns a copy of the job with the jobSet updated.

func (*Job) WithNewRun 

func (job *Job) WithNewRun(executor string, nodeId, nodeName string, scheduledAtPriority int32) *Job

WithNewRun creates a copy of the job with a new run on the given executor.

func (*Job) WithPriority 

func (job *Job) WithPriority(priority uint32) *Job

WithPriority returns a copy of the job with the priority updated.

func (*Job) WithQueue added in v0.3.54 

func (job *Job) WithQueue(queue string) *Job

WithQueue returns a copy of the job with the queue updated.

func (*Job) WithQueued 

func (job *Job) WithQueued(queued bool) *Job

WithQueued returns a copy of the job with the queued status updated.

func (*Job) WithQueuedVersion added in v0.3.63 

func (job *Job) WithQueuedVersion(version int32) *Job

WithQueuedVersion returns a copy of the job with the queued version updated.

func (*Job) WithRequestedPriority 

func (job *Job) WithRequestedPriority(priority uint32) *Job

WithRequestedPriority returns a copy of the job with the priority updated.

func (*Job) WithSubmittedTime added in v0.4.12 

func (job *Job) WithSubmittedTime(submittedTime int64) *Job

WithSubmittedTime returns a copy of the job with submittedTime updated.

func (*Job) WithSucceeded 

func (job *Job) WithSucceeded(succeeded bool) *Job

WithSucceeded returns a copy of the job with the succeeded status updated.

func (*Job) WithUpdatedRun 

func (job *Job) WithUpdatedRun(run *JobRun) *Job

WithUpdatedRun returns a copy of the job with the provided run upserted.

type JobDb 

type JobDb struct {
	// contains filtered or unexported fields
}

func NewJobDb 

func NewJobDb(priorityClasses map[string]types.PriorityClass, defaultPriorityClassName string, stringInternerCacheSize uint32) *JobDb

func NewJobDbWithSchedulingKeyGenerator added in v0.3.100 

func NewJobDbWithSchedulingKeyGenerator(
	priorityClasses map[string]types.PriorityClass,
	defaultPriorityClassName string,
	skg *schedulerobjects.SchedulingKeyGenerator,
	stringInternerCacheSize uint32,
) *JobDb

func (*JobDb) Clone added in v0.4.12 

func (jobDb *JobDb) Clone() *JobDb

Clone returns a copy of the jobDb.

func (*JobDb) CreateRun added in v0.4.5 

func (jobDb *JobDb) CreateRun(
	id uuid.UUID,
	jobId string,
	creationTime int64,
	executor string,
	nodeId string,
	nodeName string,
	scheduledAtPriority *int32,
	running bool,
	succeeded bool,
	failed bool,
	cancelled bool,
	returned bool,
	runAttempted bool,
) *JobRun

CreateRun creates a new scheduler job run from a database job run

func (*JobDb) NewJob added in v0.3.100 

func (jobDb *JobDb) NewJob(
	jobId string,
	jobSet string,
	queue string,
	priority uint32,
	schedulingInfo *schedulerobjects.JobSchedulingInfo,
	queued bool,
	queuedVersion int32,
	cancelRequested bool,
	cancelByJobSetRequested bool,
	cancelled bool,
	created int64,
) *Job

NewJob creates a new scheduler job. The new job is not automatically inserted into the jobDb; call jobDb.Upsert to upsert it.

func (*JobDb) ReadTxn 

func (jobDb *JobDb) ReadTxn() *Txn

ReadTxn returns a read-only transaction. Multiple read-only transactions can access the db concurrently

func (*JobDb) ReconcileDifferences added in v0.4.5 

func (jobDb *JobDb) ReconcileDifferences(txn *Txn, jobRepoJobs []database.Job, jobRepoRuns []database.Run) ([]JobStateTransitions, error)

ReconcileDifferences reconciles any differences between jobs stored in the jobDb with those provided to this function and returns the updated jobs together with a summary of the state transitions applied to those jobs.

func (*JobDb) SetClock added in v0.4.12 

func (jobDb *JobDb) SetClock(clock clock.PassiveClock)

func (*JobDb) SetUUIDProvider added in v0.4.12 

func (jobDb *JobDb) SetUUIDProvider(uuidProvider UUIDProvider)

func (*JobDb) WriteTxn 

func (jobDb *JobDb) WriteTxn() *Txn

WriteTxn returns a writeable transaction. Only a single write transaction may access the db at any given time so note that this function will block until any outstanding write transactions have been committed or aborted

type JobPriorityComparer added in v0.3.53 

type JobPriorityComparer struct{}

func (JobPriorityComparer) Compare added in v0.3.53 

func (JobPriorityComparer) Compare(job, other *Job) int

type JobQueueTtlComparer added in v0.3.95 

type JobQueueTtlComparer struct{}

func (JobQueueTtlComparer) Compare added in v0.3.95 

func (j JobQueueTtlComparer) Compare(a, b *Job) int

Compare jobs by their remaining queue time before expiry Invariants:

  • Job.queueTtl must be > 0
  • Job.created must be < `t`

type JobRun 

type JobRun struct {
	// contains filtered or unexported fields
}

JobRun is the scheduler-internal representation of a job run.

func MinimalRun added in v0.3.54 

func MinimalRun(id uuid.UUID, creationTime int64) *JobRun

func (*JobRun) Assert added in v0.4.12 

func (run *JobRun) Assert() error

Assert makes the assertions outlined below and returns - nil if the run is valid and - an error explaining why not otherwise.

Assertions: Required fields must be set.

The states {Running, Cancelled, Failed, Succeeded, Returned} are mutually exclusive.

func (*JobRun) Cancelled 

func (run *JobRun) Cancelled() bool

Cancelled Returns true if the user has cancelled the job run

func (*JobRun) Created 

func (run *JobRun) Created() int64

Created Returns the creation time of the job run

func (*JobRun) DeepCopy added in v0.3.53 

func (run *JobRun) DeepCopy() *JobRun

func (*JobRun) Equal added in v0.4.0 

func (run *JobRun) Equal(other *JobRun) bool

func (*JobRun) Executor 

func (run *JobRun) Executor() string

Executor returns the executor to which the JobRun is assigned.

func (*JobRun) Failed 

func (run *JobRun) Failed() bool

Failed Returns true if the executor has reported the job run as failed

func (*JobRun) Id 

func (run *JobRun) Id() uuid.UUID

Id returns the id of the JobRun.

func (*JobRun) InTerminalState 

func (run *JobRun) InTerminalState() bool

InTerminalState returns true if the JobRun is in a terminal state

func (*JobRun) JobId added in v0.3.54 

func (run *JobRun) JobId() string

JobId returns the id of the job this run is associated with.

func (*JobRun) NodeId added in v0.3.82 

func (run *JobRun) NodeId() string

NodeId returns the id of the node to which the JobRun is assigned.

func (*JobRun) NodeName added in v0.3.82 

func (run *JobRun) NodeName() string

NodeName returns the name of the node to which the JobRun is assigned.

func (*JobRun) Returned 

func (run *JobRun) Returned() bool

Returned Returns true if the executor has returned the job run.

func (*JobRun) RunAttempted added in v0.3.63 

func (run *JobRun) RunAttempted() bool

RunAttempted Returns true if the executor has attempted to run the job.

func (*JobRun) Running 

func (run *JobRun) Running() bool

Running Returns true if the executor has reported the job run as running

func (*JobRun) ScheduledAtPriority added in v0.4.8 

func (run *JobRun) ScheduledAtPriority() *int32

func (*JobRun) String added in v0.4.12 

func (run *JobRun) String() string

func (*JobRun) Succeeded 

func (run *JobRun) Succeeded() bool

Succeeded Returns true if the executor has reported the job run as successful

func (*JobRun) WithAttempted added in v0.3.63 

func (run *JobRun) WithAttempted(attempted bool) *JobRun

WithAttempted returns a copy of the job run with the runAttempted status updated.

func (*JobRun) WithCancelled 

func (run *JobRun) WithCancelled(cancelled bool) *JobRun

WithCancelled returns a copy of the job run with the cancelled status updated.

func (*JobRun) WithFailed 

func (run *JobRun) WithFailed(failed bool) *JobRun

WithFailed returns a copy of the job run with the failed status updated.

func (*JobRun) WithReturned 

func (run *JobRun) WithReturned(returned bool) *JobRun

func (*JobRun) WithRunning 

func (run *JobRun) WithRunning(running bool) *JobRun

WithRunning returns a copy of the job run with the running status updated.

func (*JobRun) WithSucceeded 

func (run *JobRun) WithSucceeded(succeeded bool) *JobRun

WithSucceeded returns a copy of the job run with the succeeded status updated.

type JobStateTransitions added in v0.4.5 

type JobStateTransitions struct {
	Job *Job

	Queued    bool
	Scheduled bool
	Pending   bool
	Running   bool
	Cancelled bool
	Preempted bool
	Failed    bool
	Succeeded bool
}

JobStateTransitions captures the process of updating a job. It bundles the updated job with booleans indicating which state transitions were applied to produce it. These are cumulative in the sense that a job with transitions queued -> scheduled -> queued -> running -> failed will have the fields queued, scheduled, running, and failed set to true.

type RealUUIDProvider added in v0.4.12 

type RealUUIDProvider struct{}

RealUUIDProvider calls uuid.New.

func (RealUUIDProvider) New added in v0.4.12 

func (_ RealUUIDProvider) New() uuid.UUID

type RunStateTransitions added in v0.4.5 

type RunStateTransitions struct {
	JobRun *JobRun

	Scheduled bool
	Returned  bool
	Pending   bool
	Running   bool
	Cancelled bool
	Preempted bool
	Failed    bool
	Succeeded bool
}

RunStateTransitions captures the process of updating a run. It works in the same way as JobStateTransitions does for jobs.

type Txn added in v0.3.53 

type Txn struct {
	// contains filtered or unexported fields
}

Txn is a JobDb Transaction. Transactions provide a consistent view of the database, allowing readers to perform multiple actions without the database changing from underneath them. Write transactions also allow callers to perform write operations that will not be visible to other users until the transaction is committed.

func (*Txn) Abort added in v0.3.53 

func (txn *Txn) Abort()

func (*Txn) Assert added in v0.4.12 

func (txn *Txn) Assert(assertOnlyActiveJobs bool) error

Assert returns an error if the jobDb, or any job stored in the jobDb, is in an invalid state. If assertOnlyActiveJobs is true, it also asserts that all jobs in the jobDb are active.

func (*Txn) AssertEqual added in v0.4.12 

func (txn *Txn) AssertEqual(otherTxn *Txn) error

func (*Txn) BatchDelete added in v0.4.1 

func (txn *Txn) BatchDelete(jobIds []string) error

BatchDelete deletes the jobs with the given ids from the database. Any ids not in the database are ignored.

func (*Txn) Commit added in v0.3.53 

func (txn *Txn) Commit()

func (*Txn) GetAll added in v0.4.1 

func (txn *Txn) GetAll() []*Job

GetAll returns all jobs in the database. The Jobs returned by this function *must not* be subsequently modified

func (*Txn) GetById added in v0.4.1 

func (txn *Txn) GetById(id string) *Job

GetById returns the job with the given Id or nil if no such job exists The Job returned by this function *must not* be subsequently modified

func (*Txn) GetByRunId added in v0.4.1 

func (txn *Txn) GetByRunId(runId uuid.UUID) *Job

GetByRunId returns the job with the given run id or nil if no such job exists The Job returned by this function *must not* be subsequently modified

func (*Txn) HasQueuedJobs added in v0.4.1 

func (txn *Txn) HasQueuedJobs(queue string) bool

HasQueuedJobs returns true if the queue has any jobs in the running state or false otherwise

func (*Txn) QueuedJobs added in v0.4.1 

func (txn *Txn) QueuedJobs(queue string) *immutable.SortedSetIterator[*Job]

QueuedJobs returns true if the queue has any jobs in the running state or false otherwise

func (*Txn) QueuedJobsByTtl added in v0.4.1 

func (txn *Txn) QueuedJobsByTtl() *immutable.SortedSetIterator[*Job]

QueuedJobsByTtl returns an iterator for jobs ordered by queue ttl time - the closest to expiry first

func (*Txn) Upsert added in v0.4.1 

func (txn *Txn) Upsert(jobs []*Job) error

Upsert will insert the given jobs if they don't already exist or update them if they do.

type UUIDHasher added in v0.3.89 

type UUIDHasher struct{}

UUIDHasher is an implementation of Hasher for UUID.

func (UUIDHasher) Equal added in v0.3.89 

func (h UUIDHasher) Equal(a, b uuid.UUID) bool

Equal checks if two UUIDs are equal.

func (UUIDHasher) Hash added in v0.3.89 

func (h UUIDHasher) Hash(key uuid.UUID) uint32

Hash computes a hash for a UUID.

type UUIDProvider added in v0.4.12 

type UUIDProvider interface {
	New() uuid.UUID
}

UUIDProvider is an interface used to mock UUID generation for tests.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.