submit

package
v0.0.0-...-d3da680 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 4, 2024 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Overview

Package submit contains utilities for Run submission.

Index

Constants

This section is empty.

Variables

View Source
var ErrTransientSubmissionFailure = errors.New("submission failed transiently", transient.Tag)

ErrTransientSubmissionFailure indicates that the submission has failed transiently and the same task should be retried.

Functions

func ComputeOrder

func ComputeOrder(cls []*run.RunCL) ([]*run.RunCL, error)

ComputeOrder computes the best order for submission of CLs based on their dependencies. Dependency cycle is broken by arbitrarily but deterministically breaking soft requirement dependencies and approximately minimizing their number.

Returns error if duplicate CLs are provided or the hard requirement dependencies solely form a cycle (This should not be possible within current CV context as hard requirement deps can only be established if CLs follow Git's child -> parent relationship).

Overall, this is as hard as a minimum feedback arc set, which is NP-Hard: https://en.wikipedia.org/wiki/Feedback_arc_set#Minimum_feedback_arc_set However, approximation is fine within CV context so long as hard dependencies are satisfied.

func CurrentRun

func CurrentRun(ctx context.Context, luciProject string) (common.RunID, error)

CurrentRun returns the RunID that is currently submitting in the submit queue of the provided LUCI Project.

func DeleteQueue

func DeleteQueue(ctx context.Context, luciProject string) error

DeleteQueue deletes the submit queue of the provided LUCI Project.

func LoadCurrentAndWaitlist

func LoadCurrentAndWaitlist(ctx context.Context, runID common.RunID) (current common.RunID, waitlist common.RunIDs, err error)

LoadCurrentAndWaitlist loads the current submission slot and the waitlist.

func MustCurrentRun

func MustCurrentRun(ctx context.Context, luciProject string) common.RunID

MustCurrentRun is like `CurrentRun(...)` but panic on error.

func Release

func Release(ctx context.Context, notifyFn NotifyFn, runID common.RunID) error

Release releases the slot occupied by the provided Run.

If the provided Run occupies the current slot, give it up and notify the first Run in the waitlist is ready for submission. If the provided Run is in waitlist, remove it from the waitlist. If the provided Run is not present in the submit queue, no-op.

MUST be called in a datastore transaction.

func ReleaseOnSuccess

func ReleaseOnSuccess(ctx context.Context, notifyFn NotifyFn, runID common.RunID, submittedAt time.Time) error

ReleaseOnSuccess, in addition to releasing the slot like `Release`, also records the timestamp of a successful submission.

Note that the submitted time will only be recorded if the provided Run takes the current slot. It is used to support rate limiting based on the `SubmitOptions` defined in the ProjectConfig.

MUST be called in a datastore transaction.

func TryAcquire

func TryAcquire(ctx context.Context, notifyFn NotifyFn, runID common.RunID, opts *cfgpb.SubmitOptions) (waitlisted bool, err error)

TryAcquire tries to acquire the current submission slot from the submit queue.

Succeeds iff both

  1. queue is empty or the requested Run is the first in the waitlist and no Run is occupying the current submission slot at this moment
  2. acquisition of the current slot satisfies the rate limit.

Otherwise, this Run will be added to the waitlist and will be notified later after all Runs ahead of it have released their slots and the rate limit allows. Acquiring a slot for an existing Run in the submit queue won't change its position in the queue unless this Run is the first Run in the waitlist and the aforementioned conditions are met.

The caller SHOULD submit the Run only after it successfully acquires the current submission slot. The caller is also RECOMMENDED to submit a task in the same transaction that runs later to check if the submission have failed unexpectedly without releasing the slot so that the submission slot is blocked on this Run.

MUST be called in a datastore transaction.

Types

type NotifyFn

type NotifyFn func(ctx context.Context, runID common.RunID, eta time.Time) error

NotifyFn is used to notify the run is ready for submission at `eta`

In production, it is run.Notifier.NotifyReadyForSubmission(...)

type RM

type RM interface {
	Invoke(ctx context.Context, runID common.RunID, eta time.Time) error
	NotifyReadyForSubmission(ctx context.Context, runID common.RunID, eta time.Time) error
	NotifyCLsSubmitted(ctx context.Context, runID common.RunID, clids common.CLIDs) error
	NotifySubmissionCompleted(ctx context.Context, runID common.RunID, sc *eventpb.SubmissionCompleted, invokeRM bool) error
}

RM encapsulates interaction with Run Manager by `RunCLsSubmitter`.

type RunCLsSubmitter

type RunCLsSubmitter struct {
	// contains filtered or unexported fields
}

RunCLsSubmitter submits CLs for a Run.

func NewSubmitter

func NewSubmitter(ctx context.Context, runID common.RunID, submission *run.Submission, rm RM, g gerrit.Factory) *RunCLsSubmitter

NewSubmitter creates a new RunCLsSubmitter.

func (RunCLsSubmitter) Submit

func (s RunCLsSubmitter) Submit(ctx context.Context) error

Submit tries to incrementally submits all CLs within deadline.

Notifies RunManager whenever a CL is submitted successfully. Reports the final submission result to RunManager and releases the submit queue at the end.

Submission is considered as failure if this Run no longer holds the submit queue or can't submit all CLs within deadline.

Returns `ErrTransientSubmissionFailure` or error that happened when reporting final result to Run Manager, or releasing submit queue only. The rest of the errors that happened during the submission will be reported to Run Manager as part of the submission result.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL