ledcmd

Documentation

Overview

Package ledcmd implements the subcommands for the `led` command line tool which interact with external services.

Index

• Constants
• func ConsolidateIsolateSources(ctx context.Context, authClient *http.Client, jd *job.Definition) error
• func ConsolidateRbeCasSources(ctx context.Context, authOpts auth.Options, jd *job.Definition) error
• func EditIsolated(ctx context.Context, authClient *http.Client, authOpts auth.Options, ...) error
• func EditRecipeBundle(ctx context.Context, authClient *http.Client, authOpts auth.Options, ...) error
• func GetBuild(ctx context.Context, authClient *http.Client, opts GetBuildOpts) (*job.Definition, error)
• func GetBuilder(ctx context.Context, authClient *http.Client, opts GetBuildersOpts) (*job.Definition, error)
• func GetFromSwarmingTask(ctx context.Context, authClient *http.Client, opts GetFromSwarmingTaskOpts) (*job.Definition, error)
• func GetUID(ctx context.Context, authenticator *auth.Authenticator) (string, error)
• func LaunchSwarming(ctx context.Context, authClient *http.Client, jd *job.Definition, ...) (*swarming.SwarmingRpcsNewTaskRequest, ...)
• type EditRecipeBundleOpts
• type GetBuildOpts
• type GetBuildersOpts
• type GetFromSwarmingTaskOpts
• type IsolatedTransformer
  • func ProgramIsolatedTransformer(args ...string) IsolatedTransformer
  • func PromptIsolatedTransformer() IsolatedTransformer
• type LaunchSwarmingOpts

Constants

const RecipeDirectory = "kitchen-checkout"

RecipeDirectory is a very unfortunate constant which is here for a combination of reasons:

1. swarming doesn't allow you to 'checkout' an isolate relative to any path in the task (other than the task root). This means that whatever value we pick for EditRecipeBundle must be used EVERYWHERE the isolated hash is used.
2. Currently the 'recipe_engine/led' module will blindly take the isolated input and 'inject' it into further uses of led. This module currently doesn't specify the checkout dir, relying on kitchen's default value of (you guessed it) "kitchen-checkout".

In order to fix this (and it will need to be fixed for bbagent support):

• The 'recipe_engine/led' module needs to accept 'checkout-dir' as a parameter in its input properties.
• led needs to start passing the checkout dir to the led module's input properties.
• `led edit` needs a way to manipulate the checkout directory in a job
• The 'recipe_engine/led' module needs to set this in the job alongside the isolate hash when it's doing the injection.

For now, we just hard-code it.

TODO(crbug.com/1072117): Fix this, it's weird.

Functions

func ConsolidateIsolateSources

func ConsolidateIsolateSources(ctx context.Context, authClient *http.Client, jd *job.Definition) error

ConsolidateIsolateSources will, for Swarming tasks:

• Extract Cmd/Cwd from the slice.Properties.CasInputs (if set) and assign them directly into the swarming task slice.
• Synthesize a new CasInput (and push it to the isolate server) which:
• has no includes, cwd, cmd, etc.
• is the union of UserPayload and slice.Properties.CasInputs
• Combined isolated in slice.Properties.CasInputs.

func ConsolidateRbeCasSources

func ConsolidateRbeCasSources(ctx context.Context, authOpts auth.Options, jd *job.Definition) error

ConsolidateRbeCasSources combines RBE-CAS inputs in slice.Properties.CasInputRoot and CasUserPayload for swarming tasks. For the same file, the one in CasUserPayload will replace the one in slice.Properties.CasInputRoot.

func EditIsolated

func EditIsolated(ctx context.Context, authClient *http.Client, authOpts auth.Options, jd *job.Definition, xform IsolatedTransformer) error

EditIsolated allows you to edit the isolated (input_ref or cas_input_root) contents of the job.Definition.

This implicitly collapses all isolated sources in the job.Definition into a single isolated source. The output job.Definition always has cas_user_payload and no user_payload.

func EditRecipeBundle

func EditRecipeBundle(ctx context.Context, authClient *http.Client, authOpts auth.Options, jd *job.Definition, opts *EditRecipeBundleOpts) error

EditRecipeBundle overrides the recipe bundle in the given job with one located on disk.

It isolates the recipes from the repository in the given working directory into the UserPayload under the directory "kitchen-checkout/". If there's an existing directory in the UserPayload at that location, it will be removed.

func GetBuild

func GetBuild(ctx context.Context, authClient *http.Client, opts GetBuildOpts) (*job.Definition, error)

GetBuild retrieves a job Definition from a Buildbucket build.

func GetBuilder

func GetBuilder(ctx context.Context, authClient *http.Client, opts GetBuildersOpts) (*job.Definition, error)

GetBuilder retrieves a new job Definition from a Buildbucket builder.

func GetFromSwarmingTask

func GetFromSwarmingTask(ctx context.Context, authClient *http.Client, opts GetFromSwarmingTaskOpts) (*job.Definition, error)

GetFromSwarmingTask retrieves and renders a JobDefinition from the given swarming task, printing it to stdout and returning an error.

func GetUID

func GetUID(ctx context.Context, authenticator *auth.Authenticator) (string, error)

GetUID derives a user id string from the Authenticator for use with LaunchSwarming.

If the given authenticator has the userinfo.email scope, this will be the email associated with the Authenticator. Otherwise, this will be 'uid:<opaque user id>'.

func LaunchSwarming

func LaunchSwarming(ctx context.Context, authClient *http.Client, jd *job.Definition, opts LaunchSwarmingOpts) (*swarming.SwarmingRpcsNewTaskRequest, *swarming.SwarmingRpcsTaskRequestMetadata, error)

LaunchSwarming launches the given job Definition on swarming, returning the NewTaskRequest launched, as well as the launch metadata.

Types

type EditRecipeBundleOpts

type EditRecipeBundleOpts struct {
	// Path on disk to the repo to extract the recipes from. May be a subdirectory
	// of the repo, as long as `git rev-parse --show-toplevel` can find the root
	// of the repository.
	//
	// If empty, uses the current working directory.
	RepoDir string

	// Overrides is a mapping of recipe project id (e.g. "recipe_engine") to
	// a local path to a checkout of that repo (e.g. "/path/to/recipes-py.git").
	//
	// When the bundle is created, this local repo will be used instead of the
	// pinned version of this recipe project id. This is helpful for preparing
	// bundles which have code changes in multiple recipe repos.
	Overrides map[string]string

	// DebugSleep is the amount of time to wait after the recipe completes
	// execution (either success or failure). This is injected into the generated
	// recipe bundle as a 'sleep X' command after the invocation of the recipe
	// itself.
	DebugSleep time.Duration
}

EditRecipeBundleOpts are user-provided options for the recipe bundling process.

type GetBuildOpts

type GetBuildOpts struct {
	BuildbucketHost string
	BuildID         int64
	PinBotID        bool
	PriorityDiff    int
	KitchenSupport  job.KitchenSupport
}

GetBuildOpts are the options for GetBuild.

type GetBuildersOpts

type GetBuildersOpts struct {
	BuildbucketHost string
	Bucket          string
	Builder         string
	Canary          bool
	ExtraTags       []string
	PriorityDiff    int

	KitchenSupport job.KitchenSupport
}

GetBuildersOpts are the options for GetBuilder.

type GetFromSwarmingTaskOpts

type GetFromSwarmingTaskOpts struct {
	// The swarming host to retrieve the task from.
	SwarmingHost string

	// The ID of the task to retrieve.
	TaskID string

	// If the resulting Definition should be pinned to the same bot id that the
	// original task ran on (replaces job dimensions with just the 'id'
	// dimension).
	//
	// NOTE: This only "works" for bots which are managed statically. Dynamically
	// allocated bots (e.g. those from GCE Provider) have names which may recycle
	// from the time of the original swarming task to when GetFromSwarmingTask
	// runs, which means that 'pinning' will only get you a bot with the same
	// name, not necessarially the original bot.
	//
	// TODO: Remove this when we no longer mangage bots statically.
	PinBotID bool

	// The "name" of the resulting job Definition.
	Name string

	// The difference from the original priority.
	PriorityDiff int

	KitchenSupport job.KitchenSupport
}

GetFromSwarmingTaskOpts are the options for GetFromSwarmingTask.

type IsolatedTransformer

type IsolatedTransformer func(ctx context.Context, directory string) error

IsolatedTransformer is a function which receives a directory on the local disk with the contents of an isolate and is expected to manipulate the contents of that directory however it chooses.

EditIsolated takes these functions as a callback in order to manipulate the isolated content of a job.Definition.

func ProgramIsolatedTransformer

func ProgramIsolatedTransformer(args ...string) IsolatedTransformer

ProgramIsolatedTransformer returns an IsolatedTransformer which alters the contents of the isolated by running a program specified with `args` in the directory where the isolated content has been unpacked.

func PromptIsolatedTransformer

func PromptIsolatedTransformer() IsolatedTransformer

PromptIsolatedTransformer returns an IsolatedTransformer which prompts the user to navigate to the directory with the isolated content and manipulate it manually. When the user is done they should press "enter" to indicate that they're finished.

type LaunchSwarmingOpts

type LaunchSwarmingOpts struct {
	// If true, just generates the NewTaskRequest but does not send it to swarming
	// (SwarmingRpcsTaskRequestMetadata will be nil).
	DryRun bool

	// Must be a unique user identity string and must not be empty.
	//
	// Picking a bad value here means that generated logdog prefixes will
	// possibly collide, and the swarming task's User field will be misreported.
	//
	// See GetUID to obtain a standardized value here.
	UserID string

	// If launched from within a swarming task, this will be the current swarming
	// task's task id to be attached as the parent of the launched task.
	ParentTaskId string

	// A path, relative to ${ISOLATED_OUTDIR} of where to place the final
	// build.proto from this build. If omitted, the build.proto will not be
	// dumped.
	FinalBuildProto string

	KitchenSupport job.KitchenSupport

	// A flag for swarming/ResultDB integration on the launched task.
	ResultDB job.RDBEnablement
}

LaunchSwarmingOpts are the options for LaunchSwarming.