inputprocessor

Documentation

Overview

Package inputprocessor contains code for processing actions to determine their inputs/outputs.

Index

• type ActionSpec
  • func Compute(p Preprocessor, options Options) (*ActionSpec, error)
• type BasePreprocessor
  • func (c *BasePreprocessor) AppendSpec(s *ActionSpec)
  • func (c *BasePreprocessor) ComputeSpec() error
  • func (c *BasePreprocessor) Error() error
  • func (c *BasePreprocessor) FilterInputsUnderExecRoot()
  • func (c *BasePreprocessor) FilterVirtualInputs()
  • func (c *BasePreprocessor) FlagsToActionSpec()
  • func (c *BasePreprocessor) Init(options Options)
  • func (c *BasePreprocessor) ParseFlags() error
  • func (c *BasePreprocessor) ProcessToolchains() error
  • func (c *BasePreprocessor) RewriteEnvironmentVariables()
  • func (c *BasePreprocessor) Sanitize()
  • func (c *BasePreprocessor) Spec() (*ActionSpec, error)
• type Executor
• type Options
• type Preprocessor

Types

type ActionSpec

type ActionSpec struct {
	// InputSpec holds information about files and environment variables required to
	// run the command.
	InputSpec *command.InputSpec
	// OutputFiles is a list of output files produced by the command.
	OutputFiles []string
	// OutputDirectories is a list of output directories produced by the command.
	OutputDirectories []string
	// EmiitedDependencyFile is the name of the dependency file produced by the command.
	EmittedDependencyFile string
	// UsedShallowMode indicates whether the shallow input processor was used to
	// determine inputs.
	UsedShallowMode bool
}

ActionSpec encapsulates the inputs and outputs a command. All paths are relative to the exec root.

func Compute

func Compute(p Preprocessor, options Options) (*ActionSpec, error)

Compute computes the ActionSpec using the given preprocessor and options.

type BasePreprocessor

type BasePreprocessor struct {
	// Ctx is the context to use for internal preprocessing actions.
	Ctx context.Context
	// Executor is an entity that can execute commands on the local system.
	Executor Executor
	// ResourceManager manages available local resources to ensure local operations do not
	// overwhelm the machine.
	ResourceManager *localresources.Manager
	// Options is the options used to initialize the preprocessor. Should not change once the
	// preprocessor is initialized.
	Options Options
	// Flags is the set of flags determined from parsing the command. Should not change once
	// ParseFlags has been called.
	Flags *flags.CommandFlags
	// Err is an error encountered during preprocessing. An action that has an error could
	// not be remotely executed or cached.
	Err error

	// FileMetadataCache is used to obtain the metadata of files.
	FileMetadataCache filemetadata.Cache
	// NormalizedFileCache is used to cache normalized paths.
	NormalizedFileCache *cache.SingleFlight
	// FileStatCache caches the results of os.Stat calls.
	FileStatCache *cache.SingleFlight
	// contains filtered or unexported fields
}

BasePreprocessor is the base preprocessor with commonly objects and preprocessing logic.

func (*BasePreprocessor) AppendSpec

func (c *BasePreprocessor) AppendSpec(s *ActionSpec)

AppendSpec appends the given ActionSpec to the ActionSpec stored in the preprocessor.

func (*BasePreprocessor) ComputeSpec

func (c *BasePreprocessor) ComputeSpec() error

ComputeSpec computes any further action specification that is not immediately inferrable from flags or toolchain configuration.

func (*BasePreprocessor) Error

func (c *BasePreprocessor) Error() error

Error returns the fatal error encountered during input processing if exists.

func (*BasePreprocessor) FilterInputsUnderExecRoot

func (c *BasePreprocessor) FilterInputsUnderExecRoot()

FilterInputsUnderExecRoot removes any inputs that do not exist under the exec root.

func (*BasePreprocessor) FilterVirtualInputs

func (c *BasePreprocessor) FilterVirtualInputs()

FilterVirtualInputs removes virtual inputs that are not physically existing directories.

func (*BasePreprocessor) FlagsToActionSpec

func (c *BasePreprocessor) FlagsToActionSpec()

FlagsToActionSpec populates the ActionSpec struct with inputs parsed fromt the command flags.

func (*BasePreprocessor) Init

func (c *BasePreprocessor) Init(options Options)

Init initializes the preprocessor with the given options.

func (*BasePreprocessor) ParseFlags

func (c *BasePreprocessor) ParseFlags() error

ParseFlags parses the commands flags and populates the ActionSpec object with inferred information.

func (*BasePreprocessor) ProcessToolchains

func (c *BasePreprocessor) ProcessToolchains() error

ProcessToolchains determines toolchain inputs required for the command.

func (*BasePreprocessor) RewriteEnvironmentVariables

func (c *BasePreprocessor) RewriteEnvironmentVariables()

RewriteEnvironmentVariables makes all environment variables specified in the input spec relative to the working directory.

func (*BasePreprocessor) Sanitize

func (c *BasePreprocessor) Sanitize()

Sanitize cleans up the spec by removing unwanted entries. Normally this includes deduping and removing non-existent inputs.

func (*BasePreprocessor) Spec

func (c *BasePreprocessor) Spec() (*ActionSpec, error)

Spec retrieves the ActionSpec currently inferred for the options passed to the context. For complete input/output processing, call ComputeSpec first.

type Executor

type Executor interface {
	Execute(ctx context.Context, cmd *command.Command) (string, string, error)
}

Executor can run commands and retrieve their outputs.

type Options

type Options struct {
	// ExecutionID is the ID of the action.
	ExecutionID string
	// Cmd is the list of args.
	Cmd []string
	// WorkingDir is the working directory of the action.
	WorkingDir string
	// ExecRoot is the exec root of the action.
	ExecRoot string
	// Inputs is the InputSpec passed explicitly with the action request.
	Inputs *command.InputSpec
	// Labels is a map of label keys to values.
	Labels map[string]string
	// ToolchainInputs is a list of toolchain inputs in addition to the toolchains
	// inferred from the command.
	ToolchainInputs []string
	// ShallowFallback indicates whether preprocessing is allowed to fallback to shallow
	// mode if an error is encountered.
	ShallowFallback bool

	// WindowsCross indicates whether to use Linux worker for Windows.
	WindowsCross bool
}

Options encapsulates options for initializing a preprocessor.

type Preprocessor

type Preprocessor interface {
	// Init initializes the preprocessor with the given options.
	Init(options Options)
	// ParseFlags parses the commands flags and populates the ActionSpec object with inferred
	// information. Returning an error means preprocessing should not continue, and the user
	// should Sanitize() the currently processed ActionSpec and then read it if available using
	// Spec().
	ParseFlags() error
	// ProcessToolchains determines toolchain inputs required for the command. Returning an
	// error means preprocessing should not continue, and the user should Sanitize() the
	// currently processed ActionSpec and then read it if available using Spec().
	ProcessToolchains() error
	// ComputeSpec computes any further action specification that is not immediately inferrable
	// from flags or toolchain configuration. Returning an error means preprocessing should not
	// continue, and the user should Sanitize() the currently processed ActionSpec and then read
	// it if available using Spec().
	ComputeSpec() error
	// Sanitize cleans up the spec by removing unwanted entries. Normally this includes
	// deduping and removing non-existent inputs.
	Sanitize()
	// Spec retrieves the ActionSpec currently inferred for the options passed to the
	// context.
	Spec() (*ActionSpec, error)
	// Error returns the fatal error encountered during preprocessing, if exists.
	Error() error
}

Preprocessor is an interface for determining specs of an action. Refer to Compute() below to see how implementers of the interface should be called.