Documentation
¶
Index ¶
- Variables
- func ClusterInstance(config infra.Config) (runner.Cluster, error)
- func HttpClient(config infra.Config) (*http.Client, error)
- func PredictorConfig(cfg infra.Config, validate bool) (*infra2.PredictorConfig, error)
- type CommonRunFlags
- type Eval
- type FlagName
- type ImageResolver
- type ReflowRunner
- type ReflowRuntime
- type RunConfig
- type RunFlags
- type RunnerParams
- type RuntimeParams
Constants ¶
This section is empty.
Variables ¶
var RunSchema = infra.Schema{ infra2.Assoc: new(assoc.Assoc), infra2.Cache: new(infra2.CacheProvider), infra2.Labels: make(pool.Labels), infra2.Repository: new(reflow.Repository), infra2.RunID: new(taskdb.RunID), infra2.Session: new(session.Session), infra2.Username: new(infra2.User), infra2.Log: new(log.Logger), }
RunSchema is the infra schema for a run.
Functions ¶
func ClusterInstance ¶
ClusterInstance returns a configured cluster and sets up repository credentials so that remote repositories can be dialed.
func PredictorConfig ¶
PredictorConfig returns a PredictorConfig (possibly nil) and an error (if any).
Types ¶
type CommonRunFlags ¶
type CommonRunFlags struct {
// Assert is the policy used to assert cached flow result compatibility. e.g. never, exact.
Assert string
// EvalStrategy is the evaluation strategy. Supported modes are "topdown" and "bottomup".
EvalStrategy string
// Invalidate is a regular expression for node identifiers that should be invalidated.
Invalidate string
// PostUseChecksum indicates whether input filesets are checksummed after use.
PostUseChecksum bool
// RecomputeEmpty indicates if cache results with empty filesets be automatically recomputed.
RecomputeEmpty bool
}
CommonRunFlags are the run flags that are common across various run modes (run, batch, etc)
func (*CommonRunFlags) Configure ¶
func (r *CommonRunFlags) Configure(c *flow.EvalConfig) (err error)
Configure stores the RunFlags's configuration into the provided EvalConfig.
func (*CommonRunFlags) Err ¶
func (r *CommonRunFlags) Err() error
Err checks if the flag values are consistent and valid.
func (*CommonRunFlags) Flags ¶
func (r *CommonRunFlags) Flags(flags *flag.FlagSet)
Flags adds the common run flags to the provided flagset.
type Eval ¶
type Eval struct {
// InputArgs is the raw input arguments of the evaluation, as passed
// in by user.
InputArgs []string
// Program stores the reflow program's path.
Program string
// Params stores the evaluation's module parameters and raw values.
Params map[string]string
// Args stores the evaluation's command line arguments.
Args []string
// V1 tells whether this program is a "V1" (".rf") program.
V1 bool
// Images is the list of images that were parsed from a syntax session.
Images []string
// ImageMap stores a mapping between image names and resolved
// image names, to be used in evaluation.
ImageMap map[string]string
// Type is the module type of the toplevel module that has been
// evaluated.
Type *types.T
// Module is the module value that was evaluated.
Module values.Module
}
Eval represents the evaluation of a single module. Evaluations are performed by Cmd.Eval.
func (*Eval) ResolveImages ¶
Resolve images resolves the images in an evaluated program.
func (*Eval) Run ¶
Run evaluates a reflow program to a flow. It can evaluate both legacy (".reflow") and modern (".rf") programs. It interprets flags as module parameters. Input arguments and options are specified in the passed-in Eval; results are deposited there, too. Run will also return a reflow Bundle for the given reflow program, if getBundle is set to true and if applicable (ie, only if it is a V1 reflow program with .rf or .rfx extension)
type FlagName ¶
type FlagName string
const ( Unknown FlagName = "unknown" // CommonRunFlags flag names FlagNameAssert FlagName = "assert" FlagNameEvalStrategy FlagName = "eval" FlagNameInvalidate FlagName = "invalidate" // Deprecated: externs are no longer cached. // TODO(pfialho): remove flag. FlagNameNoCacheExtern FlagName = "nocacheextern" FlagNamePostUseChecksum FlagName = "postusechecksum" FlagNameRecomputeEmpty FlagName = "recomputeempty" // RunFlags flag names FlagNameBackgroundTimeout FlagName = "backgroundtimeout" FlagNameDotGraph FlagName = "dotgraph" FlagNamePred FlagName = "pred" FlagNameTrace FlagName = "traceflow" )
type ImageResolver ¶
type ImageResolver struct {
// Authenticator will have a nil AWS session if the config does not have
// AWS credentials, in which case it is expected that none of the images are
// ECR images.
Authenticator ecrauth.Interface
// contains filtered or unexported fields
}
ImageResolver maintains maps of image descriptions to their canonical values (fully qualified registry host, and digest based references)
func (*ImageResolver) ResolveImages ¶
type ReflowRunner ¶
type ReflowRunner interface {
Go(context.Context) (runner.State, error)
GetRunID() taskdb.RunID
// TODO(swami): Remove this.
SetDotWriter(io.Writer)
}
ReflowRunner supports the ability to run a reflow program.
type ReflowRuntime ¶
type ReflowRuntime interface {
// Start starts the reflow runtime.
// Start uses the provided context to run the necessary background processes,
// upon cancellation of which, these processes are stopped and the runtime becomes unviable.
Start(ctx context.Context)
// WaitDone waits for the reflow runtime to be done (which happens only if ctx to `Start` was cancelled).
WaitDone()
// Scheduler returns the underlying scheduler.
// TODO(swami): Remove this. This is exposed temporarily and shouldn't need to be eventually.
Scheduler() *sched.Scheduler
// NewRunner returns a new reflow runner.
NewRunner(RunnerParams) (ReflowRunner, error)
}
ReflowRuntime represents a reflow runtime environment for submitting reflow runs. ReflowRuntime manages all underlying infrastructure components necessary to process reflow runs (eg: Cluster, Scheduler, etc)
func NewRuntime ¶
func NewRuntime(p RuntimeParams) (ReflowRuntime, error)
NewRuntime creates a new reflow runtime from the given parameters. NewRuntime returns an error upon failure to setup and initialize a runtime.
type RunConfig ¶
type RunConfig struct {
// Program is the local path to the reflow bundle/program.
Program string
// Args is the arguments to be passed to the reflow program in the form of "--<argname>=<value>"
Args []string
// RunFlags is the run flags for this run.
RunFlags RunFlags
// Config is the config for this run.
Config infra.Config
}
RunConfig defines all the material (configuration, program and args) for a specific run.
type RunFlags ¶
type RunFlags struct {
CommonRunFlags
// LocalDir is the directory where execution state is store in Local mode.
LocalDir string
// Dir is the where execution state is stored in local mode (alias for backwards compatibility).
Dir string
// Local enables execution using the local docker instance.
Local bool
// Trace when set enable tracing flow evaluation.
Trace bool
Cache bool
Pred bool
// DotGraph enables computation of an evaluation graph.
DotGraph bool
// BackgroundTimeout is the duration to wait for background tasks (such as cache writes, etc) to complete.
// ie, this is the amount of time we wait after the user's program execution finishes but before reflow exits.
BackgroundTimeout time.Duration
}
RunFlags is the supported flags and parameters for a run.
func (*RunFlags) FlagsNoLocal ¶
FlagsNoLocal adds run flags to the provided flagset with the given prefix excluding flags pertaining to local runs.
type RunnerParams ¶
type RunnerParams struct {
RunConfig RunConfig
Logger *log.Logger
// Status is the status object specific to a run.
Status *status.Status
}
RunnerParams defines the set of parameters necessary to initialize a ReflowRunner.
type RuntimeParams ¶
type RuntimeParams struct {
// Config is the main configuration for the reflow runtime.
Config infra.Config
// Logger is the top-level logger passed to the various underlying components.
// Logger is optional and overrides the Config provided logger (if any).
Logger *log.Logger
// Status is the top-level status object. If provided, various internal components
// will create groups under this status object for reporting purposes.
Status *status.Status
}
RuntimeParams defines the set of parameters necessary to initialize a ReflowRuntime.
Source Files