execution

package
v0.1.0-beta.202310231325 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 23, 2023 License: AGPL-3.0 Imports: 25 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Category 

func Category(category string) map[string]interface{}

func LoadEventLogEntries 

func LoadEventLogEntries(executionID string) ([]types.EventLogEntry, error)

Types

type Execution 

type Execution struct {
	Context context.Context `json:"-"`

	// Unique identifier for this execution.
	ID string `json:"id"`

	// Pipelines triggered by the execution. Even if the pipelines are nested,
	// we maintain a flat list of all pipelines for easy lookup and querying.
	PipelineExecutions map[string]*PipelineExecution `json:"pipeline_executions"`
}

Execution represents the current state of an execution. A single execution is tied to a trigger (webhook, cronjob, etc) and may result in multiple pipelines being executed.

func NewExecution 

func NewExecution(ctx context.Context, opts ...ExecutionOption) (*Execution, error)

func (*Execution) BuildEvalContext 

func (ex *Execution) BuildEvalContext(pipelineDefn *modconfig.Pipeline, pe *PipelineExecution) (*hcl.EvalContext, error)

func (*Execution) LoadJSON 

func (ex *Execution) LoadJSON(fileName string) error

LoadFromFile loads an execution from a JSON file.

func (*Execution) LoadProcess 

func (ex *Execution) LoadProcess(e *event.Event) error

This function loads the event log file (the .jsonl file) continously and update the ex.PipelineExecutions and ex.StepExecutions

func (*Execution) LogFilePath 

func (ex *Execution) LogFilePath() (string, error)

LogFilePath returns the path to the log file for the execution.

func (*Execution) ParentStepExecution 

func (ex *Execution) ParentStepExecution(pipelineExecutionID string) (*StepExecution, error)

ParentStepExecution returns the parent step execution for the given pipeline execution ID.

func (*Execution) PipelineData 

func (ex *Execution) PipelineData(pipelineExecutionID string) (map[string]interface{}, error)

func (*Execution) PipelineDefinition 

func (ex *Execution) PipelineDefinition(pipelineExecutionID string) (*modconfig.Pipeline, error)

func (*Execution) PipelineStepExecutions 

func (ex *Execution) PipelineStepExecutions(pipelineExecutionID, stepName string) []StepExecution

PipelineStepExecutions returns a list of step executions for the given pipeline execution ID and step name.

func (*Execution) PipelineStepOutputs 

func (ex *Execution) PipelineStepOutputs(pipelineExecutionID string) (map[string]interface{}, error)

PipelineStepOutputs returns a single map of all outputs from all steps in the given pipeline execution. The map is keyed by the step name. If a step has a ForTemplate then the result is an array of outputs.

func (*Execution) Snapshot 

func (ex *Execution) Snapshot(pipelineExecutionID string) (*Snapshot, error)

func (*Execution) StepDefinition 

func (ex *Execution) StepDefinition(pipelineExecutionID, stepExecutionID string) (modconfig.IPipelineStep, error)

StepDefinition returns the step definition for the given step execution ID.

func (*Execution) StepExecutionNodeRow 

func (ex *Execution) StepExecutionNodeRow(panelName string, sd modconfig.IPipelineStep, se *StepExecution) SnapshotPanelDataRow

func (*Execution) StepExecutionSnapshotPanels 

func (ex *Execution) StepExecutionSnapshotPanels(pipelineExecutionID string, stepFullyQualifiedName string) (map[string]SnapshotPanel, error)

StepExecutionSnapshotPanels will build and return a set of panels to represent the step execution in a dashboard. The panels includes both nodes and edges, depending on the exection of the step - for example, if the step has a for loop then it will be a start node, fan out to loop items and collapse back to an end node.

type ExecutionOption 

type ExecutionOption func(*Execution) error

ExecutionOption is a function that modifies an Execution instance.

func WithEvent 

func WithEvent(e *event.Event) ExecutionOption

func WithID 

func WithID(id string) ExecutionOption

type ExecutionStepOutputs 

type ExecutionStepOutputs map[string]map[string]interface{}
  "http" = {
     "http_1": {},
     "http_2": {},
  }
}

The first level is grouping the output by the step type The next level group the output by the step name The value can be a StepOutput OR a slice of StepOutput

type PipelineExecution 

type PipelineExecution struct {
	// Unique identifier for this pipeline execution
	ID string `json:"id"`
	// The name of the pipeline
	Name string `json:"name"`
	// The input to the pipeline
	Args modconfig.Input `json:"args,omitempty"`

	// The output of the pipeline
	PipelineOutput map[string]interface{} `json:"pipeline_output,omitempty"`

	// The status of the pipeline execution: queued, planned, started, completed, failed
	Status string `json:"status"`

	// Status of each step on a per-step basis. Used to determine if dependencies
	// have been met etc. Note that each step may have multiple executions, the status
	// of which are not tracked here.
	// dependencies have been met, etc. The step status is on a per-step
	StepStatus map[string]*StepStatus `json:"-"`

	// If this is a child pipeline, then track it's parent
	ParentStepExecutionID string `json:"parent_step_execution_id,omitempty"`
	ParentExecutionID     string `json:"parent_execution_id,omitempty"`

	// All errors from the step execution + any errors that can be added to the pipeline execution manually
	Errors []modconfig.StepError `json:"errors,omitempty"`

	// The final native/primitive output for all the steps in this pipeline execution.
	AllNativeStepOutputs ExecutionStepOutputs `json:"-"`

	// The final configured output for all the steps in this pipeline execution.
	AllConfigStepOutputs ExecutionStepOutputs `json:"-"`

	// Steps triggered by pipelines in the execution.
	StepExecutions map[string]*StepExecution `json:"step_executions,omitempty"`

	// TODO: not sure if we need this, it's a different index of the step executions
	// TODO: but also a way to track the order of execution for a given step
	StepExecutionOrder map[string][]string `json:"-"`
}

PipelineExecution represents the execution of a single types.

func (*PipelineExecution) Fail 

func (pe *PipelineExecution) Fail(stepName string, stepError ...modconfig.StepError)

func (*PipelineExecution) FailStep 

func (pe *PipelineExecution) FailStep(stepFullyQualifiedName string, seID string)

func (*PipelineExecution) FinishStep 

func (pe *PipelineExecution) FinishStep(stepFullyQualifiedName string, seID string)

FinishStep marks the given step execution as started.

func (*PipelineExecution) GetExecutionVariables 

func (pe *PipelineExecution) GetExecutionVariables() (map[string]cty.Value, error)

* 

Arrange the step outputs in a way that it can be used for HCL Expression evaluation

The expressions look something like: step.echo.text_1.text

So we need to arrange the output as such:

"step": {
	"echo": {
		"text_1": {
			"text": "hello world" <-- this is the output from the step
		},
		"text_2": {
			"text": "hello world" <-- this is the output from the step
		},
	},
	"http": {
		"my_http": {
			"response_body": "hello world" <-- this is the output from the step
		},
	},
},
"param": {
	"my_param": "hello world" <-- this is set by the calling function, but maybe we should do it here?
}

func (*PipelineExecution) InitializeStep 

func (pe *PipelineExecution) InitializeStep(stepName string)

InitializeStep initializes the step status for the given step.

func (*PipelineExecution) IsCanceled 

func (pe *PipelineExecution) IsCanceled() bool

IsCanceled returns true if the pipeline has been canceled

func (*PipelineExecution) IsComplete 

func (pe *PipelineExecution) IsComplete() bool

IsComplete returns true if all steps (that have been initialized) are complete.

func (*PipelineExecution) IsFail 

func (pe *PipelineExecution) IsFail() bool

func (*PipelineExecution) IsFinished 

func (pe *PipelineExecution) IsFinished() bool

func (*PipelineExecution) IsFinishing 

func (pe *PipelineExecution) IsFinishing() bool

func (*PipelineExecution) IsPaused 

func (pe *PipelineExecution) IsPaused() bool

IsPaused returns true if the pipeline has been paused

func (*PipelineExecution) IsStepComplete 

func (pe *PipelineExecution) IsStepComplete(stepName string) bool

IsStepComplete returns true if all executions of the step are finished.

func (*PipelineExecution) IsStepFail 

func (pe *PipelineExecution) IsStepFail(stepName string) bool

func (*PipelineExecution) IsStepFinalFailure 

func (pe *PipelineExecution) IsStepFinalFailure(step modconfig.IPipelineStep, ex *Execution) bool

Calculate if this step needs to be retried, or this is the final failure of the step

func (*PipelineExecution) IsStepInitialized 

func (pe *PipelineExecution) IsStepInitialized(stepName string) bool

IsStepInitialized returns true if the step has been initialized.

func (*PipelineExecution) IsStepQueued 

func (pe *PipelineExecution) IsStepQueued(stepName string) bool

TODO: this doesn't work for step execution retry, it assumes that the entire step TODO: must be retried

func (*PipelineExecution) OrderedStepExecutions 

func (pe *PipelineExecution) OrderedStepExecutions(stepName string) []StepExecution

PipelineStepExecutions returns a list of step executions for the given pipeline execution ID and step name.

func (*PipelineExecution) QueueStep 

func (pe *PipelineExecution) QueueStep(stepFullyQualifiedName string, seID string)

QueueStep marks the given step execution as queued.

func (*PipelineExecution) ShouldFail 

func (pe *PipelineExecution) ShouldFail() bool

func (*PipelineExecution) StartStep 

func (pe *PipelineExecution) StartStep(stepFullyQualifiedName string, seID string)

StartStep marks the given step execution as started.

type Snapshot 

type Snapshot struct {
	SchemaVersion string                   `json:"schema_version"`
	StartTime     string                   `json:"start_time"`
	EndTime       string                   `json:"end_time"`
	Layout        SnapshotLayout           `json:"layout"`
	Panels        map[string]SnapshotPanel `json:"panels"`
}

type SnapshotLayout 

type SnapshotLayout struct {
	Name      string           `json:"name"`
	PanelType string           `json:"panel_type"`
	Children  []SnapshotLayout `json:"children,omitempty"`
}

type SnapshotPanel 

type SnapshotPanel struct {
	Dashboard   string                 `json:"dashboard"`
	Name        string                 `json:"name"`
	PanelType   string                 `json:"panel_type"`
	Status      string                 `json:"status"`
	Title       string                 `json:"title,omitempty"`
	DisplayType string                 `json:"display_type,omitempty"`
	Width       int                    `json:"width,omitempty"`
	Data        SnapshotPanelData      `json:"data,omitempty"`
	Properties  map[string]interface{} `json:"properties,omitempty"`
}

type SnapshotPanelData 

type SnapshotPanelData struct {
	Columns []SnapshotPanelDataColumn `json:"columns,omitempty"`
	Rows    []SnapshotPanelDataRow    `json:"rows,omitempty"`
}

type SnapshotPanelDataColumn 

type SnapshotPanelDataColumn struct {
	Name     string `json:"name"`
	DataType string `json:"data_type"`
}

type SnapshotPanelDataRow 

type SnapshotPanelDataRow map[string]interface{}

type StepExecution 

type StepExecution struct {
	// Unique identifier for this step execution
	PipelineExecutionID string `json:"pipeline_execution_id"`
	ID                  string `json:"id"`

	// The name of the step in the pipeline definition
	Name string `json:"name"`

	// The status of the step execution: "started", "finished", "failed", "skipped"
	Status string `json:"status"`

	// Input to the step
	Input modconfig.Input `json:"input"`

	// for_each controls
	StepForEach *modconfig.StepForEach `json:"step_for_each,omitempty"`

	NextStepAction modconfig.NextStepAction `json:"next_step_action,omitempty"`

	// Native/primitive output of the step
	Output *modconfig.Output `json:"output,omitempty"`

	// The output from the Step's output block:
	// output "foo" {
	//    value = <xxx>
	//	}
	//
	StepOutput map[string]interface{} `json:"step_output,omitempty"`
}

StepExecution represents the execution of a single step in a types. A given step definition may be executed multiple times.

func (*StepExecution) Key 

func (se *StepExecution) Key() *string

type StepStatus 

type StepStatus struct {
	// When the step is initializing it doesn't yet have any executions.
	// We track it as initializing until the first execution is queued.
	Initializing bool `json:"initializing"`
	// Step executions that are queued.
	Queued map[string]bool `json:"queued"`
	// Step executions that are started.
	Started map[string]bool `json:"started"`
	// Step executions that are finished.
	Finished map[string]bool `json:"finished"`
	// Step executions that are failed.
	Failed map[string]bool `json:"failed"`
}

This needs to be a map because if we have a for loop, each loop will have a different step execution id

func (*StepStatus) Fail 

func (s *StepStatus) Fail(seID string)

func (*StepStatus) FailCount 

func (s *StepStatus) FailCount() int

func (*StepStatus) Finish 

func (s *StepStatus) Finish(seID string)

Finish marks the given execution as finished.

func (*StepStatus) IsComplete 

func (s *StepStatus) IsComplete() bool

IsComplete returns true if all executions of the step are finished or failed.

func (*StepStatus) IsFail 

func (s *StepStatus) IsFail() bool

IsFail returns true if any executions of the step failed.

func (*StepStatus) Progress 

func (s *StepStatus) Progress() int

Progress returns the percentage of executions of the step that are complete.

func (*StepStatus) Queue 

func (s *StepStatus) Queue(seID string)

Queue marks the given execution as queued.

func (*StepStatus) Start 

func (s *StepStatus) Start(seID string)

Start marks the given execution as started.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.