engine

package
v3.0.0-...-a5432f4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 13, 2025 License: Apache-2.0 Imports: 45 Imported by: 0

Documentation

Overview

Package engine contains the core logic for the Codeinfra engine, which includes the following:

- Handle resource management and query operations by configuring and spawning goroutines to run the specified operations asynchronously. - Define events and their associated handlers. - Manage journal entries for resource operations. - Manage plugins, including installation, version handling, and loading.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ConvertLocalPolicyPacksToPaths 

func ConvertLocalPolicyPacksToPaths(localPolicyPack []LocalPolicyPack) []string

ConvertLocalPolicyPacksToPaths is a helper function for converting the list of LocalPolicyPacks to a list of paths.

func Destroy 

func Destroy(
	u UpdateInfo,
	ctx *Context,
	opts UpdateOptions,
	dryRun bool,
) (*deploy.Plan, display.ResourceChanges, error)

func EnsurePluginsAreInstalled 

func EnsurePluginsAreInstalled(ctx context.Context, opts *deploymentOptions, d diag.Sink, plugins PluginSet,
	projectPlugins []workspace.ProjectPlugin, reinstall, explicitInstall bool,
) error

EnsurePluginsAreInstalled inspects all plugins in the plugin set and, if any plugins are not currently installed, uses the given backend client to install them. Installations are processed in parallel, though ensurePluginsAreInstalled does not return until all installations are completed.

func GetLocalPolicyPackInfoFromEventName 

func GetLocalPolicyPackInfoFromEventName(name string) (string, string)

GetLocalPolicyPackInfoFromEventName round trips the NameForEvents back into a name/path pair.

func GetRequiredPlugins 

func GetRequiredPlugins(
	host plugin.Host,
	runtime string,
	info plugin.ProgramInfo,
) ([]workspace.PluginSpec, error)

GetRequiredPlugins lists a full set of plugins that will be required by the given program.

func HasChanges 

func HasChanges(changes display.ResourceChanges) bool

HasChanges returns true if there are any non-same changes in the resulting summary.

func Import 

func Import(u UpdateInfo, ctx *Context, opts UpdateOptions, imports []deploy.Import,
	dryRun bool,
) (*deploy.Plan, display.ResourceChanges, error)

func NewProgressReportingCloser 

func NewProgressReportingCloser(
	events eventEmitter,
	typ ProgressType,
	id string,
	message string,
	size int64,
	reportingInterval time.Duration,
	closer io.ReadCloser,
) io.ReadCloser

Creates a new ReadCloser that reports ProgressEvents as bytes are read and when it is closed. A Done ProgressEvent will only be reported once, on the first call to Close(). Subsequent calls to Close() will be forwarded to the underlying ReadCloser, but will not yield duplicate ProgressEvents.

func ProjectInfoContext 

func ProjectInfoContext(projinfo *Projinfo, host plugin.Host,
	diag, statusDiag diag.Sink, debugging plugin.DebugEventEmitter, disableProviderPreview bool,
	tracingSpan opentracing.Span, config map[config.Key]string,
) (string, string, *plugin.Context, error)

ProjectInfoContext returns information about the current project, including its pwd, main, and plugin context.

func Query 

func Query(ctx *Context, q QueryInfo, opts UpdateOptions) error

func Refresh 

func Refresh(
	u UpdateInfo,
	ctx *Context,
	opts UpdateOptions,
	dryRun bool,
) (*deploy.Plan, display.ResourceChanges, error)

func ShouldRecordReadStep 

func ShouldRecordReadStep(step deploy.Step) bool

func TranslateDetailedDiff 

func TranslateDetailedDiff(step *StepEventMetadata, refresh bool) *resource.ObjectDiff

TranslateDetailedDiff converts the detailed diff stored in the step event into an ObjectDiff that is appropriate for display.

func Update 

func Update(u UpdateInfo, ctx *Context, opts UpdateOptions, dryRun bool) (
	*deploy.Plan, display.ResourceChanges, error,
)

Types

type CombinedManager 

type CombinedManager struct {
	Managers []SnapshotManager
}

CombinedManager combines multiple SnapshotManagers into one, it simply forwards on each call to every manager.

func (*CombinedManager) BeginMutation 

func (c *CombinedManager) BeginMutation(step deploy.Step) (SnapshotMutation, error)

func (*CombinedManager) Close 

func (c *CombinedManager) Close() error

func (*CombinedManager) RegisterResourceOutputs 

func (c *CombinedManager) RegisterResourceOutputs(step deploy.Step) error

type CombinedMutation 

type CombinedMutation struct {
	Mutations []SnapshotMutation
}

func (*CombinedMutation) End 

func (c *CombinedMutation) End(step deploy.Step, success bool) error

type Context 

type Context struct {
	Cancel          *cancel.Context
	Events          chan<- Event
	SnapshotManager SnapshotManager
	BackendClient   deploy.BackendClient
	ParentSpan      opentracing.SpanContext
}

Context provides cancellation, termination, and eventing options for an engine operation. It also provides a way for the engine to persist snapshots, using the `SnapshotManager`.

type DecryptError 

type DecryptError struct {
	Key config.Key // The configuration key whose value couldn't be decrypted
	Err error      // The error that occurred while decrypting
}

DecryptError is the type of errors that arise when the engine can't decrypt a configuration key. The most common reason why this happens is that this key is being decrypted in a stack that's not the same one that encrypted it.

func AsDecryptError 

func AsDecryptError(err error) (*DecryptError, bool)

Returns a tuple in which the second element is true if and only if any error in the given error's tree is a DecryptError. In that case, the first element will be the first DecryptError in the tree. In the event that there is no such DecryptError, the first element will be nil.

func (DecryptError) Error 

func (d DecryptError) Error() string

type DiagEventPayload 

type DiagEventPayload struct {
	URN       resource.URN
	Prefix    string
	Message   string
	Color     colors.Colorization
	Severity  diag.Severity
	StreamID  int32
	Ephemeral bool
}

DiagEventPayload is the payload for an event with type `diag`

type Event 

type Event struct {
	Type EventType
	// contains filtered or unexported fields
}

Event represents an event generated by the engine during an operation. The underlying type for the `Payload` field will differ depending on the value of the `Type` field

func NewCancelEvent 

func NewCancelEvent() Event

func NewEvent 

func NewEvent[T EventPayload](payload T) Event

func (Event) Ephemeral 

func (e Event) Ephemeral() bool

Returns true if and only if this is an ephemeral event that should not be persisted. Ephemeral events are intended for display and reporting purposes only (e.g. progress).

func (Event) Internal 

func (e Event) Internal() bool

Returns true if this is a ResourcePreEvent or ResourceOutputsEvent with the internal flag set.

func (Event) Payload 

func (e Event) Payload() interface{}

type EventPayload 

type EventPayload interface {
	StdoutEventPayload | DiagEventPayload | PreludeEventPayload | SummaryEventPayload |
		ResourcePreEventPayload | ResourceOutputsEventPayload | ResourceOperationFailedPayload |
		PolicyViolationEventPayload | PolicyRemediationEventPayload | PolicyLoadEventPayload | StartDebuggingEventPayload |
		ProgressEventPayload
}

type EventType 

type EventType string

EventType is the kind of event being emitted. 

const (
	CancelEvent             EventType = "cancel"
	StdoutColorEvent        EventType = "stdoutcolor"
	DiagEvent               EventType = "diag"
	PreludeEvent            EventType = "prelude"
	SummaryEvent            EventType = "summary"
	ResourcePreEvent        EventType = "resource-pre"
	ResourceOutputsEvent    EventType = "resource-outputs"
	ResourceOperationFailed EventType = "resource-operationfailed"
	PolicyViolationEvent    EventType = "policy-violation"
	PolicyRemediationEvent  EventType = "policy-remediation"
	PolicyLoadEvent         EventType = "policy-load"
	StartDebuggingEvent     EventType = "debugging-start"
	ProgressEvent           EventType = "progress"
)

type Journal 

type Journal struct {
	// contains filtered or unexported fields
}

func NewJournal 

func NewJournal() *Journal

func (*Journal) BeginMutation 

func (j *Journal) BeginMutation(step deploy.Step) (SnapshotMutation, error)

func (*Journal) Close 

func (j *Journal) Close() error

func (*Journal) End 

func (j *Journal) End(step deploy.Step, success bool) error

func (*Journal) Entries 

func (j *Journal) Entries() JournalEntries

func (*Journal) RecordPlugin 

func (j *Journal) RecordPlugin(plugin workspace.PluginInfo) error

func (*Journal) RegisterResourceOutputs 

func (j *Journal) RegisterResourceOutputs(step deploy.Step) error

func (*Journal) Snap 

func (j *Journal) Snap(base *deploy.Snapshot) (*deploy.Snapshot, error)

type JournalEntries 

type JournalEntries []JournalEntry

func (JournalEntries) Snap 

func (entries JournalEntries) Snap(base *deploy.Snapshot) (*deploy.Snapshot, error)

type JournalEntry 

type JournalEntry struct {
	Kind JournalEntryKind
	Step deploy.Step
}

type JournalEntryKind 

type JournalEntryKind int
const (
	JournalEntryBegin   JournalEntryKind = 0
	JournalEntrySuccess JournalEntryKind = 1
	JournalEntryFailure JournalEntryKind = 2
	JournalEntryOutputs JournalEntryKind = 4
)

type LocalPolicyPack 

type LocalPolicyPack struct {
	// Name provides the user-specified name of the Policy Pack.
	Name string
	// Version of the local Policy Pack.
	Version string
	// Path of the local Policy Pack.
	Path string
	// Path of the local Policy Pack's JSON config file.
	Config string
}

LocalPolicyPack represents a set of local Policy Packs to apply during an update.

func MakeLocalPolicyPacks 

func MakeLocalPolicyPacks(localPaths []string, configPaths []string) []LocalPolicyPack

MakeLocalPolicyPacks is a helper function for converting the list of local Policy Pack paths to list of LocalPolicyPack. The name of the Local Policy Pack is not set since we must load up the Policy Pack plugin to determine its name.

func (LocalPolicyPack) NameForEvents 

func (pack LocalPolicyPack) NameForEvents() string

NameForEvents encodes a local policy pack's information in a single string which can be used for engine events. It is done this way so we don't lose path information.

type PackageSet 

type PackageSet map[string]workspace.PackageDescriptor

PackageSet represents a set of packages.

func NewPackageSet 

func NewPackageSet(pkgs ...workspace.PackageDescriptor) PackageSet

NewPackageSet creates a new PackageSet from the specified PackageDescriptors.

func (PackageSet) Add 

func (p PackageSet) Add(pkg workspace.PackageDescriptor)

Add adds a package to this package set.

func (PackageSet) ToPluginSet 

func (p PackageSet) ToPluginSet() PluginSet

ToPluginSet converts this PackageSet to a PluginSet by discarding all parameterization information.

func (PackageSet) Union 

func (p PackageSet) Union(other PackageSet) PackageSet

Union returns the union of this PackageSet with another PackageSet.

func (PackageSet) UpdatesTo 

func (p PackageSet) UpdatesTo(old PackageSet) []PackageUpdate

UpdatesTo returns a list of PackageUpdates that represent the updates to the argument PackageSet present in this PackageSet. For instance, if the argument contains a package P at version 3, and this PackageSet contains the same package P (as identified by name and kind) at version 5, this method will return an update where the Old field contains the version 3 instance from the argument and the New field contains the version 5 instance from this PackageSet. This also considers parameterization information, so a parameterized package P at version 3 will be considered different from a parameterized package P at version 5 even if the base plugin is the same.

type PackageUpdate 

type PackageUpdate struct {
	// The old package version.
	Old workspace.PackageDescriptor
	// The new package version.
	New workspace.PackageDescriptor
}

A PackageUpdate represents an update from one version of a package to another.

type PluginSet 

type PluginSet map[string]workspace.PluginSpec

PluginSet represents a set of plugins.

func NewPluginSet 

func NewPluginSet(plugins ...workspace.PluginSpec) PluginSet

NewPluginSet creates a new PluginSet from the specified PluginSpecs.

func (PluginSet) Add 

func (p PluginSet) Add(plug workspace.PluginSpec)

Add adds a plugin to this plugin set.

func (PluginSet) Deduplicate 

func (p PluginSet) Deduplicate() PluginSet

Removes less-specific entries.

For example, the plugin aws would be removed if there was an already existing plugin aws-5.4.0.

func (PluginSet) Values 

func (p PluginSet) Values() []workspace.PluginSpec

Values returns a slice of all of the plugins contained within this set.

type PolicyLoadEventPayload 

type PolicyLoadEventPayload struct{}

PolicyLoadEventPayload is the payload for an event with type `policy-load`.

type PolicyPackInfo 

type PolicyPackInfo struct {
	Proj *workspace.PolicyPackProject
	Root string
}

func (*PolicyPackInfo) GetPwdMain 

func (projinfo *PolicyPackInfo) GetPwdMain() (string, string, error)

GetPwdMain returns the working directory and main entrypoint to use for this package.

type PolicyRemediationEventPayload 

type PolicyRemediationEventPayload struct {
	ResourceURN       resource.URN
	Color             colors.Colorization
	PolicyName        string
	PolicyPackName    string
	PolicyPackVersion string
	Before            resource.PropertyMap
	After             resource.PropertyMap
}

PolicyRemediationEventPayload is the payload for an event with type `policy-remediation`.

type PolicyViolationEventPayload 

type PolicyViolationEventPayload struct {
	ResourceURN       resource.URN
	Message           string
	Color             colors.Colorization
	PolicyName        string
	PolicyPackName    string
	PolicyPackVersion string
	EnforcementLevel  apitype.EnforcementLevel
	Prefix            string
}

PolicyViolationEventPayload is the payload for an event with type `policy-violation`.

type PreludeEventPayload 

type PreludeEventPayload struct {
	IsPreview bool              // true if this prelude is for a plan operation
	Config    map[string]string // the keys and values for config. For encrypted config, the values may be blinded
}

type ProgressEventPayload 

type ProgressEventPayload struct {
	// The type of process (e.g. plugin download, plugin install).
	Type ProgressType
	// A unique identifier for the process.
	ID string
	// A message accompanying the process.
	Message string
	// The number of items completed so far (e.g. bytes received, items installed,
	// etc.)
	Completed int64
	// The total number of items that must be completed.
	Total int64
	// True if and only if the process has completed.
	Done bool
}

ProgressEventPayload is the payload for an event with type `progress`. This payload reports on the progress of a potentially long-running process being managed by the engine (e.g. a plugin download, or a plugin installation).

type ProgressType 

type ProgressType string

ProgressType is the type of download occurring. 

const (
	// PluginDownload represents a download of a plugin.
	PluginDownload ProgressType = "plugin-download"
	// PluginInstall represents the installation of a plugin.
	PluginInstall ProgressType = "plugin-install"
)

type Projinfo 

type Projinfo struct {
	Proj *workspace.Project
	Root string
}

func (*Projinfo) GetPwdMain 

func (projinfo *Projinfo) GetPwdMain() (string, string, error)

GetPwdMain returns the working directory and main entrypoint to use for this package.

type QueryInfo 

type QueryInfo interface {
	// GetRoot returns the root directory for this update. This defines the scope for any filesystem resources
	// accessed by this update.
	GetRoot() string
	// GetProject returns information about the project associated with this update. This includes information such as
	// the runtime that will be used to execute the Codeinfra program and the program's relative working directory.
	GetProject() *workspace.Project
}

QueryInfo handles information common to query operations (list, watch).

type QueryOptions 

type QueryOptions struct {
	Events     eventEmitter // the channel to write events from the engine to.
	Diag       diag.Sink    // the sink to use for diag'ing.
	StatusDiag diag.Sink    // the sink to use for diag'ing status messages.
	// contains filtered or unexported fields
}

type RequiredPolicy 

type RequiredPolicy interface {
	// Name provides the user-specified name of the PolicyPack.
	Name() string
	// Version of the PolicyPack.
	Version() string
	// Install will install the PolicyPack locally, returning the path it was installed to.
	Install(ctx context.Context) (string, error)
	// Config returns the PolicyPack's configuration.
	Config() map[string]*json.RawMessage
}

RequiredPolicy represents a set of policies to apply during an update.

type ResourceOperationFailedPayload 

type ResourceOperationFailedPayload struct {
	Metadata StepEventMetadata
	Status   resource.Status
	Steps    int
}

type ResourceOutputsEventPayload 

type ResourceOutputsEventPayload struct {
	Metadata StepEventMetadata
	Planning bool
	Debug    bool
	// Internal is set for events that should not be shown to a user but are expected to be used in other parts of the
	// Codeinfra system.
	Internal bool
}

type ResourcePreEventPayload 

type ResourcePreEventPayload struct {
	Metadata StepEventMetadata
	Planning bool
	Debug    bool
	// Internal is set for events that should not be shown to a user but are expected to be used in other parts of the
	// Codeinfra system.
	Internal bool
}

type SnapshotManager 

type SnapshotManager interface {
	io.Closer

	// BeginMutation signals to the SnapshotManager that the planner intends to mutate the global
	// snapshot. It provides the step that it intends to execute. Based on that step, BeginMutation
	// will record this intent in the global snapshot and return a `SnapshotMutation` that, when ended,
	// will complete the transaction.
	BeginMutation(step deploy.Step) (SnapshotMutation, error)

	// RegisterResourceOutputs registers the set of resource outputs generated by performing the
	// given step. These outputs are persisted in the snapshot.
	RegisterResourceOutputs(step deploy.Step) error
}

SnapshotManager manages an in-memory resource graph.

type SnapshotMutation 

type SnapshotMutation interface {
	// End terminates the transaction and commits the results to the snapshot, returning an error if this
	// failed to complete.
	End(step deploy.Step, successful bool) error
}

SnapshotMutation represents an outstanding mutation that is yet to be completed. When the engine completes a mutation, it must call `End` in order to record the successful completion of the mutation.

type StartDebuggingEventPayload 

type StartDebuggingEventPayload struct {
	Config map[string]interface{} // the debug configuration (language-specific, see Debug Adapter Protocol)
}

StartDebuggingEventPayload is the payload for an event of type `debugging-start`

type StdoutEventPayload 

type StdoutEventPayload struct {
	Message string
	Color   colors.Colorization
}

type StepEventMetadata 

type StepEventMetadata struct {
	Op           display.StepOp                 // the operation performed by this step.
	URN          resource.URN                   // the resource URN (for before and after).
	Type         tokens.Type                    // the type affected by this step.
	Old          *StepEventStateMetadata        // the state of the resource before performing this step.
	New          *StepEventStateMetadata        // the state of the resource after performing this step.
	Res          *StepEventStateMetadata        // the latest state for the resource that is known (worst case, old).
	Keys         []resource.PropertyKey         // the keys causing replacement (only for CreateStep and ReplaceStep).
	Diffs        []resource.PropertyKey         // the keys causing diffs
	DetailedDiff map[string]plugin.PropertyDiff // the rich, structured diff
	Logical      bool                           // true if this step represents a logical operation in the program.
	Provider     string                         // the provider that performed this step.
}

StepEventMetadata contains the metadata associated with a step the engine is performing.

type StepEventStateMetadata 

type StepEventStateMetadata struct {
	// State contains the raw, complete state, for this resource.
	State *resource.State
	// the resource's type.
	Type tokens.Type
	// the resource's object urn, a human-friendly, unique name for the resource.
	URN resource.URN
	// true if the resource is custom, managed by a plugin.
	Custom bool
	// true if this resource is pending deletion due to a replacement.
	Delete bool
	// the resource's unique ID, assigned by the resource provider (or blank if none/uncreated).
	ID resource.ID
	// an optional parent URN that this resource belongs to.
	Parent resource.URN
	// true to "protect" this resource (protected resources cannot be deleted).
	Protect bool
	// RetainOnDelete is true if the resource is not physically deleted when it is logically deleted.
	RetainOnDelete bool `json:"retainOnDelete"`
	// the resource's input properties (as specified by the program). Note: because this will cross
	// over rpc boundaries it will be slightly different than the Inputs found in resource_state.
	// Specifically, secrets will have been filtered out, and large values (like assets) will be
	// have a simple hash-based representation.  This allows clients to display this information
	// properly, without worrying about leaking sensitive data, and without having to transmit huge
	// amounts of data.
	Inputs resource.PropertyMap
	// the resource's complete output state (as returned by the resource provider).  See "Inputs"
	// for additional details about how data will be transformed before going into this map.
	Outputs resource.PropertyMap
	// the resource's provider reference
	Provider string
	// InitErrors is the set of errors encountered in the process of initializing resource (i.e.,
	// during create or update).
	InitErrors []string
}

StepEventStateMetadata contains detailed metadata about a resource's state pertaining to a given step.

type SummaryEventPayload 

type SummaryEventPayload struct {
	IsPreview       bool                    // true if this summary is for a plan operation
	MaybeCorrupt    bool                    // true if one or more resources may be corrupt
	Duration        time.Duration           // the duration of the entire update operation (zero values for previews)
	ResourceChanges display.ResourceChanges // count of changed resources, useful for reporting
	PolicyPacks     map[string]string       // {policy-pack: version} for each policy pack applied
}

type UpdateInfo 

type UpdateInfo interface {
	// GetRoot returns the root directory for this update. This defines the scope for any filesystem resources
	// accessed by this update.
	GetRoot() string
	// GetProject returns information about the project associated with this update. This includes information such as
	// the runtime that will be used to execute the Codeinfra program and the program's relative working directory.
	GetProject() *workspace.Project
	// GetTarget returns information about the target of this update. This includes the name of the stack being
	// updated, the configuration values associated with the target and the target's latest snapshot.
	GetTarget() *deploy.Target
}

UpdateInfo handles information common to resource operations (update, preview, destroy, import, refresh).

type UpdateOptions 

type UpdateOptions struct {
	// LocalPolicyPacks contains an optional set of policy packs to run as part of this deployment.
	LocalPolicyPacks []LocalPolicyPack

	// RequiredPolicies is the set of policies that are required to run as part of the update.
	RequiredPolicies []RequiredPolicy

	// the degree of parallelism for resource operations (<=1 for serial).
	Parallel int32

	// true if debugging output it enabled
	Debug bool

	// true if the plan should refresh before executing.
	Refresh bool

	// Specific resources to replace during an update operation.
	ReplaceTargets deploy.UrnTargets

	// Specific resources to update during a deployment.
	Targets deploy.UrnTargets

	// true if we're allowing dependent targets to change, even if not specified in one of the above
	// XXXTargets lists.
	TargetDependents bool

	// true if the engine should use legacy diffing behavior during an update.
	UseLegacyDiff bool

	// true if the engine should use legacy refresh diffing behavior and report
	// only output changes, as opposed to computing diffs against desired state.
	UseLegacyRefreshDiff bool

	// true if the engine should disable provider previews.
	DisableProviderPreview bool

	// true if the engine should disable resource reference support.
	DisableResourceReferences bool

	// true if the engine should disable output value support.
	DisableOutputValues bool

	// the plugin host to use for this update
	Host plugin.Host

	// The plan to use for the update, if any.
	Plan *deploy.Plan

	// GeneratePlan when true cause plans to be generated, we skip this if we know their not needed (e.g. during up)
	GeneratePlan bool

	// Experimental is true if the engine is in experimental mode (i.e. CODEINFRA_EXPERIMENTAL was set)
	Experimental bool

	// ContinueOnError is true if the engine should continue processing resources after an error is encountered.
	ContinueOnError bool

	// AttachDebugger to launch the language host in debug mode.
	AttachDebugger bool

	// Autonamer can resolve user's preference for custom autonaming options for a given resource.
	Autonamer autonaming.Autonamer
}

UpdateOptions contains all the settings for customizing how an update (deploy, preview, or destroy) is performed.

This structure is embedded in another which uses some of the unexported fields, which trips up the `structcheck` linter.

Source Files

View all Source files

Directories

Path Synopsis
lifecycletest
framework
fuzzing

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.