providercache

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 24, 2024 License: MPL-2.0 Imports: 18 Imported by: 0

Documentation

Overview

Package providercache contains the logic for auto-installing providers from packages obtained elsewhere, and for managing the local directories that serve as global or single-configuration caches of those auto-installed providers.

It builds on the lower-level provider source functionality provided by the internal/getproviders package, adding the additional behaviors around obtaining the discovered providers and placing them in the cache directories for subsequent use.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CachedProvider 

type CachedProvider struct {
	// Provider and Version together identify the specific provider version
	// this cache entry represents.
	Provider addrs.Provider
	Version  getproviders.Version

	// PackageDir is the local filesystem path to the root directory where
	// the provider's distribution archive was unpacked.
	//
	// The path always uses slashes as path separators, even on Windows, so
	// that the results are consistent between platforms. Windows accepts
	// both slashes and backslashes as long as the separators are consistent
	// within a particular path string.
	PackageDir string
}

CachedProvider represents a provider package in a cache directory.

func (*CachedProvider) ExecutableFile 

func (cp *CachedProvider) ExecutableFile() (string, error)

ExecutableFile inspects the cached provider's unpacked package directory for something that looks like it's intended to be the executable file for the plugin.

This is a bit messy and heuristic-y because historically Terraform used the filename itself for local filesystem discovery, allowing some variance in the filenames to capture extra metadata, whereas now we're using the directory structure leading to the executable instead but need to remain compatible with the executable names bundled into existing provider packages.

It will return an error if it can't find a file following the expected convention in the given directory.

If found, the path always uses slashes as path separators, even on Windows, so that the results are consistent between platforms. Windows accepts both slashes and backslashes as long as the separators are consistent within a particular path string.

func (*CachedProvider) Hash 

func (cp *CachedProvider) Hash() (getproviders.Hash, error)

Hash computes a hash of the contents of the package directory associated with the receiving cached provider, using whichever hash algorithm is the current default.

If you need a specific version of hash rather than just whichever one is current default, call that version's corresponding method (e.g. HashV1) directly instead.

func (*CachedProvider) HashV1 

func (cp *CachedProvider) HashV1() (getproviders.Hash, error)

HashV1 computes a hash of the contents of the package directory associated with the receiving cached provider using hash algorithm 1.

The hash covers the paths to files in the directory and the contents of those files. It does not cover other metadata about the files, such as permissions.

This function is named "HashV1" in anticipation of other hashing algorithms being added (in a backward-compatible way) in future. The result from HashV1 always begins with the prefix "h1:" so that callers can distinguish the results of potentially multiple different hash algorithms in future.

func (*CachedProvider) MatchesAnyHash 

func (cp *CachedProvider) MatchesAnyHash(allowed []getproviders.Hash) (bool, error)

MatchesAnyHash returns true if the package on disk matches the given hash, or false otherwise. If it cannot traverse the package directory and read all of the files in it, MatchesAnyHash returns an error.

Unlike the singular MatchesHash, MatchesAnyHash considers unsupported hash formats as successfully non-matching, rather than returning an error.

func (*CachedProvider) MatchesHash 

func (cp *CachedProvider) MatchesHash(want getproviders.Hash) (bool, error)

MatchesHash returns true if the package on disk matches the given hash, or false otherwise. If it cannot traverse the package directory and read all of the files in it, or if the hash is in an unsupported format, MatchesHash returns an error.

MatchesHash may accept hashes in a number of different formats. Over time the set of supported formats may grow and shrink.

func (*CachedProvider) PackageLocation 

func (cp *CachedProvider) PackageLocation() getproviders.PackageLocalDir

PackageLocation returns the package directory given in the PackageDir field as a getproviders.PackageLocation implementation.

Because cached providers are always in the unpacked structure, the result is always of the concrete type getproviders.PackageLocalDir.

type Dir 

type Dir struct {
	// contains filtered or unexported fields
}

Dir represents a single local filesystem directory containing cached provider plugin packages that can be both read from (to find providers to use for operations) and written to (during provider installation).

The contents of a cache directory follow the same naming conventions as a getproviders.FilesystemMirrorSource, except that the packages are always kept in the "unpacked" form (a directory containing the contents of the original distribution archive) so that they are ready for direct execution.

A Dir also pays attention only to packages for the current host platform, silently ignoring any cached packages for other platforms.

Various Dir methods return values that are technically mutable due to the restrictions of the Go typesystem, but callers are not permitted to mutate any part of the returned data structures.

func NewDir 

func NewDir(baseDir string) *Dir

NewDir creates and returns a new Dir object that will read and write provider plugins in the given filesystem directory.

If two instances of Dir are concurrently operating on a particular base directory, or if a Dir base directory is also used as a filesystem mirror source directory, the behavior is undefined.

func NewDirWithPlatform 

func NewDirWithPlatform(baseDir string, platform getproviders.Platform) *Dir

NewDirWithPlatform is a variant of NewDir that allows selecting a specific target platform, rather than taking the current one where this code is running.

This is primarily intended for portable unit testing and not particularly useful in "real" callers.

func (*Dir) AllAvailablePackages 

func (d *Dir) AllAvailablePackages() map[addrs.Provider][]CachedProvider

AllAvailablePackages returns a description of all of the packages already present in the directory. The cache entries are grouped by the provider they relate to and then sorted by version precedence, with highest precedence first.

This function will return an empty result both when the directory is empty and when scanning the directory produces an error.

The caller is forbidden from modifying the returned data structure in any way, even though the Go type system permits it.

func (*Dir) BasePath 

func (d *Dir) BasePath() string

BasePath returns the filesystem path of the base directory of this cache directory.

func (*Dir) InstallPackage 

func (d *Dir) InstallPackage(ctx context.Context, meta getproviders.PackageMeta, allowedHashes []getproviders.Hash) (*getproviders.PackageAuthenticationResult, error)

InstallPackage takes a metadata object describing a package available for installation, retrieves that package, and installs it into the receiving cache directory.

If the allowedHashes set has non-zero length then at least one of the hashes in the set must match the package that "entry" refers to. If none of the hashes match then the returned error message assumes that the hashes came from a lock file.

func (*Dir) LinkFromOtherCache 

func (d *Dir) LinkFromOtherCache(entry *CachedProvider, allowedHashes []getproviders.Hash) error

LinkFromOtherCache takes a CachedProvider value produced from another Dir and links it into the cache represented by the receiver Dir.

This is used to implement tiered caching, where new providers are first populated into a system-wide shared cache and then linked from there into a configuration-specific local cache.

It's invalid to link a CachedProvider from a particular Dir into that same Dir, because that would otherwise potentially replace a real package directory with a circular link back to itself.

If the allowedHashes set has non-zero length then at least one of the hashes in the set must match the package that "entry" refers to. If none of the hashes match then the returned error message assumes that the hashes came from a lock file.

func (*Dir) ProviderLatestVersion 

func (d *Dir) ProviderLatestVersion(provider addrs.Provider) *CachedProvider

ProviderLatestVersion returns the cache entry for the latest version of the requested provider already available in the cache, or nil if there are no versions of that provider available.

func (*Dir) ProviderVersion 

func (d *Dir) ProviderVersion(provider addrs.Provider, version getproviders.Version) *CachedProvider

ProviderVersion returns the cache entry for the requested provider version, or nil if the requested provider version isn't present in the cache.

type InstallMode 

type InstallMode rune

InstallMode customizes the details of how an install operation treats providers that have versions already cached in the target directory. 

const (
	// InstallNewProvidersOnly is an InstallMode that causes the installer
	// to accept any existing version of a requested provider that is already
	// cached as long as it's in the given version sets, without checking
	// whether new versions are available that are also in the given version
	// sets.
	InstallNewProvidersOnly InstallMode = 'N'

	// InstallNewProvidersForce is an InstallMode that follows the same
	// logic as InstallNewProvidersOnly except it does not verify existing
	// checksums but force installs new checksums for all given providers.
	InstallNewProvidersForce InstallMode = 'F'

	// InstallUpgrades is an InstallMode that causes the installer to check
	// all requested providers to see if new versions are available that
	// are also in the given version sets, even if a suitable version of
	// a given provider is already available.
	InstallUpgrades InstallMode = 'U'
)

type Installer 

type Installer struct {
	// contains filtered or unexported fields
}

Installer is the main type in this package, representing a provider installer with a particular configuration-specific cache directory and an optional global cache directory.

func NewInstaller 

func NewInstaller(targetDir *Dir, source getproviders.Source) *Installer

NewInstaller constructs and returns a new installer with the given target directory and provider source.

A newly-created installer does not have a global cache directory configured, but a caller can make a follow-up call to SetGlobalCacheDir to provide one prior to taking any installation actions.

The target directory MUST NOT also be an input consulted by the given source, or the result is undefined.

func (*Installer) Clone 

func (i *Installer) Clone(targetDir *Dir) *Installer

Clone returns a new Installer which has the a new target directory but the same optional global cache directory, the same installation sources, and the same built-in/unmanaged providers. The result can be mutated further using the various setter methods without affecting the original.

func (*Installer) EnsureProviderVersions 

func (i *Installer) EnsureProviderVersions(ctx context.Context, locks *depsfile.Locks, reqs getproviders.Requirements, mode InstallMode) (*depsfile.Locks, error)

EnsureProviderVersions compares the given provider requirements with what is already available in the installer's target directory and then takes appropriate installation actions to ensure that suitable packages are available in the target cache directory.

The given mode modifies how the operation will treat providers that already have acceptable versions available in the target cache directory. See the documentation for InstallMode and the InstallMode values for more information.

The given context can be used to cancel the overall installation operation (causing any operations in progress to fail with an error), and can also include an InstallerEvents value for optional intermediate progress notifications.

If a given InstallerEvents subscribes to notifications about installation failures then those notifications will be redundant with the ones included in the final returned error value so callers should show either one or the other, and not both.

func (*Installer) HasGlobalCacheDir 

func (i *Installer) HasGlobalCacheDir() bool

HasGlobalCacheDir returns true if someone has previously called SetGlobalCacheDir to configure a global cache directory for this installer.

func (*Installer) ProviderSource 

func (i *Installer) ProviderSource() getproviders.Source

ProviderSource returns the getproviders.Source that the installer would use for installing any new providers.

func (*Installer) SetBuiltInProviderTypes 

func (i *Installer) SetBuiltInProviderTypes(types []string)

SetBuiltInProviderTypes tells the receiver to consider the type names in the given slice to be valid as providers in the special special terraform.io/builtin/... namespace that we use for providers that are built in to OpenTofu and thus do not need a separate installation step.

If a caller requests installation of a provider in that namespace, the installer will treat it as a no-op if its name exists in this list, but will produce an error if it does not.

The default, if this method isn't called, is for there to be no valid builtin providers.

Do not modify the buffer under the given slice after passing it to this method.

func (*Installer) SetGlobalCacheDir 

func (i *Installer) SetGlobalCacheDir(cacheDir *Dir)

SetGlobalCacheDir activates a second tier of caching for the receiving installer, with the given directory used as a read-through cache for installation operations that need to retrieve new packages.

The global cache directory for an installer must never be the same as its target directory, and must not be used as one of its provider sources. If these overlap then undefined behavior will result.

func (*Installer) SetGlobalCacheDirMayBreakDependencyLockFile 

func (i *Installer) SetGlobalCacheDirMayBreakDependencyLockFile(mayBreak bool)

SetGlobalCacheDirMayBreakDependencyLockFile activates or deactivates our temporary exception to the rule that the global cache directory can be used only when entries are confirmed by existing entries in the dependency lock file.

If this is set then if we install a provider for the first time from the cache then the dependency lock file will include only the checksum from the package in the global cache, which means the lock file won't be portable to OpenTofu running on another operating system or CPU architecture.

func (*Installer) SetUnmanagedProviderTypes 

func (i *Installer) SetUnmanagedProviderTypes(types map[addrs.Provider]struct{})

SetUnmanagedProviderTypes tells the receiver to consider the providers indicated by the passed addrs.Providers as unmanaged. OpenTofu does not need to control the lifecycle of these providers, and they are assumed to be running already when OpenTofu is started. Because these are essentially processes, not binaries, OpenTofu will not do any work to ensure presence or versioning of these binaries.

type InstallerError 

type InstallerError struct {
	ProviderErrors map[addrs.Provider]error
}

InstallerError is an error type that may be returned (but is not guaranteed) from Installer.EnsureProviderVersions to indicate potentially several separate failed installation outcomes for different providers included in the overall request.

func (InstallerError) Error 

func (err InstallerError) Error() string

type InstallerEvents 

type InstallerEvents struct {
	// The PendingProviders event is called prior to other events to give
	// the recipient prior notice of the full set of distinct provider
	// addresses it can expect to see mentioned in the other events.
	//
	// A recipient driving a UI might, for example, use this to pre-allocate
	// UI space for status reports for all of the providers and then update
	// those positions in-place as other events arrive.
	PendingProviders func(reqs map[addrs.Provider]getproviders.VersionConstraints)

	// ProviderAlreadyInstalled is called for any provider that was included
	// in PendingProviders but requires no further action because a suitable
	// version is already present in the local provider cache directory.
	//
	// This event can also appear after the QueryPackages... series if
	// querying determines that a version already available is the newest
	// available version.
	ProviderAlreadyInstalled func(provider addrs.Provider, selectedVersion getproviders.Version)

	// The BuiltInProvider... family of events describe the outcome for any
	// requested providers that are built in to OpenTofu. Only one of these
	// methods will be called for each such provider, and no other method
	// will be called for them except that they are included in the
	// aggregate PendingProviders map.
	//
	// The "Available" event reports that the requested builtin provider is
	// available in this release of OpenTofu. The "Failure" event reports
	// either that the provider is unavailable or that the request for it
	// is invalid somehow.
	BuiltInProviderAvailable func(provider addrs.Provider)
	BuiltInProviderFailure   func(provider addrs.Provider, err error)

	// The QueryPackages... family of events delimit the operation of querying
	// a provider source for information about available packages matching
	// a particular version constraint, prior to selecting a single version
	// to install.
	//
	// A particular install operation includes only one query per distinct
	// provider, so a caller can use the provider argument as a unique
	// identifier to correlate between successive events.
	//
	// The Begin, Success, and Failure events will each occur only once per
	// distinct provider.
	//
	// The Warning event is unique to the registry source
	QueryPackagesBegin   func(provider addrs.Provider, versionConstraints getproviders.VersionConstraints, locked bool)
	QueryPackagesSuccess func(provider addrs.Provider, selectedVersion getproviders.Version)
	QueryPackagesFailure func(provider addrs.Provider, err error)
	QueryPackagesWarning func(provider addrs.Provider, warn []string)

	// The LinkFromCache... family of events delimit the operation of linking
	// a selected provider package from the system-wide shared cache into the
	// current configuration's local cache.
	//
	// This sequence occurs instead of the FetchPackage... sequence if the
	// QueryPackages... sequence selects a version that is already in the
	// system-wide cache, and thus we will skip fetching it from the
	// originating provider source and take it from the shared cache instead.
	//
	// Linking should, in most cases, be a much faster operation than
	// fetching. However, it could still potentially be slow in some unusual
	// cases like a particularly large source package on a system where symlinks
	// are impossible, or when either of the cache directories are on a network
	// filesystem accessed over a slow link.
	LinkFromCacheBegin   func(provider addrs.Provider, version getproviders.Version, cacheRoot string)
	LinkFromCacheSuccess func(provider addrs.Provider, version getproviders.Version, localDir string)
	LinkFromCacheFailure func(provider addrs.Provider, version getproviders.Version, err error)

	// The FetchPackage... family of events delimit the operation of retrieving
	// a package from a particular source location.
	//
	// A particular install operation includes only one fetch per distinct
	// provider, so a caller can use the provider argument as a unique
	// identifier to correlate between successive events.
	//
	// A particular provider will either notify the LinkFromCache... events
	// or the FetchPackage... events, never both in the same install operation.
	//
	// The Query, Begin, Success, and Failure events will each occur only once
	// per distinct provider.
	FetchPackageMeta    func(provider addrs.Provider, version getproviders.Version) // fetching metadata prior to real download
	FetchPackageBegin   func(provider addrs.Provider, version getproviders.Version, location getproviders.PackageLocation)
	FetchPackageSuccess func(provider addrs.Provider, version getproviders.Version, localDir string, authResult *getproviders.PackageAuthenticationResult)
	FetchPackageFailure func(provider addrs.Provider, version getproviders.Version, err error)

	// The ProvidersLockUpdated event is called whenever the lock file will be
	// updated. It provides the following information:
	//
	//   - `localHashes`: Hashes computed on the local system by analyzing
	//                    files on disk.
	//   - `signedHashes`: Hashes signed by the private key that the origin
	//                     registry claims is the owner of this provider.
	//   - `priorHashes`: Hashes already present in the lock file before we
	//                    made any changes.
	//
	// The final lock file will be updated with a union of all the provided
	// hashes. It not just likely, but expected that there will be duplicates
	// shared between all three collections of hashes i.e. the local hash and
	// remote hashes could already be in the cached hashes.
	//
	// In addition, we place a guarantee that the hash slices will be ordered
	// in the same manner enforced by the lock file within NewProviderLock.
	ProvidersLockUpdated func(provider addrs.Provider, version getproviders.Version, localHashes []getproviders.Hash, signedHashes []getproviders.Hash, priorHashes []getproviders.Hash)

	// The ProvidersFetched event is called after all fetch operations if at
	// least one provider was fetched successfully.
	ProvidersFetched func(authResults map[addrs.Provider]*getproviders.PackageAuthenticationResult)
}

InstallerEvents is a collection of function references that can be associated with an Installer object in order to be notified about various installation lifecycle events during an install operation.

The set of supported events is primarily motivated by allowing ongoing progress reports in the UI of the command running provider installation, and so this only exposes information interesting to display and does not allow the recipient of the events to influence the ongoing process.

Any of the fields may be left as nil to signal that the caller is not interested in the associated event. It's better to leave a field set to nil than to assign a do-nothing function into it because the installer may choose to skip preparing certain temporary data structures if it can see that a particular event is not used.

func (*InstallerEvents) OnContext 

func (e *InstallerEvents) OnContext(ctx context.Context) context.Context

OnContext produces a context with all of the same behaviors as the given context except that it will additionally carry the receiving InstallerEvents.

Passing the resulting context to an installer request will cause the installer to send event notifications via the callbacks inside.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.