terraform

Documentation

Overview

Package terraform contains functions and routines for interacting with OpenTofu/Terraform.

Package tfr contains functions and routines for interacting with terraform registries using the Module Registry Protocol documented in the official terraform docs: https://www.terraform.io/docs/internals/module-registry-protocol.html

MAINTAINER'S NOTE: Ideally we would be able to reuse code from Terraform. However, terraform has moved to packaging all its libraries under internal so that you can't use them as a library outside of Terraform. To respect the direction and spirit of the Terraform team, we opted for not doing anything funky to workaround the limitation (like copying those files in here). We also opted to keep this functionality internal to align with the Terraform team's decision to not support client libraries for accessing the Terraform Registry.

Index

• Constants
• func BuildRequestURL(registryDomain string, moduleRegistryBasePath string, modulePath string, ...) (*url.URL, error)
• func GetDownloadURLFromHeader(moduleURL url.URL, terraformGet string) (string, error)
• func GetModuleRegistryURLBasePath(ctx context.Context, logger log.Logger, domain string) (string, error)
• func GetTerraformGetHeader(ctx context.Context, logger log.Logger, url url.URL) (string, error)
• func IsLocalSource(sourceURL *url.URL) bool
• func ModuleVariables(modulePath string) ([]string, []string, error)
• func ParseLog(str string) (msg string, ptrTime *time.Time, ptrLevel *log.Level, err error)
• func ParseLogFunc(msgPrefix string, returnError bool) writer.WriterParseFunc
• func SplitSourceURL(sourceURL *url.URL, logger log.Logger) (*url.URL, string, error)
• func ToSourceURL(source string, workingDir string) (*url.URL, error)
• type MalformedRegistryURLErr
  • func (err MalformedRegistryURLErr) Error() string
• type ModuleDownloadErr
  • func (err ModuleDownloadErr) Error() string
• type RegistryAPIErr
  • func (err RegistryAPIErr) Error() string
• type RegistryGetter
  • func (tfrGetter *RegistryGetter) ClientMode(u *url.URL) (getter.ClientMode, error)
  • func (tfrGetter *RegistryGetter) Context() context.Context
  • func (tfrGetter *RegistryGetter) Get(dstPath string, srcURL *url.URL) error
  • func (tfrGetter *RegistryGetter) GetFile(dst string, src *url.URL) error
  • func (tfrGetter *RegistryGetter) SetClient(client *getter.Client)
• type RegistryServicePath
• type ServiceDiscoveryErr
  • func (err ServiceDiscoveryErr) Error() string
• type Source
  • func NewSource(source string, downloadDir string, workingDir string, logger log.Logger, ...) (*Source, error)
  • func (src Source) EncodeSourceVersion() (string, error)
  • func (src Source) String() string
  • func (src Source) WriteVersionFile() error

Constants

const (
	CommandNameInit           = "init"
	CommandNameInitFromModule = "init-from-module"
	CommandNameImport         = "import"
	CommandNamePlan           = "plan"
	CommandNameApply          = "apply"
	CommandNameDestroy        = "destroy"
	CommandNameValidate       = "validate"
	CommandNameOutput         = "output"
	CommandNameProviders      = "providers"
	CommandNameState          = "state"
	CommandNameLock           = "lock"
	CommandNameGet            = "get"
	CommandNameTaint          = "taint"
	CommandNameUntaint        = "untaint"
	CommandNameConsole        = "console"
	CommandNameForceUnlock    = "force-unlock"
	CommandNameShow           = "show"
	CommandNameVersion        = "version"

	FlagNameDetailedExitCode = "-detailed-exitcode"
	FlagNameHelpLong         = "-help"
	FlagNameHelpShort        = "-h"
	FlagNameVersion          = "-version"
	FlagNameJSON             = "-json"
	FlagNameNoColor          = "-no-color"
	// `apply -destroy` is alias for `destroy`
	FlagNameDestroy = "-destroy"

	// `platform` is a flag used with the `providers lock` command.
	FlagNamePlatform = "-platform"

	EnvNameTFCLIConfigFile                         = "TF_CLI_CONFIG_FILE"
	EnvNameTFPluginCacheDir                        = "TF_PLUGIN_CACHE_DIR"
	EnvNameTFPluginCacheMayBreakDependencyLockFile = "TF_PLUGIN_CACHE_MAY_BREAK_DEPENDENCY_LOCK_FILE"
	EnvNameTFTokenFmt                              = "TF_TOKEN_%s"
	EnvNameTFVarFmt                                = "TF_VAR_%s"

	TerraformLockFile = ".terraform.lock.hcl"

	TerraformPlanFile     = "tfplan.tfplan"
	TerraformPlanJSONFile = "tfplan.json"
)

Functions

func BuildRequestURL

func BuildRequestURL(registryDomain string, moduleRegistryBasePath string, modulePath string, version string) (*url.URL, error)

BuildRequestURL - create url to download module using moduleRegistryBasePath

func GetDownloadURLFromHeader

func GetDownloadURLFromHeader(moduleURL url.URL, terraformGet string) (string, error)

GetDownloadURLFromHeader checks if the content of the X-Terraform-GET header contains the base url and prepends it if not

func GetModuleRegistryURLBasePath

func GetModuleRegistryURLBasePath(ctx context.Context, logger log.Logger, domain string) (string, error)

GetModuleRegistryURLBasePath uses the service discovery protocol (https://www.terraform.io/docs/internals/remote-service-discovery.html) to figure out where the modules are stored. This will return the base path where the modules can be accessed

func GetTerraformGetHeader

func GetTerraformGetHeader(ctx context.Context, logger log.Logger, url url.URL) (string, error)

GetTerraformGetHeader makes an http GET call to the given registry URL and return the contents of location json body or the header X-Terraform-Get. This function will return an error if the response does not contain the header.

func IsLocalSource

func IsLocalSource(sourceURL *url.URL) bool

IsLocalSource returns true if the given URL refers to a path on the local file system

func ModuleVariables

func ModuleVariables(modulePath string) ([]string, []string, error)

ModuleVariables will return all the variables defined in the downloaded terraform modules, taking into account all the generated sources. This function will return the required and optional variables separately.

func ParseLog

func ParseLog(str string) (msg string, ptrTime *time.Time, ptrLevel *log.Level, err error)

func ParseLogFunc

func ParseLogFunc(msgPrefix string, returnError bool) writer.WriterParseFunc

ParseLogFunc wraps `ParseLog` to add msg prefix and bypasses the parse error if `returnError` is false, since returning the error for `log/writer` will cause TG to fall with a `broken pipe` error.

func SplitSourceURL

func SplitSourceURL(sourceURL *url.URL, logger log.Logger) (*url.URL, string, error)

SplitSourceURL splits a source URL into the root repo and the path. The root repo is the part of the URL before the double-slash (//), which typically represents the root of a modules repo (e.g. github.com/foo/infrastructure-modules) and the path is everything after the double slash. If there is no double-slash in the URL, the root repo is the entire sourceUrl and the path is an empty string.

func ToSourceURL

func ToSourceURL(source string, workingDir string) (*url.URL, error)

ToSourceURL converts the given source into a URL struct. This method should be able to handle all source URLs that the terraform init command can handle, parsing local file paths, Git paths, and HTTP URLs correctly.

Types

type MalformedRegistryURLErr

type MalformedRegistryURLErr struct {
	// contains filtered or unexported fields
}

MalformedRegistryURLErr is returned if the Terraform Registry URL passed to the Getter is malformed.

func (MalformedRegistryURLErr) Error

func (err MalformedRegistryURLErr) Error() string

type ModuleDownloadErr

type ModuleDownloadErr struct {
	// contains filtered or unexported fields
}

ModuleDownloadErr is returned if Terragrunt failed to download the module.

func (ModuleDownloadErr) Error

func (err ModuleDownloadErr) Error() string

type RegistryAPIErr

type RegistryAPIErr struct {
	// contains filtered or unexported fields
}

RegistryAPIErr is returned if we get an unsuccessful HTTP return code from the registry.

func (RegistryAPIErr) Error

func (err RegistryAPIErr) Error() string

type RegistryGetter

type RegistryGetter struct {
	TerragruntOptions *options.TerragruntOptions
	// contains filtered or unexported fields
}

RegistryGetter is a Getter (from go-getter) implementation that will download from the terraform module registry. This supports getter URLs encoded in the following manner:

tfr://REGISTRY_DOMAIN/MODULE_PATH?version=VERSION

Where the REGISTRY_DOMAIN is the terraform registry endpoint (e.g., registry.terraform.io), MODULE_PATH is the registry path for the module (e.g., terraform-aws-modules/vpc/aws), and VERSION is the specific version of the module to download (e.g., 2.2.0).

This protocol will use the Module Registry Protocol (documented at https://www.terraform.io/docs/internals/module-registry-protocol.html) to lookup the module source URL and download it.

Authentication to private module registries is handled via environment variables. The authorization API token is expected to be provided to Terragrunt via the TG_TF_REGISTRY_TOKEN environment variable. This token can be any registry API token generated on Terraform Cloud / Enterprise.

MAINTAINER'S NOTE: Ideally we implement the full credential system that terraform uses as part of `terraform login`, but all the relevant packages are internal to the terraform repository, thus making it difficult to use as a library. For now, we keep things simple by supporting providing tokens via env vars and in the future, we can consider implementing functionality to load credentials from terraform. GH issue: https://github.com/gruntwork-io/terragrunt/issues/1771

MAINTAINER'S NOTE: Ideally we can support a shorthand notation that omits the tfr:// protocol to detect that it is referring to a terraform registry, but this requires implementing a complex detector and ensuring it has precedence over the file detector. We deferred the implementation for that to a future release. GH issue: https://github.com/gruntwork-io/terragrunt/issues/1772

func (*RegistryGetter) ClientMode

func (tfrGetter *RegistryGetter) ClientMode(u *url.URL) (getter.ClientMode, error)

ClientMode returns the download mode based on the given URL. Since this getter is designed around the Terraform module registry, we always use Dir mode so that we can download the full Terraform module.

func (*RegistryGetter) Context

func (tfrGetter *RegistryGetter) Context() context.Context

Context returns the go context to use for the underlying fetch routines. This depends on what client is set.

func (*RegistryGetter) Get

func (tfrGetter *RegistryGetter) Get(dstPath string, srcURL *url.URL) error

Get is the main routine to fetch the module contents specified at the given URL and download it to the dstPath. This routine assumes that the srcURL points to the Terraform registry URL, with the Path configured to the module path encoded as `:namespace/:name/:system` as expected by the Terraform registry. Note that the URL query parameter must have the `version` key to specify what version to download.

func (*RegistryGetter) GetFile

func (tfrGetter *RegistryGetter) GetFile(dst string, src *url.URL) error

GetFile is not implemented for the Terraform module registry Getter since the terraform module registry doesn't serve a single file.

func (*RegistryGetter) SetClient

func (tfrGetter *RegistryGetter) SetClient(client *getter.Client)

SetClient allows the getter to know what getter client (different from the underlying HTTP client) to use for progress tracking.

type RegistryServicePath

type RegistryServicePath struct {
	ModulesPath string `json:"modules.v1"`
}

RegistryServicePath is a struct for extracting the modules service path in the Registry.

type ServiceDiscoveryErr

type ServiceDiscoveryErr struct {
	// contains filtered or unexported fields
}

ServiceDiscoveryErr is returned if Terragrunt failed to identify the module API endpoint through the service discovery protocol.

func (ServiceDiscoveryErr) Error

func (err ServiceDiscoveryErr) Error() string

type Source

type Source struct {
	// A canonical version of RawSource, in URL format
	CanonicalSourceURL *url.URL

	// The folder where we should download the source to
	DownloadDir string

	// The folder in DownloadDir that should be used as the working directory for Terraform
	WorkingDir string

	// The path to a file in DownloadDir that stores the version number of the code
	VersionFile string

	// Logger to use for logging
	Logger log.Logger

	// WalkWithSymlinks controls whether to walk symlinks in the downloaded source
	WalkWithSymlinks bool
}

Source represents information about Terraform source code that needs to be downloaded.

func NewSource

func NewSource(source string, downloadDir string, workingDir string, logger log.Logger, walkWithSymlinks bool) (*Source, error)

NewSource takes the given source path and create a Source struct from it, including the folder where the source should be downloaded to. Our goal is to reuse the download folder for the same source URL between Terragrunt runs. Otherwise, for every Terragrunt command, you'd have to wait for Terragrunt to download your Terraform code, download that code's dependencies (terraform get), and configure remote state (terraform remote config), which is very slow.

To maximize reuse, given a working directory w and a source URL s, we download code from S into the folder /T/W/H where:

1. S is the part of s before the double-slash (//). This typically represents the root of the repo (e.g. github.com/foo/infrastructure-modules). We download the entire repo so that relative paths to other files in that repo resolve correctly. If no double-slash is specified, all of s is used.
1. T is the OS temp dir (e.g. /tmp).
2. W is the base 64 encoded sha1 hash of w. This ensures that if you are running Terragrunt concurrently in multiple folders (e.g. during automated tests), then even if those folders are using the same source URL s, they do not overwrite each other.
3. H is the base 64 encoded sha1 of S without its query string. For remote source URLs (e.g. Git URLs), this is based on the assumption that the scheme/host/path of the URL (e.g. git::github.com/foo/bar) identifies the repo, and we always want to download the same repo into the same folder (see the encodeSourceName method). We also assume the version of the module is stored in the query string (e.g. ref=v0.0.3), so we store the base 64 encoded sha1 of the query string in a file called .terragrunt-source-version within /T/W/H.

The downloadTerraformSourceIfNecessary decides when we should download the Terraform code and when not to. It uses the following rules:

1. Always download source URLs pointing to local file paths.
2. Only download source URLs pointing to remote paths if /T/W/H doesn't already exist or, if it does exist, if the version number in /T/W/H/.terragrunt-source-version doesn't match the current version.

func (Source) EncodeSourceVersion

func (src Source) EncodeSourceVersion() (string, error)

EncodeSourceVersion encodes a version number for the given source. When calculating a version number, we take the query string of the source URL, calculate its sha1, and base 64 encode it. For remote URLs (e.g. Git URLs), this is based on the assumption that the scheme/host/path of the URL (e.g. git::github.com/foo/bar) identifies the module name and the query string (e.g. ?ref=v0.0.3) identifies the version. For local file paths, there is no query string, so the same file path (/foo/bar) is always considered the same version. To detect changes the file path will be hashed and returned as version. In case of hash error the default encoded source version will be returned. See also the encodeSourceName and ProcessTerraformSource methods.

func (Source) String

func (src Source) String() string

func (Source) WriteVersionFile

func (src Source) WriteVersionFile() error

WriteVersionFile writes a file into the DownloadDir that contains the version number of this source code. The version number is calculated using the EncodeSourceVersion method.