hcl_interpreter

package
v0.24.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 1, 2023 License: Apache-2.0 Imports: 20 Imported by: 1

README

hcl_interpreter

This package supports loading HCL configurations. Since it is a bit more complex than the other loaders, it has it's own package. What follows is a brief but complete overview of how it works.

It tries to mimic terraform behavior as much as possible by using terraform as a library. Since some packages inside terraform are in an /internal package, we vendor in terraform and rename the packages to make them accessible from our code. This is automated in the top-level Makefile.

names.go

names.go holds some utilities to work with names, for example parsing them from strings and rendering them back to strings. A name consists of two parts:

  • A module part (e.g. module.child1.). This is empty for the root module.
  • A local part (e.g. aws_s3_bucket.my_bucket or var.my_var).

Importantly, there are a number of AsXYZ() methods, for example AsModuleOutput(). These manipulate the names to point to their "real" location: for example, if you use module.child1.my_output in the root module, the "real" location is the output inside the child module, so module.child1.outputs.my_output. These methods form a big part of the logic and having it here allows us to keep the code in other files relatively clean.

moduletree.go

moduletree.go is responsible for parsing a directory of terraform files, including children (submodules). We end up with a hierarchical structure:

  • root module
    • child 1
    • child 2
      • grandchild

We can "walk" this tree using a visitor pattern and visit every term.

A term can be a simple expression or a block with attributes and sub-blocks. Each resource forms a term, and so does each other "entity" in the input, like a a local variable or a module output. For more details, see term.go.

For example:

  • aws_security_group.invalid_sg_1
  • module.child1.output.bucket

Because we pass in both the full name (see above) as well as the term, a visitor can store these in a flat rather than hierarchical map, which is more convenient to work with in Go.

This file uses an additional file moduleregister.go to deal with the locations of remote (downloaded) terraform modules.

valtree.go

Once expressions are evaluated, they become values of the type cty.Value. This module has a number of utilities to construct and merge Values.

phantom_attrs.go

In IaC files, it is common to depend on values which are not filled in:

resource "aws_kms_key" "rds-db-instance-kms" {
  deletion_window_in_days = 10
}

resource "aws_db_instance" "default" {
  kms_key_id = "${aws_kms_key.rds-db-instance-kms.arn}"
  ...
}

rds-db-instance-kms does not have an .arn attribute, but evaluating kms_key_id needs one. We solve this by collecting all references to unknown attributes in the expressions, and setting these as "phantom attributes" on the resources. They are not included in the output.

hcl_interpreter.go

Finally, using these foundations, hcl_interpreter.go implements the main logic. This happens in the following imperative steps:

  1. We use the visitor from moduletree.go to obtain a full list of everything in the module and its children.

  2. For every term, we can compute its dependencies (possibly renaming some using the logic in names.go). This gives us enough info to run a topological sort; which tells us the exact order in which all terms should be evaluated.

  3. We run through and evaluate each expression.

    • We have a single big cty.Value that holds everything we have evaluated so far per module.

    • Before evaluating, we add extra dependencies to this cty.Value scope based on the code in dependencies() and prepareVariables(). This is used to e.g. get outputs from other modules.

    • After evaluating, we merge the result back into the cty.Value for that module.

  4. We convert the individual cty.Values for the resources into the resources view (this involves only some minor bookkeeping like adding the id and _provider fields).

Documentation

Overview

Implements the `Data` interface. Doesn't really do anything.

Look in `/pkg/hcl_interpreter/README.md` for an explanation of how this works.

This module contains utilities for parsing and traversing everything in a configuration tree.

Look in `/pkg/hcl_interpreter/README.md` for an explanation of how this works.

This module contains utilities for parsing and traversing everything in a configuration tree.

cty.Value utilities

Index

Constants

This section is empty.

Variables

View Source
var EmptyModuleName = []string{}

Functions

func LocalNameToString

func LocalNameToString(name LocalName) string

func LookupVal added in v0.15.0

func LookupVal(tree cty.Value, name LocalName) cty.Value

Look up a given subtree, returns Null if not found

func MergeVal added in v0.15.0

func MergeVal(left cty.Value, right cty.Value) cty.Value

Merges two Values with a preference given to the right object.

func ModuleNameToString

func ModuleNameToString(moduleName ModuleName) string

func NestVal added in v0.15.0

func NestVal(prefix []string, val cty.Value) cty.Value

func TfFilePathJoin

func TfFilePathJoin(leading, trailing string) string

TfFilePathJoin is like `filepath.Join` but avoids cleaning the path. This allows to get unique paths for submodules including a parent module, e.g.:

.
examples/mssql/../../
examples/complete/../../

func TraversalToString

func TraversalToString(traversal hcl.Traversal) string

func ValToVariables added in v0.15.0

func ValToVariables(tree cty.Value) map[string]cty.Value

Some HCL functions require it to be a map. Returns an empty map if we have anything but an object at the root.

func ValueToInt

func ValueToInt(val cty.Value) *int

func ValueToInterface

func ValueToInterface(val cty.Value) (interface{}, []error)

func ValueToString

func ValueToString(val cty.Value) *string

Types

type Analysis

type Analysis struct {
	Fs afero.Fs

	// Module metadata
	Modules map[string]*ModuleMeta

	// Resource metadata
	Resources map[string]*ResourceMeta

	// Terms in the evaluation tree.
	Terms *TermTree
	// contains filtered or unexported fields
}

func AnalyzeModuleTree

func AnalyzeModuleTree(mtree *ModuleTree) *Analysis

func (*Analysis) VisitModule

func (v *Analysis) VisitModule(name ModuleName, meta *ModuleMeta)

func (*Analysis) VisitResource added in v0.15.0

func (v *Analysis) VisitResource(name FullName, resource *ResourceMeta)

func (*Analysis) VisitTerm added in v0.15.0

func (v *Analysis) VisitTerm(name FullName, term Term)

type Data

type Data struct {
}

func (*Data) GetCountAttr

func (*Data) GetForEachAttr

func (*Data) GetInputVariable

func (c *Data) GetInputVariable(v addrs.InputVariable, s tfdiags.SourceRange) (cty.Value, tfdiags.Diagnostics)

func (*Data) GetLocalValue

func (*Data) GetModule

func (*Data) GetPathAttr

func (c *Data) GetPathAttr(attr addrs.PathAttr, diags tfdiags.SourceRange) (cty.Value, tfdiags.Diagnostics)

func (*Data) GetResource

func (*Data) GetTerraformAttr

func (*Data) StaticValidateReferences

func (c *Data) StaticValidateReferences(refs []*addrs.Reference, self addrs.Referenceable) tfdiags.Diagnostics

type Evaluation

type Evaluation struct {
	Analysis *Analysis
	Modules  map[string]cty.Value
	// contains filtered or unexported fields
}

func EvaluateAnalysis

func EvaluateAnalysis(analysis *Analysis) (*Evaluation, error)

func (*Evaluation) Errors

func (e *Evaluation) Errors() []error

Errors returns the non-fatal errors encountered during evaluation

func (*Evaluation) Location

func (v *Evaluation) Location(
	resourceId string,
	path []interface{},
) []hcl.Range

func (*Evaluation) Resources

func (v *Evaluation) Resources() []Resource

type EvaluationError added in v0.22.0

type EvaluationError struct {
	Diags hcl.Diagnostics
}

func (EvaluationError) Error added in v0.22.0

func (err EvaluationError) Error() string

type FullName

type FullName struct {
	Module ModuleName
	Local  LocalName
}

func ArrayToFullName

func ArrayToFullName(parts []string) FullName

func EmptyFullName

func EmptyFullName(module ModuleName) FullName

func ProviderConfigName

func ProviderConfigName(module ModuleName, providerName string) FullName

func StringToFullName

func StringToFullName(name string) (*FullName, error)

func (FullName) Add added in v0.15.0

func (name FullName) Add(p string) FullName

func (FullName) AsModuleInput

func (name FullName) AsModuleInput() *FullName

Parses "module.child.var.foo" into "input.child.foo"

func (FullName) AsModuleOutput

func (name FullName) AsModuleOutput() *FullName

Parses the use of an output (e.g. "module.child.x") to the fully expanded output name (e.g. module.child.output.x")

func (FullName) AsResourceName

func (name FullName) AsResourceName() (*FullName, LocalName)

func (FullName) AsVariable

func (name FullName) AsVariable() (*FullName, *FullName, LocalName)

Parses "var.my_var.key" into "variable.my_var", "var.my_var" and "key".

func (FullName) ToString

func (name FullName) ToString() string

type LocalName

type LocalName = []string

func TraversalToLocalName

func TraversalToLocalName(traversal hcl.Traversal) (LocalName, error)

TODO: Refactor to TraversalToName?

type MissingRemoteSubmodulesError added in v0.22.0

type MissingRemoteSubmodulesError struct {
	Dir            string
	MissingModules []string
}

func (MissingRemoteSubmodulesError) Error added in v0.22.0

type MissingTermError added in v0.22.0

type MissingTermError struct {
	Term string
}

func (MissingTermError) Error added in v0.22.0

func (err MissingTermError) Error() string

type ModuleMeta

type ModuleMeta struct {
	Dir                  string
	Recurse              bool
	Filepaths            []string
	MissingRemoteModules []string
	Location             *hcl.Range
}

type ModuleName

type ModuleName = []string

type ModuleTree

type ModuleTree struct {
	// contains filtered or unexported fields
}

We load the entire tree of submodules in one pass.

func ParseDirectory

func ParseDirectory(
	moduleRegister *TerraformModuleRegister,
	parserFs afero.Fs,
	dir string,
	varFiles []string,
) (*ModuleTree, error)

func ParseFiles

func ParseFiles(
	moduleRegister *TerraformModuleRegister,
	parserFs afero.Fs,
	recurse bool,
	dir string,
	filepaths []string,
	varfiles []string,
) (*ModuleTree, error)

func (*ModuleTree) Errors

func (mtree *ModuleTree) Errors() []error

func (*ModuleTree) FilePath

func (mtree *ModuleTree) FilePath() string

func (*ModuleTree) LoadedFiles

func (mtree *ModuleTree) LoadedFiles() []string

func (*ModuleTree) Walk

func (mtree *ModuleTree) Walk(v Visitor)

type Resource added in v0.15.0

type Resource struct {
	Meta  *ResourceMeta
	Model models.ResourceState
}

type ResourceMeta

type ResourceMeta struct {
	Data                      bool
	Type                      string
	ProviderType              string
	ProviderName              string
	ProviderVersionConstraint string
	Multiple                  bool
	Location                  hcl.Range
	Body                      hcl.Body // For source code locations only.
}

type SubmoduleLoadingError added in v0.22.0

type SubmoduleLoadingError struct {
	Module string
	Err    error
}

func (SubmoduleLoadingError) Error added in v0.22.0

func (err SubmoduleLoadingError) Error() string

type Term added in v0.15.0

type Term struct {
	// contains filtered or unexported fields
}

func TermFromBody added in v0.15.0

func TermFromBody(body hcl.Body) Term

func TermFromExpr added in v0.15.0

func TermFromExpr(expr hcl.Expression) Term

func (Term) Attributes added in v0.15.0

func (t Term) Attributes() map[string]Term

Attr retrieves a term attribute, or nil if it doesn't exist, or the term doesn't have attributes.

func (Term) Dependencies added in v0.15.0

func (t Term) Dependencies() []hcl.Traversal

func (Term) Evaluate added in v0.15.0

func (t Term) Evaluate(
	evalExpr func(expr hcl.Expression, extraVars cty.Value) (cty.Value, hcl.Diagnostics),
) (cty.Value, hcl.Diagnostics)

func (Term) VisitExpressions added in v0.15.0

func (t Term) VisitExpressions(f func(hcl.Expression))

VisitExpressions recursively visits all expressions in a tree of terms.

func (Term) WithCount added in v0.15.0

func (t Term) WithCount(expr hcl.Expression) Term

func (Term) WithForEach added in v0.15.0

func (t Term) WithForEach(expr hcl.Expression) Term

type TermTree added in v0.15.0

type TermTree struct {
	// contains filtered or unexported fields
}

func NewTermTree added in v0.15.0

func NewTermTree() *TermTree

func (*TermTree) AddTerm added in v0.15.0

func (t *TermTree) AddTerm(name FullName, term Term)

func (*TermTree) LookupByPrefix added in v0.15.0

func (t *TermTree) LookupByPrefix(name FullName) (*FullName, *Term)

func (*TermTree) VisitTerms added in v0.15.0

func (t *TermTree) VisitTerms(f func(name FullName, term Term))

type TerraformModuleRegister

type TerraformModuleRegister struct {
	// contains filtered or unexported fields
}

func NewTerraformRegister

func NewTerraformRegister(fsys afero.Fs, dir string) *TerraformModuleRegister

func (*TerraformModuleRegister) GetDir

func (r *TerraformModuleRegister) GetDir(source string) *string

type UnsupportedOperationDiag

type UnsupportedOperationDiag struct {
}

func (UnsupportedOperationDiag) Description

func (UnsupportedOperationDiag) ExtraInfo added in v0.18.0

func (d UnsupportedOperationDiag) ExtraInfo() interface{}

func (UnsupportedOperationDiag) FromExpr

func (UnsupportedOperationDiag) Severity

func (UnsupportedOperationDiag) Source

type Visitor

type Visitor interface {
	VisitModule(name ModuleName, meta *ModuleMeta)
	VisitResource(name FullName, resource *ResourceMeta)
	VisitTerm(name FullName, term Term)
}

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL