hcl_interpreter

package
v0.24.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 1, 2023 License: Apache-2.0 Imports: 20 Imported by: 1

README

hcl_interpreter

This package supports loading HCL configurations. Since it is a bit more complex than the other loaders, it has it's own package. What follows is a brief but complete overview of how it works.

It tries to mimic terraform behavior as much as possible by using terraform as a library. Since some packages inside terraform are in an /internal package, we vendor in terraform and rename the packages to make them accessible from our code. This is automated in the top-level Makefile.

names.go

names.go holds some utilities to work with names, for example parsing them from strings and rendering them back to strings. A name consists of two parts:

  • A module part (e.g. module.child1.). This is empty for the root module.
  • A local part (e.g. aws_s3_bucket.my_bucket or var.my_var).

Importantly, there are a number of AsXYZ() methods, for example AsModuleOutput(). These manipulate the names to point to their "real" location: for example, if you use module.child1.my_output in the root module, the "real" location is the output inside the child module, so module.child1.outputs.my_output. These methods form a big part of the logic and having it here allows us to keep the code in other files relatively clean.

moduletree.go

moduletree.go is responsible for parsing a directory of terraform files, including children (submodules). We end up with a hierarchical structure:

  • root module
    • child 1
    • child 2
      • grandchild

We can "walk" this tree using a visitor pattern and visit every term.

A term can be a simple expression or a block with attributes and sub-blocks. Each resource forms a term, and so does each other "entity" in the input, like a a local variable or a module output. For more details, see term.go.

For example:

  • aws_security_group.invalid_sg_1
  • module.child1.output.bucket

Because we pass in both the full name (see above) as well as the term, a visitor can store these in a flat rather than hierarchical map, which is more convenient to work with in Go.

This file uses an additional file moduleregister.go to deal with the locations of remote (downloaded) terraform modules.

valtree.go

Once expressions are evaluated, they become values of the type cty.Value. This module has a number of utilities to construct and merge Values.

phantom_attrs.go

In IaC files, it is common to depend on values which are not filled in:

resource "aws_kms_key" "rds-db-instance-kms" {
  deletion_window_in_days = 10
}

resource "aws_db_instance" "default" {
  kms_key_id = "${aws_kms_key.rds-db-instance-kms.arn}"
  ...
}

rds-db-instance-kms does not have an .arn attribute, but evaluating kms_key_id needs one. We solve this by collecting all references to unknown attributes in the expressions, and setting these as "phantom attributes" on the resources. They are not included in the output.

hcl_interpreter.go

Finally, using these foundations, hcl_interpreter.go implements the main logic. This happens in the following imperative steps:

  1. We use the visitor from moduletree.go to obtain a full list of everything in the module and its children.

  2. For every term, we can compute its dependencies (possibly renaming some using the logic in names.go). This gives us enough info to run a topological sort; which tells us the exact order in which all terms should be evaluated.

  3. We run through and evaluate each expression.

    • We have a single big cty.Value that holds everything we have evaluated so far per module.

    • Before evaluating, we add extra dependencies to this cty.Value scope based on the code in dependencies() and prepareVariables(). This is used to e.g. get outputs from other modules.

    • After evaluating, we merge the result back into the cty.Value for that module.

  4. We convert the individual cty.Values for the resources into the resources view (this involves only some minor bookkeeping like adding the id and _provider fields).

Documentation

Overview

Implements the `Data` interface. Doesn't really do anything.

Look in `/pkg/hcl_interpreter/README.md` for an explanation of how this works.

This module contains utilities for parsing and traversing everything in a configuration tree.

Look in `/pkg/hcl_interpreter/README.md` for an explanation of how this works.

This module contains utilities for parsing and traversing everything in a configuration tree.

cty.Value utilities

Index

Constants

This section is empty.

Variables

View Source 
var EmptyModuleName = []string{}

Functions

func LocalNameToString 

func LocalNameToString(name LocalName) string

func LookupVal added in v0.15.0 

func LookupVal(tree cty.Value, name LocalName) cty.Value

Look up a given subtree, returns Null if not found

func MergeVal added in v0.15.0 

func MergeVal(left cty.Value, right cty.Value) cty.Value

Merges two Values with a preference given to the right object.

func ModuleNameToString 

func ModuleNameToString(moduleName ModuleName) string

func NestVal added in v0.15.0 

func NestVal(prefix []string, val cty.Value) cty.Value

func TfFilePathJoin 

func TfFilePathJoin(leading, trailing string) string

TfFilePathJoin is like `filepath.Join` but avoids cleaning the path. This allows to get unique paths for submodules including a parent module, e.g.: 

.
examples/mssql/../../
examples/complete/../../

func TraversalToString 

func TraversalToString(traversal hcl.Traversal) string

func ValToVariables added in v0.15.0 

func ValToVariables(tree cty.Value) map[string]cty.Value

Some HCL functions require it to be a map. Returns an empty map if we have anything but an object at the root.

func ValueToInt 

func ValueToInt(val cty.Value) *int

func ValueToInterface 

func ValueToInterface(val cty.Value) (interface{}, []error)

func ValueToString 

func ValueToString(val cty.Value) *string

Types

type Analysis 

type Analysis struct {
	Fs afero.Fs

	// Module metadata
	Modules map[string]*ModuleMeta

	// Resource metadata
	Resources map[string]*ResourceMeta

	// Terms in the evaluation tree.
	Terms *TermTree
	// contains filtered or unexported fields
}

func AnalyzeModuleTree 

func AnalyzeModuleTree(mtree *ModuleTree) *Analysis

func (*Analysis) VisitModule 

func (v *Analysis) VisitModule(name ModuleName, meta *ModuleMeta)

func (*Analysis) VisitResource added in v0.15.0 

func (v *Analysis) VisitResource(name FullName, resource *ResourceMeta)

func (*Analysis) VisitTerm added in v0.15.0 

func (v *Analysis) VisitTerm(name FullName, term Term)

type Data 

type Data struct {
}

func (*Data) GetCountAttr 

func (c *Data) GetCountAttr(addrs.CountAttr, tfdiags.SourceRange) (cty.Value, tfdiags.Diagnostics)

func (*Data) GetForEachAttr 

func (c *Data) GetForEachAttr(addrs.ForEachAttr, tfdiags.SourceRange) (cty.Value, tfdiags.Diagnostics)

func (*Data) GetInputVariable 

func (c *Data) GetInputVariable(v addrs.InputVariable, s tfdiags.SourceRange) (cty.Value, tfdiags.Diagnostics)

func (*Data) GetLocalValue 

func (c *Data) GetLocalValue(addrs.LocalValue, tfdiags.SourceRange) (cty.Value, tfdiags.Diagnostics)

func (*Data) GetModule 

func (c *Data) GetModule(addrs.ModuleCall, tfdiags.SourceRange) (cty.Value, tfdiags.Diagnostics)

func (*Data) GetPathAttr 

func (c *Data) GetPathAttr(attr addrs.PathAttr, diags tfdiags.SourceRange) (cty.Value, tfdiags.Diagnostics)

func (*Data) GetResource 

func (c *Data) GetResource(addrs.Resource, tfdiags.SourceRange) (cty.Value, tfdiags.Diagnostics)

func (*Data) GetTerraformAttr 

func (c *Data) GetTerraformAttr(addrs.TerraformAttr, tfdiags.SourceRange) (cty.Value, tfdiags.Diagnostics)

func (*Data) StaticValidateReferences 

func (c *Data) StaticValidateReferences(refs []*addrs.Reference, self addrs.Referenceable) tfdiags.Diagnostics

type Evaluation 

type Evaluation struct {
	Analysis *Analysis
	Modules  map[string]cty.Value
	// contains filtered or unexported fields
}

func EvaluateAnalysis 

func EvaluateAnalysis(analysis *Analysis) (*Evaluation, error)

func (*Evaluation) Errors 

func (e *Evaluation) Errors() []error

Errors returns the non-fatal errors encountered during evaluation

func (*Evaluation) Location 

func (v *Evaluation) Location(
	resourceId string,
	path []interface{},
) []hcl.Range

func (*Evaluation) Resources 

func (v *Evaluation) Resources() []Resource

type EvaluationError added in v0.22.0 

type EvaluationError struct {
	Diags hcl.Diagnostics
}

func (EvaluationError) Error added in v0.22.0 

func (err EvaluationError) Error() string

type FullName 

type FullName struct {
	Module ModuleName
	Local  LocalName
}

func ArrayToFullName 

func ArrayToFullName(parts []string) FullName

func EmptyFullName 

func EmptyFullName(module ModuleName) FullName

func ProviderConfigName 

func ProviderConfigName(module ModuleName, providerName string) FullName

func StringToFullName 

func StringToFullName(name string) (*FullName, error)

func (FullName) Add added in v0.15.0 

func (name FullName) Add(p string) FullName

func (FullName) AsModuleInput 

func (name FullName) AsModuleInput() *FullName

Parses "module.child.var.foo" into "input.child.foo"

func (FullName) AsModuleOutput 

func (name FullName) AsModuleOutput() *FullName

Parses the use of an output (e.g. "module.child.x") to the fully expanded output name (e.g. module.child.output.x")

func (FullName) AsResourceName 

func (name FullName) AsResourceName() (*FullName, LocalName)

func (FullName) AsVariable 

func (name FullName) AsVariable() (*FullName, *FullName, LocalName)

Parses "var.my_var.key" into "variable.my_var", "var.my_var" and "key".

func (FullName) ToString 

func (name FullName) ToString() string

type LocalName 

type LocalName = []string

func TraversalToLocalName 

func TraversalToLocalName(traversal hcl.Traversal) (LocalName, error)

TODO: Refactor to TraversalToName?

type MissingRemoteSubmodulesError added in v0.22.0 

type MissingRemoteSubmodulesError struct {
	Dir            string
	MissingModules []string
}

func (MissingRemoteSubmodulesError) Error added in v0.22.0 

func (err MissingRemoteSubmodulesError) Error() string

type MissingTermError added in v0.22.0 

type MissingTermError struct {
	Term string
}

func (MissingTermError) Error added in v0.22.0 

func (err MissingTermError) Error() string

type ModuleMeta 

type ModuleMeta struct {
	Dir                  string
	Recurse              bool
	Filepaths            []string
	MissingRemoteModules []string
	Location             *hcl.Range
}

type ModuleName 

type ModuleName = []string

type ModuleTree 

type ModuleTree struct {
	// contains filtered or unexported fields
}

We load the entire tree of submodules in one pass.

func ParseDirectory 

func ParseDirectory(
	moduleRegister *TerraformModuleRegister,
	parserFs afero.Fs,
	dir string,
	varFiles []string,
) (*ModuleTree, error)

func ParseFiles 

func ParseFiles(
	moduleRegister *TerraformModuleRegister,
	parserFs afero.Fs,
	recurse bool,
	dir string,
	filepaths []string,
	varfiles []string,
) (*ModuleTree, error)

func (*ModuleTree) Errors 

func (mtree *ModuleTree) Errors() []error

func (*ModuleTree) FilePath 

func (mtree *ModuleTree) FilePath() string

func (*ModuleTree) LoadedFiles 

func (mtree *ModuleTree) LoadedFiles() []string

func (*ModuleTree) Walk 

func (mtree *ModuleTree) Walk(v Visitor)

type Resource added in v0.15.0 

type Resource struct {
	Meta  *ResourceMeta
	Model models.ResourceState
}

type ResourceMeta 

type ResourceMeta struct {
	Data                      bool
	Type                      string
	ProviderType              string
	ProviderName              string
	ProviderVersionConstraint string
	Multiple                  bool
	Location                  hcl.Range
	Body                      hcl.Body // For source code locations only.
}

type SubmoduleLoadingError added in v0.22.0 

type SubmoduleLoadingError struct {
	Module string
	Err    error
}

func (SubmoduleLoadingError) Error added in v0.22.0 

func (err SubmoduleLoadingError) Error() string

type Term added in v0.15.0 

type Term struct {
	// contains filtered or unexported fields
}

func TermFromBody added in v0.15.0 

func TermFromBody(body hcl.Body) Term

func TermFromExpr added in v0.15.0 

func TermFromExpr(expr hcl.Expression) Term

func (Term) Attributes added in v0.15.0 

func (t Term) Attributes() map[string]Term

Attr retrieves a term attribute, or nil if it doesn't exist, or the term doesn't have attributes.

func (Term) Dependencies added in v0.15.0 

func (t Term) Dependencies() []hcl.Traversal

func (Term) Evaluate added in v0.15.0 

func (t Term) Evaluate(
	evalExpr func(expr hcl.Expression, extraVars cty.Value) (cty.Value, hcl.Diagnostics),
) (cty.Value, hcl.Diagnostics)

func (Term) VisitExpressions added in v0.15.0 

func (t Term) VisitExpressions(f func(hcl.Expression))

VisitExpressions recursively visits all expressions in a tree of terms.

func (Term) WithCount added in v0.15.0 

func (t Term) WithCount(expr hcl.Expression) Term

func (Term) WithForEach added in v0.15.0 

func (t Term) WithForEach(expr hcl.Expression) Term

type TermTree added in v0.15.0 

type TermTree struct {
	// contains filtered or unexported fields
}

func NewTermTree added in v0.15.0 

func NewTermTree() *TermTree

func (*TermTree) AddTerm added in v0.15.0 

func (t *TermTree) AddTerm(name FullName, term Term)

func (*TermTree) LookupByPrefix added in v0.15.0 

func (t *TermTree) LookupByPrefix(name FullName) (*FullName, *Term)

func (*TermTree) VisitTerms added in v0.15.0 

func (t *TermTree) VisitTerms(f func(name FullName, term Term))

type TerraformModuleRegister 

type TerraformModuleRegister struct {
	// contains filtered or unexported fields
}

func NewTerraformRegister 

func NewTerraformRegister(fsys afero.Fs, dir string) *TerraformModuleRegister

func (*TerraformModuleRegister) GetDir 

func (r *TerraformModuleRegister) GetDir(source string) *string

type UnsupportedOperationDiag 

type UnsupportedOperationDiag struct {
}

func (UnsupportedOperationDiag) Description 

func (d UnsupportedOperationDiag) Description() tfdiags.Description

func (UnsupportedOperationDiag) ExtraInfo added in v0.18.0 

func (d UnsupportedOperationDiag) ExtraInfo() interface{}

func (UnsupportedOperationDiag) FromExpr 

func (d UnsupportedOperationDiag) FromExpr() *tfdiags.FromExpr

func (UnsupportedOperationDiag) Severity 

func (d UnsupportedOperationDiag) Severity() tfdiags.Severity

func (UnsupportedOperationDiag) Source 

func (d UnsupportedOperationDiag) Source() tfdiags.Source

type Visitor 

type Visitor interface {
	VisitModule(name ModuleName, meta *ModuleMeta)
	VisitResource(name FullName, resource *ResourceMeta)
	VisitTerm(name FullName, term Term)
}

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.