hil

package module
v0.0.0-...-4415e05 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 19, 2024 License: MPL-2.0 Imports: 12 Imported by: 507

README

HIL

GoDoc Build Status

HIL (HashiCorp Interpolation Language) is a lightweight embedded language used primarily for configuration interpolation. The goal of HIL is to make a simple language for interpolations in the various configurations of HashiCorp tools.

HIL is built to interpolate any string, but is in use by HashiCorp primarily with HCL. HCL is not required in any way for use with HIL.

HIL isn't meant to be a general purpose language. It was built for basic configuration interpolations. Therefore, you can't currently write functions, have conditionals, set intermediary variables, etc. within HIL itself. It is possible some of these may be added later but the right use case must exist.

Why?

Many of our tools have support for something similar to templates, but within the configuration itself. The most prominent requirement was in Terraform where we wanted the configuration to be able to reference values from elsewhere in the configuration. Example:

foo = "hi ${var.world}"

We originally used a full templating language for this, but found it was too heavy weight. Additionally, many full languages required bindings to C (and thus the usage of cgo) which we try to avoid to make cross-compilation easier. We then moved to very basic regular expression based string replacement, but found the need for basic arithmetic and function calls resulting in overly complex regular expressions.

Ultimately, we wrote our own mini-language within Terraform itself. As we built other projects such as Nomad and Otto, the need for basic interpolations arose again.

Thus HIL was born. It is extracted from Terraform, cleaned up, and better tested for general purpose use.

Syntax

For a complete grammar, please see the parser itself. A high-level overview of the syntax and grammar is listed here.

Code begins within ${ and }. Outside of this, text is treated literally. For example, foo is a valid HIL program that is just the string "foo", but foo ${bar} is an HIL program that is the string "foo " concatened with the value of bar. For the remainder of the syntax docs, we'll assume you're within ${}.

  • Identifiers are any text in the format of [a-zA-Z0-9-.]. Example identifiers: foo, var.foo, foo-bar.

  • Strings are double quoted and can contain any UTF-8 characters. Example: "Hello, World"

  • Numbers are assumed to be base 10. If you prefix a number with 0x, it is treated as a hexadecimal. If it is prefixed with 0, it is treated as an octal. Numbers can be in scientific notation: "1e10".

  • Unary - can be used for negative numbers. Example: -10 or -0.2

  • Boolean values: true, false

  • The following arithmetic operations are allowed: +, -, *, /, %.

  • Function calls are in the form of name(arg1, arg2, ...). Example: add(1, 5). Arguments can be any valid HIL expression, example: add(1, var.foo) or even nested function calls: add(1, get("some value")).

  • Within strings, further interpolations can be opened with ${}. Example: "Hello ${nested}". A full example including the original ${} (remember this list assumes were inside of one already) could be: foo ${func("hello ${var.foo}")}.

Language Changes

We've used this mini-language in Terraform for years. For backwards compatibility reasons, we're unlikely to make an incompatible change to the language but we're not currently making that promise, either.

The internal API of this project may very well change as we evolve it to work with more of our projects. We recommend using some sort of dependency management solution with this package.

Future Changes

The following changes are already planned to be made at some point:

  • Richer types: lists, maps, etc.

  • Convert to a more standard Go parser structure similar to HCL. This will improve our error messaging as well as allow us to have automatic formatting.

  • Allow interpolations to result in more types than just a string. While within the interpolation basic types are honored, the result is always a string.

Documentation

Overview

Example (Basic) 
package main

import (
	"fmt"
	"log"

	"github.com/hashicorp/hil"
)

func main() {
	input := "${6 + 2}"

	tree, err := hil.Parse(input)
	if err != nil {
		log.Fatal(err)
	}

	result, err := hil.Eval(tree, &hil.EvalConfig{})
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Type: %s\n", result.Type)
	fmt.Printf("Value: %s\n", result.Value)
}

Output:

Type: TypeString
Value: 8
Example (Functions) 
package main

import (
	"fmt"
	"log"
	"strings"

	"github.com/hashicorp/hil"
	"github.com/hashicorp/hil/ast"
)

func main() {
	input := "${lower(var.test)} - ${6 + 2}"

	tree, err := hil.Parse(input)
	if err != nil {
		log.Fatal(err)
	}

	lowerCase := ast.Function{
		ArgTypes:   []ast.Type{ast.TypeString},
		ReturnType: ast.TypeString,
		Variadic:   false,
		Callback: func(inputs []interface{}) (interface{}, error) {
			input := inputs[0].(string)
			return strings.ToLower(input), nil
		},
	}

	config := &hil.EvalConfig{
		GlobalScope: &ast.BasicScope{
			VarMap: map[string]ast.Variable{
				"var.test": ast.Variable{
					Type:  ast.TypeString,
					Value: "TEST STRING",
				},
			},
			FuncMap: map[string]ast.Function{
				"lower": lowerCase,
			},
		},
	}

	result, err := hil.Eval(tree, config)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Type: %s\n", result.Type)
	fmt.Printf("Value: %s\n", result.Value)
}

Output:

Type: TypeString
Value: test string - 8
Example (Variables) 
package main

import (
	"fmt"
	"log"

	"github.com/hashicorp/hil"
	"github.com/hashicorp/hil/ast"
)

func main() {
	input := "${var.test} - ${6 + 2}"

	tree, err := hil.Parse(input)
	if err != nil {
		log.Fatal(err)
	}

	config := &hil.EvalConfig{
		GlobalScope: &ast.BasicScope{
			VarMap: map[string]ast.Variable{
				"var.test": ast.Variable{
					Type:  ast.TypeString,
					Value: "TEST STRING",
				},
			},
		},
	}

	result, err := hil.Eval(tree, config)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Type: %s\n", result.Type)
	fmt.Printf("Value: %s\n", result.Value)
}

Output:

Type: TypeString
Value: TEST STRING - 8

Index

Examples

Constants

View Source 
const UnknownValue = "74D93920-ED26-11E3-AC10-0800200C9A66"

UnknownValue is a sentinel value that can be used to denote that a value of a variable (or map element, list element, etc.) is unknown. This will always have the type ast.TypeUnknown.

Variables

View Source 
var InvalidResult = EvaluationResult{Type: TypeInvalid, Value: nil}

InvalidResult is a structure representing the result of a HIL interpolation which has invalid syntax, missing variables, or some other type of error. The error is described out of band in the accompanying error return value.

Functions

func FixedValueTransform 

func FixedValueTransform(root ast.Node, Value *ast.LiteralNode) ast.Node

FixedValueTransform transforms an AST to return a fixed value for all interpolations. i.e. you can make "hi ${anything}" always turn into "hi foo".

The primary use case for this is for config validations where you can verify that interpolations result in a certain type of string.

func InterfaceToVariable 

func InterfaceToVariable(input interface{}) (ast.Variable, error)

func Parse 

func Parse(v string) (ast.Node, error)

Parse parses the given program and returns an executable AST tree.

Syntax errors are returned with error having the dynamic type *parser.ParseError, which gives the caller access to the source position where the error was found, which allows (for example) combining it with a known source filename to add context to the error message.

func ParseWithPosition 

func ParseWithPosition(v string, pos ast.Pos) (ast.Node, error)

ParseWithPosition is like Parse except that it overrides the source row and column position of the first character in the string, which should be 1-based.

This can be used when HIL is embedded in another language and the outer parser knows the row and column where the HIL expression started within the overall source file.

func VariableToInterface 

func VariableToInterface(input ast.Variable) (interface{}, error)

func Walk 

func Walk(v interface{}, cb WalkFn) error

Walk will walk an arbitrary Go structure and parse any string as an HIL program and call the callback cb to determine what to replace it with.

This function is very useful for arbitrary HIL program interpolation across a complex configuration structure. Due to the heavy use of reflection in this function, it is recommend to write many unit tests with your typical configuration structures to hilp mitigate the risk of panics.

Types

type EvalConfig 

type EvalConfig struct {
	// GlobalScope is the global scope of execution for evaluation.
	GlobalScope *ast.BasicScope

	// SemanticChecks is a list of additional semantic checks that will be run
	// on the tree prior to evaluating it. The type checker, identifier checker,
	// etc. will be run before these automatically.
	SemanticChecks []SemanticChecker
}

EvalConfig is the configuration for evaluating.

type EvalNode 

type EvalNode interface {
	Eval(ast.Scope, *ast.Stack) (interface{}, ast.Type, error)
}

EvalNode is the interface that must be implemented by any ast.Node to support evaluation. This will be called in visitor pattern order. The result of each call to Eval is automatically pushed onto the stack as a LiteralNode. Pop elements off the stack to get child values.

type EvalType 

type EvalType uint32

EvalType represents the type of the output returned from a HIL evaluation. 

const (
	TypeInvalid EvalType = 0
	TypeString  EvalType = 1 << iota
	TypeBool
	TypeList
	TypeMap
	TypeUnknown
)

func (EvalType) String 

func (i EvalType) String() string

type EvaluationResult 

type EvaluationResult struct {
	Type  EvalType
	Value interface{}
}

EvaluationResult is a struct returned from the hil.Eval function, representing the result of an interpolation. Results are returned in their "natural" Go structure rather than in terms of the HIL AST. For the types currently implemented, this means that the Value field can be interpreted as the following Go types: 

TypeInvalid: undefined
TypeString:  string
TypeList:    []interface{}
TypeMap:     map[string]interface{}
TypBool:     bool

func Eval 

func Eval(root ast.Node, config *EvalConfig) (EvaluationResult, error)

type IdentifierCheck 

type IdentifierCheck struct {
	Scope ast.Scope
	// contains filtered or unexported fields
}

IdentifierCheck is a SemanticCheck that checks that all identifiers resolve properly and that the right number of arguments are passed to functions.

func (*IdentifierCheck) Visit 

func (c *IdentifierCheck) Visit(root ast.Node) error

type SemanticChecker 

type SemanticChecker func(ast.Node) error

SemanticChecker is the type that must be implemented to do a semantic check on an AST tree. This will be called with the root node.

type TypeCheck 

type TypeCheck struct {
	Scope ast.Scope

	// Implicit is a map of implicit type conversions that we can do,
	// and that shouldn't error. The key of the first map is the from type,
	// the key of the second map is the to type, and the final string
	// value is the function to call (which must be registered in the Scope).
	Implicit map[ast.Type]map[ast.Type]string

	// Stack of types. This shouldn't be used directly except by implementations
	// of TypeCheckNode.
	Stack []ast.Type
	// contains filtered or unexported fields
}

TypeCheck implements ast.Visitor for type checking an AST tree. It requires some configuration to look up the type of nodes.

It also optionally will not type error and will insert an implicit type conversions for specific types if specified by the Implicit field. Note that this is kind of organizationally weird to put into this structure but we'd rather do that than duplicate the type checking logic multiple times.

func (*TypeCheck) ImplicitConversion 

func (v *TypeCheck) ImplicitConversion(
	actual ast.Type, expected ast.Type, n ast.Node) ast.Node

func (*TypeCheck) StackPeek 

func (v *TypeCheck) StackPeek() ast.Type

func (*TypeCheck) StackPop 

func (v *TypeCheck) StackPop() ast.Type

func (*TypeCheck) StackPush 

func (v *TypeCheck) StackPush(t ast.Type)

func (*TypeCheck) Visit 

func (v *TypeCheck) Visit(root ast.Node) error

type TypeCheckNode 

type TypeCheckNode interface {
	TypeCheck(*TypeCheck) (ast.Node, error)
}

TypeCheckNode is the interface that must be implemented by any ast.Node that wants to support type-checking. If the type checker encounters a node that doesn't implement this, it will error.

type WalkData 

type WalkData struct {
	// Root is the parsed root of this HIL program
	Root ast.Node

	// Location is the location within the structure where this
	// value was found. This can be used to modify behavior within
	// slices and so on.
	Location reflectwalk.Location

	// The below two values must be set by the callback to have any effect.
	//
	// Replace, if true, will replace the value in the structure with
	// ReplaceValue. It is up to the caller to make sure this is a string.
	Replace      bool
	ReplaceValue string
}

WalkData is the structure passed to the callback of the Walk function.

This structure contains data passed in as well as fields that are expected to be written by the caller as a result. Please see the documentation for each field for more information.

type WalkFn 

type WalkFn func(*WalkData) error

WalkFn is the type of function to pass to Walk. Modify fields within WalkData to control whether replacement happens.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.