hcl

README

Parsing, encoding and decoding of HCL to and from Go types

This package provides idiomatic Go functions for marshalling and unmarshalling HCL, as well as an AST

It supports the same tags as the Hashicorp hcl2 gohcl package, but is much less complex.

Unlike gohcl it also natively supports time.Duration, time.Time, encoding.TextUnmarshaler and json.Unmarshaler.

It is HCL1 compatible and does not support any HCL2 specific features.

Design

HCL -> AST -> Go -> AST -> HCL

Mapping can start from any point in this cycle.

Marshalling, unmarshalling, parsing and serialisation are all structurally isomorphic operations. That is, HCL can be deserialised into an AST or Go, or vice versa, and the structure on both ends will be identical.

HCL is always parsed into an AST before unmarshaling and, similarly, Go structures are always mapped to an AST before being serialised to HCL.

Between	And	Preserves
HCL	AST	Structure, values, order, comments.
HCL	Go	Structure, values, partial comments (via the help:"" tag).
AST	Go	Structure, values.

Schema reflection

HCL has no real concept of schemas (that I can find), but there is precedent for something similar in Terraform variable definition files. This package supports reflecting a rudimentary schema from Go, where the value for each attribute is one of the scalar types number, string or boolean. Lists and maps are typed by example.

Here's an example schema.

// A string field.
str = string
num = number
bool = boolean
list = [string]

// A map.
map = {
  string: number,
}

// A block.
block "name" {
  attr = string
}

// Repeated blocks.
block_slice "label0" "label1" {
  attr = string
}

Comments are from help:"" tags. See schema_test.go for details.

Struct field tags

The tag format is as with other similar serialisation packages:

hcl:"[<name>][,<option>]"

The supported options are:

Tag	Description
attr (default)	Specifies that the value is to be populated from an attribute.
block	Specifies that the value is to populated from a block.
label	Specifies that the value is to populated from a block label.
optional	As with attr, but the field is optional.
remain	Specifies that the value is to be populated from the remaining body after populating other fields. The field must be of type []*hcl.Entry.

Additionally, a separate help:"" tag can be specified to populate comment fields in the AST when serialising Go structures.

Position

Any block with a field named Pos of the type hcl.Position will have that field populated with positional information:

Pos Position `hcl:"-"`

Documentation

Overview

Package hcl implements parsing, encoding and decoding of HCL from Go types.

Its purpose is to provide idiomatic Go functions and types for HCL.

Index

• func AddParentRefs(node Node) error
• func Marshal(v interface{}, options ...MarshalOption) ([]byte, error)
• func MarshalAST(ast Node) ([]byte, error)
• func MarshalASTToWriter(ast Node, w io.Writer) error
• func MarshalJSON(ast *AST, options ...MarshalJSONOption) ([]byte, error)
• func StripComments(node Node) error
• func Unmarshal(data []byte, v interface{}, options ...MarshalOption) error
• func UnmarshalAST(ast *AST, v interface{}, options ...MarshalOption) error
• func UnmarshalBlock(block *Block, v interface{}, options ...MarshalOption) error
• func Visit(node Node, visitor func(node Node, next func() error) error) error
• type AST
  • func BlockSchema(name string, v interface{}, options ...MarshalOption) (*AST, error)
  • func MarshalToAST(v interface{}, options ...MarshalOption) (*AST, error)
  • func MustBlockSchema(name string, v interface{}, options ...MarshalOption) *AST
  • func MustSchema(v interface{}, options ...MarshalOption) *AST
  • func Parse(r io.Reader) (*AST, error)
  • func ParseBytes(data []byte) (*AST, error)
  • func ParseString(str string) (*AST, error)
  • func Schema(v interface{}, options ...MarshalOption) (*AST, error)
  • func (a *AST) Clone() *AST
  • func (a *AST) MarshalJSON() ([]byte, error)
• type Attribute
  • func (a *Attribute) Clone() *Attribute
  • func (a *Attribute) String() string
• type Block
  • func (b *Block) Clone() *Block
  • func (b *Block) Detach() bool
• type Bool
  • func (b *Bool) Capture(values []string) error
• type Entry
  • func (e *Entry) Clone() *Entry
  • func (e *Entry) Detach() bool
  • func (e *Entry) Key() string
• type MapEntry
  • func (e *MapEntry) Clone() *MapEntry
• type MarshalJSONOption
  • func IncludeComments(ok bool) MarshalJSONOption
• type MarshalOption
  • func AllowExtra(ok bool) MarshalOption
  • func BareBooleanAttributes(v bool) MarshalOption
  • func HereDocsForMultiLine(n int) MarshalOption
  • func InferHCLTags(v bool) MarshalOption
  • func WithSchemaComments(v bool) MarshalOption
• type Node
  • func Find(node Node, names ...string) (nodes []Node)
• type Number
  • func (n *Number) Capture(values []string) error
  • func (n *Number) GoString() string
• type Position
• type Value
  • func (v *Value) Clone() *Value
  • func (v *Value) GetHeredoc() string
  • func (v *Value) String() string

Functions

func AddParentRefs

func AddParentRefs(node Node) error

AddParentRefs recursively updates an AST's parent references.

This is called automatically during Parse*(), but can be called on a manually constructed AST.

func Marshal

func Marshal(v interface{}, options ...MarshalOption) ([]byte, error)

Marshal a Go type to HCL.

func MarshalAST

func MarshalAST(ast Node) ([]byte, error)

MarshalAST marshals an AST to HCL bytes.

func MarshalASTToWriter

func MarshalASTToWriter(ast Node, w io.Writer) error

MarshalASTToWriter marshals a hcl.AST to an io.Writer.

func MarshalJSON

func MarshalJSON(ast *AST, options ...MarshalJSONOption) ([]byte, error)

MarshalJSON gives fine-grained control over JSON marshaling of an AST.

Currently this just means that emission of comments can be controlled.

func StripComments

func StripComments(node Node) error

StripComments recursively from an AST node.

func Unmarshal

func Unmarshal(data []byte, v interface{}, options ...MarshalOption) error

Unmarshal HCL into a Go struct.

func UnmarshalAST

func UnmarshalAST(ast *AST, v interface{}, options ...MarshalOption) error

UnmarshalAST unmarshalls an already parsed or constructed AST into a Go struct.

func UnmarshalBlock

func UnmarshalBlock(block *Block, v interface{}, options ...MarshalOption) error

UnmarshalBlock into a struct.

func Visit

func Visit(node Node, visitor func(node Node, next func() error) error) error

Visit nodes in the AST.

"next" may be called to continue traversal of child nodes.

Types

type AST

type AST struct {
	Pos lexer.Position `parser:"" json:"-"`

	Entries          []*Entry `parser:"@@*" json:"entries,omitempty"`
	TrailingComments []string `parser:"@Comment*" json:"trailingComments,omitempty"`
	Schema           bool     `parser:"" json:"schema,omitempty"`
}

AST for HCL.

func BlockSchema

func BlockSchema(name string, v interface{}, options ...MarshalOption) (*AST, error)

BlockSchema reflects a block schema for a Go struct.

func MarshalToAST

func MarshalToAST(v interface{}, options ...MarshalOption) (*AST, error)

MarshalToAST marshals a Go type to a hcl.AST.

func MustBlockSchema

func MustBlockSchema(name string, v interface{}, options ...MarshalOption) *AST

MustBlockSchema reflects a block schema from a Go struct, panicking if an error occurs.

func MustSchema

func MustSchema(v interface{}, options ...MarshalOption) *AST

MustSchema constructs a schema from a Go type, or panics.

func Parse

func Parse(r io.Reader) (*AST, error)

Parse HCL from an io.Reader.

func ParseBytes

func ParseBytes(data []byte) (*AST, error)

ParseBytes parses HCL from bytes.

func ParseString

func ParseString(str string) (*AST, error)

ParseString parses HCL from a string.

func Schema

func Schema(v interface{}, options ...MarshalOption) (*AST, error)

Schema reflects a schema from a Go value.

A schema is itself HCL.

func (*AST) Clone

func (a *AST) Clone() *AST

Clone the AST.

func (*AST) MarshalJSON

func (a *AST) MarshalJSON() ([]byte, error)

type Attribute

type Attribute struct {
	Pos    lexer.Position `parser:"" json:"-"`
	Parent Node           `parser:"" json:"-"`

	Comments []string `parser:"@Comment*" json:"comments,omitempty"`

	Key   string `parser:"@Ident ['='" json:"key"`
	Value *Value `parser:"@@]" json:"value"`

	// This will be populated during unmarshalling.
	Default *Value `parser:"" json:"default,omitempty"`

	// This will be parsed from the enum tag and will be helping the validation during unmarshalling
	Enum []*Value `parser:"" json:"enum,omitempty"`

	// Set for schemas when the attribute is optional.
	Optional bool `parser:"" json:"optional,omitempty"`
}

Attribute is a key+value attribute.

func (*Attribute) Clone

func (a *Attribute) Clone() *Attribute

Clone the AST.

func (*Attribute) String

func (a *Attribute) String() string

type Block

type Block struct {
	Pos    lexer.Position `parser:"" json:"-"`
	Parent Node           `parser:"" json:"-"`

	Comments []string `parser:"@Comment*" json:"comments,omitempty"`

	Name   string   `parser:"@Ident" json:"name"`
	Labels []string `parser:"@( Ident | String )*" json:"labels,omitempty"`
	Body   []*Entry `parser:"'{' @@*" json:"body"`

	TrailingComments []string `parser:"@Comment* '}'" json:"trailingComments,omitempty"`

	// The block can be repeated. This is surfaced in schemas.
	Repeated bool `parser:"" json:"repeated,omitempty"`
}

Block represents am optionally labelled HCL block.

func (*Block) Clone

func (b *Block) Clone() *Block

Clone the AST.

func (*Block) Detach

func (b *Block) Detach() bool

Detach Block from parent.

Returns true if successful.

type Bool

type Bool bool

Bool represents a parsed boolean value.

func (*Bool) Capture

func (b *Bool) Capture(values []string) error

type Entry

type Entry struct {
	Pos             lexer.Position `parser:"" json:"-"`
	Parent          Node           `parser:"" json:"-"`
	RecursiveSchema bool           `parser:"" json:"-"`

	Block     *Block     `parser:"(   @@" json:"block,omitempty"`
	Attribute *Attribute `parser:"  | @@ )" json:"attribute,omitempty"`
}

Entry at the top-level of a HCL file or block.

func (*Entry) Clone

func (e *Entry) Clone() *Entry

Clone the AST.

func (*Entry) Detach

func (e *Entry) Detach() bool

Detach Entry from parent.

func (*Entry) Key

func (e *Entry) Key() string

Key of the attribute or block.

type MapEntry

type MapEntry struct {
	Pos    lexer.Position `parser:"" json:"-"`
	Parent Node           `parser:"" json:"-"`

	Comments []string `parser:"@Comment*" json:"comments,omitempty"`

	Key   *Value `parser:"@@ ':'" json:"key"`
	Value *Value `parser:"@@" json:"value"`
}

MapEntry represents a key+value in a map.

func (*MapEntry) Clone

func (e *MapEntry) Clone() *MapEntry

Clone the AST.

type MarshalJSONOption

type MarshalJSONOption func(o *marshalJSONOptions)

MarshalJSONOption implementations control how JSON is marshalled.

func IncludeComments

func IncludeComments(ok bool) MarshalJSONOption

IncludeComments includes comments as __comments__ attributes in the generated JSON.

type MarshalOption

type MarshalOption func(options *marshalState)

MarshalOption configures optional marshalling behaviour.

func AllowExtra

func AllowExtra(ok bool) MarshalOption

AllowExtra fields in configuration to be skipped.

func BareBooleanAttributes

func BareBooleanAttributes(v bool) MarshalOption

BareBooleanAttributes specifies whether attributes without values will be treated as boolean true values.

eg.

attr

NOTE: This is non-standard HCL.

func HereDocsForMultiLine

func HereDocsForMultiLine(n int) MarshalOption

HereDocsForMultiLine will marshal multi-line strings >= n lines as indented heredocs rather than quoted strings.

func InferHCLTags

func InferHCLTags(v bool) MarshalOption

InferHCLTags specifies whether to infer behaviour if hcl:"" tags are not present.

This currently just means that all structs become blocks.

func WithSchemaComments

func WithSchemaComments(v bool) MarshalOption

WithSchemaComments will export the contents of the help struct tag as comments when marshaling.

type Node

type Node interface {
	// contains filtered or unexported methods
}

Node is the the interface implemented by all AST nodes.

func Find

func Find(node Node, names ...string) (nodes []Node)

Find and return blocks, attributes or map entries with the given key.

type Number

type Number struct{ *big.Float }

Number of arbitrary precision.

func (*Number) Capture

func (n *Number) Capture(values []string) error

Capture override because big.Float doesn't directly support 0-prefix octal parsing... why?

func (*Number) GoString

func (n *Number) GoString() string

type Position

type Position = lexer.Position

Position in source file.

type Value

type Value struct {
	Pos    lexer.Position `parser:"" json:"-"`
	Parent Node           `parser:"" json:"-"`

	Bool             *Bool       `parser:"(  @('true':Ident | 'false':Ident)" json:"bool,omitempty"`
	Number           *Number     `parser:" | @Number" json:"number,omitempty"`
	Type             *string     `parser:" | @('number':Ident | 'string':Ident | 'boolean':Ident)" json:"type,omitempty"`
	Str              *string     `parser:" | @(String | Ident)" json:"str,omitempty"`
	HeredocDelimiter string      `parser:" | (@Heredoc" json:"heredocDelimiter,omitempty"`
	Heredoc          *string     `parser:"     @(Body | EOL)* End)" json:"heredoc,omitempty"`
	HaveList         bool        `parser:" | ( @'['" json:"haveList,omitempty"` // Need this to detect empty lists.
	List             []*Value    `parser:"     ( @@ ( ',' @@ )* )? ','? ']' )" json:"list,omitempty"`
	HaveMap          bool        `parser:" | ( @'{'" json:"haveMap,omitempty"` // Need this to detect empty maps.
	Map              []*MapEntry `parser:"     ( @@ ( ',' @@ )* ','? )? '}' ) )" json:"map,omitempty"`
}

Value is a scalar, list or map.

func (*Value) Clone

func (v *Value) Clone() *Value

Clone the AST.

func (*Value) GetHeredoc

func (v *Value) GetHeredoc() string

GetHeredoc gets the heredoc as a string.

This will correctly format indented heredocs.

func (*Value) String

func (v *Value) String() string