envconfig

README

envconfig

envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct.

See the example to understand how to use it, it's pretty simple.

Supported types

• Almost all standard types plus time.Duration are supported by default.
• Slices and arrays
• Arbitrary structs
• Custom types via the Unmarshaler interface.

How does it work

envconfig takes the hierarchy of your configuration struct and the names of the fields to create a environment variable key.

For example:

var conf struct {
    Name string
    Shard struct {
        Host string
        Port int
    }
}

This will check for those 3 keys:

• NAME or name
• SHARD_HOST, or shard_host
• SHARD_PORT, or shard_port

Flexible key naming

envconfig supports having underscores in the key names where there is a word boundary. Now, that term is not super explicit, so let me show you an example:

var conf struct {
    Cassandra struct {
        SSLCert string
        SslKey string
    }
}

This will check all of the following keys:

• CASSANDRA_SSL_CERT, CASSANDRA_SSLCERT, cassandra_ssl_cert, cassandra_sslcert
• CASSANDRA_SSL_KEY, CASSANDRA_SSLKEY, cassandra_ssl_key, cassandra_sslkey

If that is not good enough, look just below.

Custom environment variable names

envconfig supports custom environment variable names:

var conf struct {
    Name string `envconfig:"myName"`
}

Default values

envconfig supports default values:

var conf struct {
    Name string `envconfig:"default=Vincent"`
}

Optional values

envconfig supports optional values:

var conf struct {
    Name string `envconfig:"optional"`
    Age int     `envconfig:"-"`
}

The two syntax are equivalent.

Combining multiple options in one tag

You can of course combine multiple options:

var conf struct {
    Name string `envconfig:"default=Vincent,myName"`
}

Slices or arrays

With slices or arrays, the same naming is applied for the slice. To put multiple elements into the slice or array, you need to separate them with a , (will probably be configurable in the future, or at least have a way to escape)

For example:

var conf struct {
    Ports []int
}

This will check for the key PORTS:

• if your variable is 9000 the slice will contain only 9000
• if your variable is 9000,100 the slice will contain 9000 and 100

For slices of structs, it's a little more complicated. The same splitting of slice elements is done with a comma, however, each token must follow a specific format like this: {<first field>,<second field>,...}

For example:

var conf struct {
    Shards []struct {
        Name string
        Port int
    }
}

This will check for the key SHARDS. Example variable content: {foobar,9000},{barbaz,20000}

This will result in two struct defined in the Shards slice.

Future work

• support for time.Time values with a layout defined via a field tag
• support for complex types

Documentation

Overview

Package envconfig implements a configuration reader which reads each value from an environment variable.

The basic idea is that you define a configuration struct, like this:

var conf struct {
    Addr string
    Port int
    Auth struct {
        Key      string
        Endpoint string
    }
    Partitions []int
    Shards     []struct {
        Name string
        Id   int
    }
}

Once you have that, you need to initialize the configuration:

if err := envconfig.Init(&conf); err != nil {
    log.Fatalln(err)
}

Then it's just a matter of setting the environment variables when calling your binary:

ADDR=localhost PORT=6379 AUTH_KEY=foobar ./mybinary

Layout of the conf struct

Your conf struct must follow the following rules:

• no unexported fields by default (can turn off with Options.AllowUnexported)
• only supported types (no map fields for example)

Naming of the keys

By default, envconfig generates all possible keys based on the field chain according to a flexible naming scheme.

The field chain is how you access your field in the configuration struct. For example:

var conf struct {
    Shard struct {
        Name string
    }
}

With that struct, you access the name field via the chain *Shard.Name*

The default naming scheme takes that and transforms it into the following:

• SHARD_NAME
• shard_name

It can handles more complicated cases, with multiple words in one field name. It needs to be in the correct case though, for example:

var conf struct {
    Cassandra struct {
        SSLCert string
        SslKey  string
    }
}

With that struct, you access the name field via the chain *Cassandra.SSLCert* or *Cassandra.SslKey*

The default naming scheme takes that and transforms it into the following:

• CASSANDRA_SSL_CERT, cassandra_ssl_cert, CASSANDRA_SSLCERT, cassandra_sslcert
• CASSANDRA_SSL_KEY, cassandra_ssl_key, CASSANDRA_SSLKEY, cassandra_sslkey

And, if that is not good enough for you, you always have the option to use a custom key:

var conf struct {
    Cassandra struct {
        Name string `envconfig:"cassandraMyName"`
    }
}

Now envconfig will only ever checks the environment variable _cassandraMyName_.

Content of the variables

There are three types of content for a single variable:

• for simple types, a single string representing the value, and parseable into the type.
• for slices or arrays, a comma-separated list of strings. Each string must be parseable into the element type of the slice or array.
• for structs, a comma-separated list of specially formatted strings representing structs.

Example of a valid slice value:

foo,bar,baz

The format for a struct is as follow:

• prefixed with {
• suffixed with }
• contains a comma-separated list of field values, in the order in which they are defined in the struct

Example of a valid struct value:

type MyStruct struct {
    Name    string
    Id      int
    Timeout time.Duration
}

{foobar,10,120s}

Example of a valid slice of struct values:

{foobar,10,120s},{barbaz,20,50s}

Special case for bytes slices

For bytes slices, you generally don't want to type out a comma-separated list of byte values.

For this use case, we support base64 encoded values.

Here's an example:

var conf struct {
    Data []byte
}

os.Setenv("DATA", "Rk9PQkFS")

This will decode DATA to FOOBAR and put that into conf.Data.

Optional values

Sometimes you don't absolutely need a value. Here's how we tell envconfig a value is optional:

var conf struct {
    Name string `envconfig:"optional"`
    Age int     `envconfig:"-"`
}

The two syntax are equivalent.

Default values

Often times you have configuration keys which almost never changes, but you still want to be able to change them.

In such cases, you might want to provide a default value.

Here's to do this with envconfig:

var conf struct {
    Timeout time.Duration `envconfig:"default=1m"`
}

Combining options

You can of course combine multiple options. The syntax is simple enough, separate each option with a comma.

For example:

var conf struct {
    Timeout time.Duration `envconfig:"default=1m,myTimeout"`
}

This would give you the default timeout of 1 minute, and lookup the myTimeout environment variable.

Supported types

envconfig supports the following list of types:

• bool
• string
• intX
• uintX
• floatX
• time.Duration
• pointers to all of the above types

Notably, we don't (yet) support complex types simply because I had no use for it yet.

Custom unmarshaler

When the standard types are not enough, you will want to use a custom unmarshaler for your types.

You do this by implementing Unmarshaler on your type. Here's an example:

type connectionType uint

const (
    tlsConnection connectionType = iota
    insecureConnection
)

func (t *connectionType) Unmarshal(s string) error {
    switch s {
        case "tls":
            *t = tlsConnection
        case "insecure":
            *t = insecureConnection
        default:
            return fmt.Errorf("unable to unmarshal %s to a connection type", s)
    }

    return nil
}

Index

• Variables
• func Init(conf interface{}) error
• func InitWithOptions(conf interface{}, opts Options) error
• func InitWithPrefix(conf interface{}, prefix string) error
• type Options
• type Unmarshaler

Examples

• Init
• InitWithPrefix

Variables

var (
	// ErrUnexportedField is the error returned by the Init* functions when a field of the config struct is not exported and the option AllowUnexported is not used.
	ErrUnexportedField = errors.New("envconfig: unexported field")
	// ErrNotAPointer is the error returned by the Init* functions when the configuration object is not a pointer.
	ErrNotAPointer = errors.New("envconfig: value is not a pointer")
	// ErrInvalidValueKind is the error returned by the Init* functions when the configuration object is not a struct.
	ErrInvalidValueKind = errors.New("envconfig: invalid value kind, only works on structs")
)

Functions

func Init

func Init(conf interface{}) error

Init reads the configuration from environment variables and populates the conf object. conf must be a pointer

Example

package main

import (
	"fmt"
	"os"
	"time"

	"github.com/HiFX/envconfig"
)

func main() {
	var conf struct {
		MySQL struct {
			Host     string
			Port     int
			Database struct {
				User     string
				Password string
				Name     string
			}
			Params struct {
				Charset string `envconfig:"-"`
			}
		}
		Log struct {
			Path   string `envconfig:"default=/var/log/mylog.log"`
			Rotate bool   `envconfig:"logRotate"`
		}
		NbWorkers int
		Timeout   time.Duration
		Cassandra struct {
			SSLCert string
			SSLKey  string
		}
	}

	os.Setenv("MYSQL_HOST", "localhost")
	os.Setenv("MYSQL_PORT", "3306")
	os.Setenv("MYSQL_DATABASE_USER", "root")
	os.Setenv("MYSQL_DATABASE_PASSWORD", "foobar")
	os.Setenv("MYSQL_DATABASE_NAME", "default")
	os.Setenv("logRotate", "true")
	os.Setenv("NBWORKERS", "10")
	os.Setenv("TIMEOUT", "120s")
	os.Setenv("CASSANDRA_SSL_CERT", "/etc/cassandra/ssl.crt")
	os.Setenv("CASSANDRA_SSL_KEY", "/etc/cassandra/ssl.key")

	if err := envconfig.Init(&conf); err != nil {
		fmt.Printf("err=%s\n", err)
	}

	fmt.Println(conf.MySQL.Database.User)
	fmt.Println(conf.Log.Rotate)
	fmt.Println(conf.Timeout)
	fmt.Println(conf.Log.Path)
	fmt.Println(conf.Cassandra.SSLCert)
	fmt.Println(conf.Cassandra.SSLKey)
}

Output:

root
true
2m0s
/var/log/mylog.log
/etc/cassandra/ssl.crt
/etc/cassandra/ssl.key

func InitWithOptions

func InitWithOptions(conf interface{}, opts Options) error

InitWithOptions reads the configuration from environment variables and populates the conf object. conf must be a pointer.

func InitWithPrefix

func InitWithPrefix(conf interface{}, prefix string) error

InitWithPrefix reads the configuration from environment variables and populates the conf object. conf must be a pointer. Each key read will be prefixed with the prefix string.

Example

package main

import (
	"fmt"
	"os"

	"github.com/HiFX/envconfig"
)

func main() {
	var conf struct {
		Name string
	}

	os.Setenv("NAME", "")
	os.Setenv("FOO_NAME", "")

	os.Setenv("NAME", "foobar")

	err := envconfig.InitWithPrefix(&conf, "FOO")
	fmt.Println(err)

	os.Setenv("FOO_NAME", "foobar")
	err = envconfig.InitWithPrefix(&conf, "FOO")
	fmt.Println(err)

	fmt.Println(conf.Name)
}

Output:

envconfig: keys FOO_NAME, foo_name not found
<nil>
foobar

Types

type Options

type Options struct {
	// Prefix allows specifying a prefix for each key.
	Prefix string

	// AllOptional determines whether to not throw errors by default for any key
	// that is not found. AllOptional=true means errors will not be thrown.
	AllOptional bool

	// LeaveNil specifies whether to not create new pointers for any pointer fields
	// found within the passed config. Rather, it behaves such that if and only if
	// there is a) a non-empty field in the value or b) a non-empty value that
	// the pointer is pointing to will a new pointer be created. By default,
	// LeaveNil=false will create all pointers in all structs if they are nil.
	//
	//	var X struct {
	//		A *struct{
	//			B string
	//		}
	//	}
	//	envconfig.InitWithOptions(&X, Options{LeaveNil: true})
	//
	// $ ./program
	//
	// X.A == nil
	//
	// $ A_B="string" ./program
	//
	// X.A.B="string" // A will not be nil
	LeaveNil bool

	// AllowUnexported allows unexported fields to be present in the passed config.
	AllowUnexported bool
}

Options is used to customize the behavior of envconfig. Use it with InitWithOptions.

type Unmarshaler

type Unmarshaler interface {
	Unmarshal(s string) error
}

Unmarshaler is the interface implemented by objects that can unmarshal a environment variable string of themselves.