README
¶
golift.io/cnfg
Overview
Procedures for parsing config files and environment variables into data structures.
Works much like json.Unmarshal and json.Marshal.
Short explanation on how the env variable mapping works below.
See GoDoc for several working examples and
further explanation of how maps and slices can be accessed with shell env vars.
Supports all base types, including slices, maps, slices of maps, maps of slices, pointers of slices to maps of slices full of ints, strings, floats and the like!
Please open an issue if you run into a bug or an unsupported type.
Better documentation is needed. Most of it is in GoDoc. This package is full featured for environment variable parsing!
Bonus, it goes the other way too. You can convert a data structure into environment variables. This is useful when you wish to pass a lot of data into a command via exec.Command. Simply set the data into a struct, marshal it into environment variables and pass it in.
Examples
type Shelter struct {
Title string `xml:"title"`
Sym float64 `xml:"sym"`
People []*Person `xml:"people"`
Dogs []*Dog `xml:"dogs"`
}
type Person struct {
Name string `xml:"name"`
Present bool `xml:"present"`
Age int `xml:"age"`
ID int64 `xml:"id"`
}
type Dog struct {
Name string
Elapsed config.Duration
Owners []string
}
type Config struct {
*Shelter `xml:"shelter"`
}
The above struct can be configured with the following environment variables,
assuming you set prefix := "APP" when you call UnmarshalENV(). Slices use env
vars with numbers in them, starting at 0 and going to infinity, or the last env
var provided + 1, whichever comes first. It just works. The ... and ++ indicate
that those parameters belong to slices, and many items may be appended or overridden.
APP_SHELTER_TITLE
APP_SHELTER_SYM
APP_SHELTER_PEOPLE_0_NAME
APP_SHELTER_PEOPLE_0_PRESENT
APP_SHELTER_PEOPLE_0_AGE
APP_SHELTER_PEOPLE_0_ID
APP_SHELTER_PEOPLE_1_NAME
...
APP_SHELTER_PEOPLE_10_ID ++
APP_SHELTER_DOGS_0_NAME
APP_SHELTER_DOGS_0_ELAPSED
APP_SHELTER_DOGS_0_OWNERS_0
...
APP_SHELTER_DOGS_0_OWNERS_10 ++
APP_SHELTER_DOGS_1_NAME
APP_SHELTER_DOGS_1_ELAPSED
APP_SHELTER_DOGS_1_OWNERS_0
APP_SHELTER_DOGS_1_OWNERS_1 ++
If you passed in the Shelter struct instead of Config, all the of the SHELTER_
portions of the tags would be omitted. You can also set which struct tag to use by
creating an &ENV{} pointer and setting Tag and/or Pfx . Tag defaults to
"xml", but you could set it to "env" and make custom names for env variables.
The env var prefix Pfx is optional, but recommended.
Documentation
¶
Overview ¶
Package cnfg provides procedures to parse a slew of environment variables into a struct pointer.
Use this package if your app uses a config file and you want to allow users to change or override the configurations in that file with environment variables. You can also use this app just to parse environment variables; handling a config file is entirely optional. Every member type is supported. If I missed one, please open an issue. If you need a non-base type supported (net.IP works already) please open an issue. New types are extremely easy to add. If this package interests you, pull requests and feature requests are welcomed!
I consider this package the pinnacle example of how to configure Go applications from a file. You can put your configuration into any file format: XML, YAML, JSON, TOML, and you can override any struct member using an environment variable. I created this package because I got tired of writing custom env parser code for every app I make. This simplifies all the heavy lifting and I don't even have to think about it now. I hope you enjoy using this simplification as much as I do!
Index ¶
Examples ¶
Constants ¶
const ENVTag = "xml"
ENVTag is the tag to look for on struct members. You may choose to use a custom tag by creating an &ENV{} struct with a different Tag. "env" is popular, but I chose "xml" because the nouns are generally singular, and those look good as env variables. "xml" is also convenient because it's brief and doesn't add yet another struct tag. Those lines can get long quickly.
const LevelSeparator = "_"
LevelSeparator is used to separate the names from different struct levels. This is hard coded here and cannot be changed or modified.
Variables ¶
var ( ErrUnsupported = fmt.Errorf("unsupported type, please report this if this type should be supported") ErrInvalidByte = fmt.Errorf("invalid byte") ErrInvalidInterface = fmt.Errorf("can only unmarshal ENV into pointer to struct") )
Custom errors this package may produce.
Functions ¶
func UnmarshalENV ¶ added in v0.0.3
UnmarshalENV copies environment variables into configuration values. This is useful for Docker users that find it easier to pass ENV variables than a specific configuration file. Uses reflection to find struct tags.
Example ¶
Complete working example for UnmarshalENV(). Use this method when the "xml" struct tag suits your application.
package main
import (
"fmt"
"os"
"golift.io/cnfg"
)
func main() {
// Systems is used to show an example of how to access nested slices.
type System struct {
Name string `xml:"name"`
Signal []byte `xml:"signal"`
Ion *map[string]string `xml:"ion"`
}
// Config represents your application's environment variable based config inputs.
// Works with or without pointers.
type Config struct {
Users []struct {
Name string `xml:"name"`
Levels []float64 `xml:"level"`
} `xml:"user"`
Systems []*System `xml:"system"`
}
// Make a pointer to your struct. It may be empty or contain defaults.
// It may contain nested pointers, structs, maps, slices, etc. It all works.
config := &Config{}
// Okay set some ENV variables. Pretend you did this in bash.
// Setting these will overwrite any existing data. If you set a slice that
// does not exist, it has to be the _following_ index number. In other words,
// if your slice is empty, setting APP_USER_1_NAME wont work, you have to start
// with 0. If your slice len is 2, you can append by setting APP_USER_2_NAME
os.Setenv("APP_USER_0_NAME", "Tim")
os.Setenv("APP_USER_0_LEVEL_0", "1")
os.Setenv("APP_USER_0_LEVEL_1", "13")
os.Setenv("APP_USER_1_NAME", "Jon")
os.Setenv("APP_USER_1_LEVEL_0", "1")
// This adds (creates) systems and signals in sub-slices.
os.Setenv("APP_SYSTEM_0_NAME", "SysWon")
os.Setenv("APP_SYSTEM_1_NAME", "SysToo")
// With []byte you can only pass a string, and it's converted.
// You cannot access a byte member directly. Do you need to? Let me know!
os.Setenv("APP_SYSTEM_0_SIGNAL", "123456")
os.Setenv("APP_SYSTEM_1_SIGNAL", "654321")
// Maps inside slices! You can nest all you want, but your variable names may get lengthy.
fmt.Printf("BEFORE => Users: %v, Systems: %v\n", len(config.Users), len(config.Systems))
os.Setenv("APP_SYSTEM_1_ION_reactor-1", "overload")
os.Setenv("APP_SYSTEM_1_ION_reactor-2", "underload")
// Run Unmarshal to parse the values into your config pointer.
// We ignore "ok" here. You may choose to capture and it do something though.
_, err := cnfg.UnmarshalENV(config, "APP")
if err != nil {
panic(err)
}
fmt.Printf("AFTER => Users: %v\n", config.Users)
for i, s := range config.Systems {
fmt.Printf(" %v: System Name: %v, Signals: %v, Ion: %v\n", i, s.Name, s.Signal, s.Ion)
}
}
Output: BEFORE => Users: 0, Systems: 0 AFTER => Users: [{Tim [1 13]} {Jon [1]}] 0: System Name: SysWon, Signals: [49 50 51 52 53 54], Ion: <nil> 1: System Name: SysToo, Signals: [54 53 52 51 50 49], Ion: &map[reactor-1:overload reactor-2:underload]
func UnmarshalMap ¶ added in v0.0.3
UnmarshalMap parses and processes a map of key/value pairs as though they were environment variables. Useful for testing, or unmarshaling values from places other than environment variables. This version of UnmarshalMap assumes default tag ("xml") and no prefix: "".
Example ¶
package main
import (
"fmt"
"golift.io/cnfg"
)
func main() {
type myConfig struct {
Key string `xml:"envkey"`
Key2 string `xml:"envkey2"`
Nested struct {
SubSlice []string `xml:"subslice"`
SubMap map[string]string `xml:"submap"`
} `xml:"nested"`
}
// Create a pointer to unmarshal your map into.
config := &myConfig{}
// Generally you'd use MapEnvPairs() to create a map from a slice of []string.
// You can also get your data from any other source, as long as it can be
// formatted into a map.
// The important part is formatting the map keys correctly. The struct tag names
// are always upcased, but nested struct member maps are not. They can be any case.
// Each nested struct is appended to the parent name(s) with an underscore _.
// Slices (except byte slices) are accessed by their position, beginning with 0.
pairs := make(cnfg.Pairs)
pairs["ENVKEY"] = "some env value"
pairs["ENVKEY2"] = "some other env value"
pairs["NESTED_SUBSLICE_0"] = "first slice value"
pairs["NESTED_SUBMAP_mapKey"] = "first map key value"
worked, err := cnfg.UnmarshalMap(pairs, config)
if err != nil {
panic(err)
}
fmt.Printf("ok: %v, key: %v, key2: %v\n", worked, config.Key, config.Key2)
fmt.Println("map:", config.Nested.SubMap)
fmt.Println("slice:", config.Nested.SubSlice)
}
Output: ok: true, key: some env value, key2: some other env value map: map[mapKey:first map key value] slice: [first slice value]
Types ¶
type Duration ¶
Duration is useful if you need to load a time Duration from a config file into your application. Use the config.Duration type to support automatic unmarshal from all sources. If you do not use a config file, do not use this type because the environment unmarshaler supports time.Duration natively.
func (Duration) MarshalJSON ¶ added in v0.0.6
MarshalJSON returns the string representation of a Duration for JSON. ie. "1m32s".
func (Duration) MarshalText ¶ added in v0.0.6
MarshalText returns the string representation of a Duration. ie. 1m32s.
func (Duration) String ¶ added in v0.1.1
String returns a Duration as string without trailing zero units.
func (*Duration) UnmarshalText ¶
UnmarshalText parses a duration type from a config file. This method works with the Duration type to allow unmarshaling of durations from files and env variables in the same struct. You won't generally call this directly.
type ENV ¶ added in v0.0.3
type ENV struct {
Tag string // Struct tag name.
Pfx string // ENV var prefix.
Low bool // Set this false to avoid capitalizing variables.
}
ENV allows you to parse environment variables using an object instead of global state. This package allows using the default ENVTag from global state, or you can pass in your own using this struct. See the UnmarshalENV function (it's 1 line) for an example of how to use this.
func (*ENV) Marshal ¶ added in v0.1.0
Marshal deconstructs a data structure into environment variable pairs.
func (*ENV) Unmarshal ¶ added in v0.0.3
Unmarshal parses and processes environment variables into the provided interface. Uses the Prefix and Tag name from the &ENV{} struct values.
Example (Simple) ¶
Complete working example for ENV.Unmarshal().
package main
import (
"fmt"
"os"
"time"
"golift.io/cnfg"
)
func main() {
// Systems is used to show an example of how to access nested slices.
type System struct {
Name string `env:"name"`
Signal *[]int `env:"signal"`
}
// Config represents your application's environment variable based config inputs.
// Works with or without pointers.
type Config struct {
Debug bool `env:"debug"`
Users []string `env:"user"`
Interval *time.Duration `env:"interval"`
Systems []*System `env:"system"`
}
// Make a pointer to your struct with some default data.
// Maybe this data came from a config file? Using ParseFile()!
config := &Config{
Debug: true,
Users: []string{"me", "you", "them"},
Interval: nil,
Systems: nil,
}
// Okay set some ENV variables. Pretend you did this in bash.
os.Setenv("APP_DEBUG", "false") // turn off debug
os.Setenv("APP_USER_1", "dad") // replace "you" with "dad"
os.Setenv("APP_USER_3", "mom") // add "mom"
os.Setenv("APP_INTERVAL", "7m1s") // don't forget the interval!!
// This adds (creates) systems and signals in sub-slices.
os.Setenv("APP_SYSTEM_0_NAME", "SysWon")
os.Setenv("APP_SYSTEM_1_NAME", "SysToo")
os.Setenv("APP_SYSTEM_1_SIGNAL_0", "12")
// You can add as many as you like, as long as they are in numerical order.
os.Setenv("APP_SYSTEM_1_SIGNAL_1", "77")
fmt.Printf("BEFORE => Debug: %v, Interval: %v, Users: %v, Systems: %v\n",
config.Debug, config.Interval, config.Users, config.Systems)
// Make a ENV Decoder with special tag and prefix.
env := &cnfg.ENV{Tag: "env", Pfx: "APP"}
// Run Unmarshal to parse the values into your config pointer:
found, err := env.Unmarshal(config)
if err != nil {
panic(err)
}
// And optionally, do something with the "found" return value.
// If you wanted to overwrite ALL configs if ANY env variables are present
// you could use ok to make and if statement that does that.
if found {
fmt.Println("~ Environment variables were parsed into the config!")
}
// If you don't set an env variable for it, it will stay nil.
// Same for structs and slices.
if config.Interval == nil {
fmt.Printf("You forgot to set an interval!")
return
}
fmt.Printf("AFTER => Debug: %v, Interval: %v, Users: %v\n", config.Debug, *config.Interval, config.Users)
// We added some systems, check them!
for i, s := range config.Systems {
fmt.Printf(" %v: System Name: %v, Signals: %v\n", i, s.Name, s.Signal)
}
}
Output: BEFORE => Debug: true, Interval: <nil>, Users: [me you them], Systems: [] ~ Environment variables were parsed into the config! AFTER => Debug: false, Interval: 7m1s, Users: [me dad them mom] 0: System Name: SysWon, Signals: <nil> 1: System Name: SysToo, Signals: &[12 77]
func (*ENV) UnmarshalMap ¶ added in v0.0.3
UnmarshalMap parses and processes a map of key/value pairs as though they were environment variables. Useful for testing, or unmarshaling values from places other than environment variables. Use this version of UnmarshalMap if you need to change the tag or prefix.
type ENVMarshaler ¶ added in v0.1.0
ENVMarshaler allows marshaling custom types into env variables.
type ENVUnmarshaler ¶
ENVUnmarshaler allows custom unmarshaling on a custom type. If your type implements this, it will be called and the logic stops there.
Example ¶
This simple example shows how you may use the ENVUnmarshaler interface. This shows how to use two environment variables to set one custom value.
package main
import (
"fmt"
"os"
"strconv"
"time"
"golift.io/cnfg"
)
// TimeX uses two environment variables to multiply a duration.
type TimeX struct {
time.Duration
}
type AppConfig struct {
Name string `xml:"name"`
Special TimeX `xml:"in"`
}
func (t *TimeX) UnmarshalENV(tag, val string) error {
xTag := tag + "_X"
xString, ok := os.LookupEnv(xTag)
if !ok {
xString = "1"
}
multiplier, err := strconv.Atoi(xString)
if err != nil {
return fmt.Errorf("multiplier invalid %s: %w", xTag, err)
}
t.Duration, err = time.ParseDuration(val)
if err != nil {
return fmt.Errorf("duration invalid %s: %w", tag, err)
}
t.Duration *= time.Duration(multiplier)
return nil
}
func main() {
config := &AppConfig{}
os.Setenv("APP_IN", "5m")
os.Setenv("APP_IN_X", "10")
os.Setenv("APP_NAME", "myApp")
_, err := cnfg.UnmarshalENV(config, "APP")
if err != nil {
panic(err)
}
fmt.Printf("%s starts in %v", config.Name, config.Special)
}
Output: myApp starts in 50m0s
type Pairs ¶ added in v0.0.3
Pairs represents pairs of environment variables. These can be used directly or converted to other usable formats.
func MapEnvPairs ¶ added in v0.0.3
MapEnvPairs turns the pairs returned by os.Environ() into a map[string]string. Providing a prefix returns only variables with that prefix.
Example ¶
MapEnvPairs can be used when you want to inspect or modify the environment variable values before unmarshaling them.
package main
import (
"fmt"
"os"
"golift.io/cnfg"
)
func main() {
type myConfig struct {
Key string `env:"envkey"`
Key2 string `env:"envkey2"`
Key3 string `env:"envkey3"`
}
os.Setenv("TESTAPP_ENVKEY", "some env value")
os.Setenv("TESTAPP_ENVKEY2", "some other env value")
// Create pairs from the current environment.
// Only consider environment variables that begin with "TESTAPP"
pairs := cnfg.MapEnvPairs("TESTAPP", os.Environ())
for k, v := range pairs {
fmt.Println(k, v)
}
// This is the magic offered by this method.
pairs["TESTAPP_ENVKEY3"] = "add (or overwrite) a third value in code"
config := &myConfig{}
// We have to use &ENV{} to set a custom prefix, and change the struct tag.
ok, err := (&cnfg.ENV{Pfx: "TESTAPP", Tag: "env"}).UnmarshalMap(pairs, config)
if err != nil {
panic(err)
}
fmt.Printf("ok: %v, key: %v, key2: %v, key3: %v\n", ok, config.Key, config.Key2, config.Key3)
}
Output: TESTAPP_ENVKEY some env value TESTAPP_ENVKEY2 some other env value ok: true, key: some env value, key2: some other env value, key3: add (or overwrite) a third value in code
func MarshalENV ¶ added in v0.1.0
MarshalENV turns a data structure into an environment variable. The resulting slice can be copied into exec.Command.Env. Prefix is optional, and will prefix returned variables.
func (Pairs) Env ¶ added in v0.1.0
Env turns the Pairs map into an envionrment variable slice. This slice can be set to exec.Command().Env.
func (*Pairs) Get ¶ added in v0.0.3
Get allows getting only specific env variables by prefix. The prefix is trimmed before returning.
Source Files