README
¶
mozey/config
Manage env vars with a flat key/value file. See architecture notes for more info.
By default ${ENV} == "dev", and your config file may be named .env.
The convention is to add the "config file" to .gitignore, and a "sample config file", sample.env, is versioned with your code.
For multiple environments, e.g. ${ENV} == "prod", the "config file naming convention" is ${ENV}.env, that is prod.env.
Other files types, like JSON, are also supported. The naming convention then is to add a prefix: config.${ENV}.json.
List of features
- Command to manage the env:
configu - Bash function to toggle env:
conf - Generate code (e.g.
pkg/config/config.go) to include in your Go module - And more...
File loading precedence
File loading precedence for configu command (default ${ENV} == "dev")
- config.dev.json
- config.json
- dev.env
- .env
- config.dev.yaml
- config.yaml
Quick start
Install from source
WARNING Do not run the command below inside a clone of this repo, or inside any folder this is a "go module", i.e. has a go.mod file
# Since Go 1.20.3
# "'go get' is no longer supported outside a module"
go install github.com/mozey/config/cmd/configu@latest
Create a config file
echo '{"APP_FOO": "foo", "APP_BAR": "foo"}' > config.dev.json
# This env var will be removed,
# it is not listed in the config file
export APP_VAR_NOT_IN_CONFIG_FILE="not_for_this_app"
printenv | sort | grep --color -E "APP_"
Print commands
export APP_DIR=$(pwd)
${GOPATH}/bin/configu
Reset env
eval "$(${GOPATH}/bin/configu)"
printenv | sort | grep --color -E "APP_"
Set a key value pair in config.dev.json
${GOPATH}/bin/configu -key APP_FOO -value xxx
Set a key value pair for all config.*.json and sample.config.*.json files in APP_DIR
${GOPATH}/bin/configu -all -key APP_FOO -value xxx
Convert config file to a different format
# dev.env
${GOPATH}/bin/configu -format env
# If you require only a single config file
mv dev.env .env
# config.dev.yaml
${GOPATH}/bin/configu -format yaml
Toggling env
This repo includes a conf.sh script that makes use of built-in Bash commands. The script creates a conf Bash function to set environment variables. Only .env files are supported
Download the script to your home dir
curl https://raw.githubusercontent.com/mozey/config/master/conf.sh --output ${HOME}/.conf.sh
Source the script on bash startup, e.g. ~/.bashrc, to create the conf func
source ${HOME}/.conf.sh
Use the func to toggle env
conf
conf prod
# Tip: don't create a prod config file on your dev machine!
conf stage
The default script for toggling env does not make use of the configu command. This facilitates using the workflow described in this repo on systems where the command is not installed. See toggling env with configu.
Toggling env with configu
This repo also includes a conf.configu.sh script that makes use of the configu command. The script creates a conf Bash function to set and unset environment variables.
Setup instruction as the same as for toggling env, except that a different file must be downloaded
curl https://raw.githubusercontent.com/mozey/config/master/conf.configu.sh --output ${HOME}/.conf.sh
Generate config package
The config package can be included in your app. It is useful for
- completion
- reading and setting env
- load config from file for testing
Use the -dry-run flag to print the result and skip the update
configu -generate pkg/config -dry-run
Refresh the package after adding or removing config keys
mkdir -p pkg/config
configu -generate pkg/config
go fmt ./pkg/config/config.go
Build script
Duplicate scripts/config.sh in your module.
Use the config script to
- create the dev config file
- generate the config package
APP_DIR=$(pwd) ./scripts/config.sh
Prod env
The configu cmd uses config.dev.json by default.
Create config.prod.json and set a key
cp ./sample.config.prod.json ./config.prod.json
configu -env prod -key APP_BEER -value pilsner
Export prod env
configu -env prod
eval "$(configu -env prod)"
printenv | sort | grep -E "APP_"
Compare config files and print un-matched keys
It's advisable for all config files to have the same keys, if a key does not apply to an env then set the value to an empty string. See architecture notes.
Comparing keys
configu -env dev -compare prod
# cmd exits with error code if the keys don't match
echo $?
Compare keys in config.dev.json with sample.config.dev.json
configu -env dev -compare sample.dev
Set a key value in config.prod.json.
./configu -env prod -key APP_FOO -value xxx
Dev setup
Get the code
git clone https://github.com/mozey/config.git
cd config
Reset to remove ignored files
APP_DIR=$(pwd) ./scripts/reset.sh
APP_DIR must be always be set to the module root. All config env vars must have a prefix, the default is APP_
Testing
Run the tests
APP_DIR=$(pwd) gotest -v ./...
Compare generated files in pkg/cmdconfig/testdata to pkg/cmdconfig/testdata/compare
Update testdata if required (after adding new features)
configu -generate pkg/config
cp pkg/config/config.go pkg/cmdconfig/testdata/config.go
cp sample.config.dev.json pkg/cmdconfig/testdata/config.dev.json
Debug
Create config.dev.json
cp ./sample.config.dev.json ./config.dev.json
Run the configu cmd. By default it reads config.dev.json, and prints the key/value pairs formatted as commands to export env vars
APP_DIR=$(pwd) go run cmd/configu/main.go
Build from source in working copy
go build -o ./configu ./cmd/configu
# WARNING Override installed binary
go build -o ${GOPATH}/bin/configu ./cmd/configu
Debug scripts to toggle env like this
source conf.sh && conf
source conf.configu.sh && conf
Advanced usage
The configu command can be customized
- Copy cmd/configu/main.go to your module
- Copy the code for the Main func, and make changes as per the comments
Build the configu command. The APP_DIR env var is required
export APP_DIR=$(pwd)
go build -o ${APP_DIR}/configu ./cmd/configu
Run your custom commands...
./configu
Windows
Installation work the same thanks to Go.
WARNING Do not run the command below inside a clone of this repo,
or inside any folder this is a "go module", i.e. has a go.mod file,
otherwise the install (or update to the latest tag) won't work
# Since Go 1.20.3
# "'go get' is no longer supported outside a module"
go install github.com/mozey/config/cmd/configu@latest
Depends on clink and gow, install them first. Then the following commands can be executed in the command prompt cmd.exe
Create a config file
echo {"APP_FOO": "foo", "APP_BAR": "foo"} > config.dev.json
REM This env var will be removed,
REM it is not listed in the config file
set APP_VAR_NOT_IN_CONFIG_FILE="not_for_this_app"
printenv | grep APP_
Print commands
set APP_DIR=%cd%
%GOPATH%/bin/configu
Reset env
eval "$(${GOPATH}/bin/configu)"
printenv | grep APP_
Set a key value pair in config.dev.json
%GOPATH%/bin/configu -key APP_FOO -value xxx
Toggle config, first update PATH to make conf.bat available
REM Right click start - System - Advanced system settings - Advanced - Environment Variables...
REM %GOPATH%/src/github.com/mozey/config
conf.bat
Run the tests
set APP_DIR=%cd%
gotest -v ./...
Architecture notes
The twelve-factor app stores config in environment variables, i.e. read config from the environment.
Nested config is not supported, the config file must have a flat key value structure: "env vars are granular controls, each fully orthogonal to other env vars. They are never grouped together...".
"Keep development, staging, and production as similar as possible", see dev/prod parity. See compare config files and print un-matched keys
Env vars must be set in the parent process. Apps must not set their own config, they read it from the environment.. An exception to this rule is base64 config that is compiled into, and distributed with binaries.
Above is in contrast to using Viper for reading config files from your application. In addition, while Viper has the ability to bind to flags, this repo encourages using the standard flag package or spf13/cobra. If you really need it, Viper can be used for managing config in combination with this repo. In short, Viper is very flexible, while this repo is more opinionated.
Key naming conventions
All keys must start with the same prefix.
Keys are case sensitive, and it's advised to make keys all uppercase.
Use (uppercase) SNAKE_CASE.
TODO Review this
Assuming the default key prefix APP_, to avoid un-defined behaviour when generating package code, do not start keys with
APP_EXEC_TEMPLATE_APP_FN_APP_SET_
In addition to the APP_ prefix, the configu command also supports additional prefixes like AWS_.
The APP_DIR key is set to the working directory when toggling env, any value specified for this key in the config file will be overridden
Directories