conf

package module
v1.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 3, 2020 License: MIT Imports: 25 Imported by: 0

README

conf GoDoc Test Status

Go package for loading program configuration from multiple sources.

Motivations

Loading program configurations is usually done by parsing the arguments passed to the command line, and in this case the standard library offers a good support with the flag package.
However, there are times where the standard is just too limiting, for example when the program needs to load configuration from other sources (like a file, or the environment variables).
The conf package was built to address these issues, here were the goals:

  • Loading the configuration has to be type-safe, there were other packages available that were covering the same use-cases but they often required doing type assertions on the configuration values which is always an opportunity to get the program to panic.

  • Keeping the API minimal, while the flag package offered the type safety we needed it is also very verbose to setup. With conf, only a single function call is needed to setup and load the entire program configuration.

  • Supporting richer syntaxes, because program configurations are often generated dynamically, the conf package accepts YAML values as input to all configuration values. It also has support for sub-commands on the command line, which is a common approach used by CLI tools.

  • Supporting multiple sources, because passing values through the command line is not always the best appraoch, programs may need to receive their configuration from files, environment variables, secret stores, or other network locations.

Basic Usage

A program using the conf package needs to declare a struct which is passed to conf.Load to populate the fields with the configuration that was made available at runtime through a configuration file, environment variables or the program arguments.

Each field of the structure may declare a conf tag which sets the name of the property, and a help tag to provide a help message for the configuration.

The conf package will automatically understand the structure of the program configuration based on the struct it receives, as well as generating the program usage and help messages if the -h or -help options are passed (or an error is detected).

The conf.Load function adds support for a --config-file option on the program arguments which accepts the path to a file that the configuration may be loaded from as well.

Here's an example of how a program would typically use the package:

package main

import (
    "fmt"

    "github.com/andrew-d/conf"
)

func main() {
    var config struct {
        Message string `conf:"m" help:"A message to print."`
    }

    // Load the configuration, either from a config file, the environment or the program arguments.
    conf.Load(&config)

    fmt.Println(config.Message)
}

$ go run ./example.go -m 'Hello World!'
Hello World!

Environment Variables

By default, conf will look for environment variables before loading command-line configuration flags with one important caveat: environment variables are prefixed with the program name. For example, given a program named "foobar":

func main() {
	config := struct {
		Name string `conf:"name"`
	}{
		Name: "default",
	}
	conf.Load(&config)
	fmt.Println("Hello", config.Name)
}

The following will be output:

$ ./foobar                                    // "Hello default"
$ FOOBAR_NAME=world ./foobar                  // "Hello world"
$ FOOBAR_NAME=world ./foobar --name neighbor  // "Hello neighbor"
$ MAIN_NAME=world go run main.go              // "Hello world"

If you want to hard-code the prefix to guarantee immutability or just to customize it, you can supply a custom loader config:

loader := conf.Loader{
	Name: "my-service",
	Args: os.Args[1:],
	Sources: []conf.Source{
		conf.NewEnvSource("MY_SVC", os.Environ()...),
	},
}
conf.LoadWith(&config, loader)

Advanced Usage

While the conf.Load function is good enough for common use cases, programs sometimes need to customize the default behavior.
A program may then use the conf.LoadWith function, which accepts a conf.Loader as second argument to gain more control over how the configuration is loaded.

Here's the conf.Loader definition:

package conf

type Loader struct {
     Name     string    // program name
     Usage    string    // program usage
     Args     []string  // list of arguments
     Commands []Command // list of commands
     Sources  []Source  // list of sources to load configuration from.
}

The conf.Load function is actually just a wrapper around conf.LoadWith that passes a default loader. The default loader gets the program name from the first program argument, supports no sub-commands, and has two custom sources setup to potentially load its configuration from a configuration file or the environment variables.

Here's an example showing how to configure a CLI tool that supports a couple of sub-commands:

package main

import (
    "fmt"

    "github.com/andrew-d/conf"
)

func main() {
    // If nil is passed instead of a configuration struct no arguments are
    // parsed, only the command is extracted.
    cmd, args := conf.LoadWith(nil, conf.Loader{
        Name:     "example",
        Args:     os.Args[1:],
        Commands: []conf.Command{
            {"print", "Print the message passed to -m"},
            {"version", "Show the program version"},
        },
    })

    switch cmd {
    case "print":
        var config struct{
            Message string `conf:"m" help:"A message to print."`
        }

        conf.LoadWith(&config, conf.Loader{
            Name: "example print",
            Args: args,
        })

        fmt.Println(config.Message)

    case "version":
        fmt.Println("1.2.3")
    }
}

$ go run ./example.go version
1.2.3
$ go run ./example.go print -m 'Hello World!'
Hello World!

Custom Sources

We mentionned the conf.Loader type supported setting custom sources that the program configuration can be loaded from. Here's the the conf.Source interface definition:

package conf

type Source interface {
    Load(dst Map)
}

The source has a single method which receives a conf.Map value which is an itermediate representation of the configuration struct that was received by the loader.
The package uses this type internally as well for loading configuration values from the program arguments, it can be seen as a reflective representiong of the original value which exposes an API that is more convenient to use that having a raw reflect.Value.

One of the advantages of the conf.Map type is that it implements the objconv.ValueDecoder interface and therefore can be used directly to load configurations from a serialized format (like JSON for example).

Validation

Last but not least, the conf package also supports automatic validation of the fields in the configuration struct. This happens after the values were loaded and is based on gopkg.in/validator.v2.

This step could have been done outside the package however it is both convenient and useful to have all configuration errors treated the same way (getting the usage and help message shown when something is wrong).

Documentation

Overview

Package conf package provides tools for easily loading program configurations from multiple sources such as the command line arguments, environment, or a configuration file.

Most applications only need to use the Load function to get their settings loaded into an object. By default, Load will read from a configurable file defined by the -config-file command line argument, load values present in the environment, and finally load the program arguments.

The object in which the configuration is loaded must be a struct, the names and types of its fields are introspected by the Load function to understand how to load the configuration.

The name deduction from the struct field obeys the same rules than those implemented by the standard encoding/json package, which means the program can set the "conf" tag to override the default field names in the command line arguments and configuration file.

A "help" tag may also be set on the fields of the configuration object to add documentation to the setting, which will be shown when the program is asked to print its help.

When values are loaded from the environment the Load function looks for variables matching the struct fields names in snake-upper-case form.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// Modifier is the default modification lib using the "mod" tag; it is
	// exposed to allow registering of custom modifiers and aliases or to
	// be set to a more central instance located in another repo.
	Modifier = modifiers.New()
)

Functions

func EqualNode 

func EqualNode(n1 Node, n2 Node) bool

EqualNode compares n1 and n2, returning true if they are deeply equal.

func Load 

func Load(cfg interface{}) (args []string)

Load the program's configuration into cfg, and returns the list of leftover arguments.

The cfg argument is expected to be a pointer to a struct type where exported fields or fields with a "conf" tag will be used to load the program configuration. The function panics if cfg is not a pointer to struct, or if it's a nil pointer.

The configuration is loaded from the command line, environment and optional configuration file if the -config-file option is present in the program arguments.

Values found in the progrma arguments take precedence over those found in the environment, which takes precedence over the configuration file.

If an error is detected with the configurable the function print the usage message to stderr and exit with status code 1.

func LoadWith 

func LoadWith(cfg interface{}, ld Loader) (cmd string, args []string)

LoadWith behaves like Load but uses ld as a loader to parse the program configuration.

The function panics if cfg is not a pointer to struct, or if it's a nil pointer and no commands were set.

func SetPPROF 

func SetPPROF(config PPROF)

SetPPROF configures the runtime profilers based on the given PPROF config.

Types

type Array 

type Array struct {
	// contains filtered or unexported fields
}

Array is a node type that wraps a slice value.

func (Array) DecodeValue 

func (a Array) DecodeValue(d objconv.Decoder) (err error)

func (Array) EncodeValue 

func (a Array) EncodeValue(e objconv.Encoder) (err error)

func (Array) Item 

func (a Array) Item(i int) Node

func (Array) Items 

func (a Array) Items() []Node

func (Array) Kind 

func (a Array) Kind() NodeKind

func (Array) Len 

func (a Array) Len() int

func (Array) Set 

func (a Array) Set(s string) error

func (Array) String 

func (a Array) String() string

func (Array) Type added in v1.3.0 

func (a Array) Type() string

func (Array) Value 

func (a Array) Value() interface{}

type Command 

type Command struct {
	Name string // name of the command
	Help string // help message describing what the command does
}

A Command represents a command supported by a configuration loader.

type FlagSource 

type FlagSource interface {
	Source

	// Flag is the name of the flag that sets the source's configuration value.
	Flag() string

	// Help is called to get the help message to display for the source's flag.
	Help() string

	// flag.Value must be implemented by a FlagSource to receive their value
	// when the loader's arguments are parsed.
	flag.Value
}

FlagSource is a special case of a source that receives a configuration value from the arguments of a loader. It makes it possible to provide runtime configuration to the source from the command line arguments of a program.

func NewFileSource 

func NewFileSource(flag string, vars interface{}, readFile func(string) ([]byte, error), unmarshal func([]byte, interface{}) error) FlagSource

NewFileSource creates a new source which loads a configuration from a file identified by a path (or URL).

The returned source satisfies the FlagSource interface because it loads the file location from the given flag.

The vars argument may be set to render the configuration file if it's a template.

The readFile function loads the file content in-memory from a file location given as argument, usually this is ioutil.ReadFile.

The unmarshal function decodes the content of the configuration file into a configuration object.

type Loader 

type Loader struct {
	Name     string    // program name
	Usage    string    // program usage
	Args     []string  // list of arguments
	Commands []Command // list of commands
	Sources  []Source  // list of sources to load configuration from.
}

A Loader exposes an API for customizing how a configuration is loaded and where it's loaded from.

func (Loader) FprintError 

func (ld Loader) FprintError(w io.Writer, err error)

FprintError outputs the error message for err to w.

func (Loader) FprintHelp 

func (ld Loader) FprintHelp(w io.Writer, cfg interface{})

FprintHelp outputs the help message for cfg to w.

func (Loader) Load 

func (ld Loader) Load(cfg interface{}) (cmd string, args []string, err error)

Load uses the loader ld to load the program configuration into cfg, and returns the list of program arguments that were not used.

The function returns flag.ErrHelp when the list of arguments contained -h, -help, or --help.

The cfg argument is expected to be a pointer to a struct type where exported fields or fields with a "conf" tag will be used to load the program configuration. The function panics if cfg is not a pointer to struct, or if it's a nil pointer and no commands were set.

func (Loader) PrintError 

func (ld Loader) PrintError(err error)

PrintError outputs the error message for err to stderr.

func (Loader) PrintHelp 

func (ld Loader) PrintHelp(cfg interface{})

PrintHelp outputs the help message for cfg to stderr.

type Map 

type Map struct {
	// contains filtered or unexported fields
}

Map is a map type that wraps a map or struct value.

func (Map) DecodeValue 

func (m Map) DecodeValue(d objconv.Decoder) error

func (Map) EncodeValue 

func (m Map) EncodeValue(e objconv.Encoder) error

func (Map) Item 

func (m Map) Item(name string) Node

func (Map) Items 

func (m Map) Items() []MapItem

func (Map) Kind 

func (m Map) Kind() NodeKind

func (Map) Len 

func (m Map) Len() int

func (Map) Scan 

func (m Map) Scan(do func([]string, MapItem))

func (Map) Set 

func (m Map) Set(s string) error

func (Map) String 

func (m Map) String() string

func (Map) Type added in v1.3.0 

func (m Map) Type() string

func (Map) Value 

func (m Map) Value() interface{}

type MapItem 

type MapItem struct {
	Name  string
	Short string
	Help  string
	Value Node
}

MapItem is the type of elements stored in a Map.

type Node 

type Node interface {
	flag.Value
	objconv.ValueEncoder
	objconv.ValueDecoder

	// Kind returns the NodeKind of the configuration node.
	Kind() NodeKind

	// Value returns the underlying value wrapped by the configuration node.
	Value() interface{}
}

The Node interface defines the common interface supported by the different types of configuration nodes supported by the conf package.

func MakeNode 

func MakeNode(v interface{}) Node

MakeNode builds a Node from the value v.

The function panics if v contains unrepresentable values.

type NodeKind 

type NodeKind int

NodeKind is an enumeration which describes the different types of nodes that are supported in a configuration. 

const (
	// ScalarNode represents configuration nodes of type Scalar.
	ScalarNode NodeKind = iota

	// ArrayNode represents configuration nodes of type Array.
	ArrayNode

	// MapNode represents configuration nodes of type Map.
	MapNode
)

type PPROF 

type PPROF struct {
	BlockProfileRate     int `` /* 164-byte string literal not displayed */
	MutexProfileFraction int `` /* 164-byte string literal not displayed */
}

PPROF is a configuration struct which can be used to configure the runtime profilers of programs. 

config := struct{
	PPROF `conf:"pprof"`
}{
	PPROF: conf.DefaultPPROF(),
}
conf.Load(&config)
conf.SetPPROF(config.PPROF)

func DefaultPPROF 

func DefaultPPROF() PPROF

DefaultPPROF returns the default value of a PPROF struct. Note that the zero-value is valid, DefaultPPROF differs because it captures the current configuration of the program's runtime.

type Scalar 

type Scalar struct {
	// contains filtered or unexported fields
}

A Scalar is a node type that wraps a basic value.

func (Scalar) DecodeValue 

func (s Scalar) DecodeValue(d objconv.Decoder) error

func (Scalar) EncodeValue 

func (s Scalar) EncodeValue(e objconv.Encoder) error

func (Scalar) IsBoolFlag 

func (s Scalar) IsBoolFlag() bool

func (Scalar) Kind 

func (s Scalar) Kind() NodeKind

func (Scalar) Set 

func (s Scalar) Set(str string) (err error)

func (Scalar) String 

func (s Scalar) String() string

func (Scalar) Type added in v1.3.0 

func (s Scalar) Type() string

func (Scalar) Value 

func (s Scalar) Value() interface{}

type Source 

type Source interface {
	Load(dst Map) error
}

Source is the interface that allow new types to be plugged into a loader to make it possible to load configuration from new places.

When the configuration is loaded the Load method of each source that was set on a loader is called with an Node representating the configuration struct. The typical implementation of a source is to load the serialized version of the configuration and use an objconv decoder to build the node.

func NewEnvSource 

func NewEnvSource(prefix string, env ...string) Source

NewEnvSource creates a new source which loads values from the environment variables given in env.

A prefix may be set to namespace the environment variables that the source will be looking at.

type SourceFunc 

type SourceFunc func(dst Map) error

SourceFunc makes it possible to use basic function types as configuration sources.

func (SourceFunc) Load 

func (f SourceFunc) Load(dst Map) error

Load calls f.

Source Files

View all Source files

Directories

Path Synopsis
This example program can be used to test the features provided by the github.com/andrew-d/conf package.
 This example program can be used to test the features provided by the github.com/andrew-d/conf package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.