configr

package module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 5, 2020 License: MIT Imports: 11 Imported by: 4

README

configr

Build Status Coverage Status GoDoc

Configr provides an abstraction above configuration sources, allowing you to use a single interface to expect and get all your configuration values.

Features:

  • Single interface for configuration values: Simple API (Get(), String(), Bool()...)
  • Extendable config sources: Load config from a file, database, environmental variables or any source you can get data from
  • Multiple source support: Add as many sources as you can manage, FILO merge strategy employed (first source added has highest priority)
  • Nested Key Support: production.payment_gateway.public_key production.payment_gateway.private_key
  • Value validation support: Any matching key from every source is validated by your custom validators
  • Required keys support: Ensure keys exist after parsing, otherwise error out
  • Blank config generator: Register as many keys as you need and use the blank config generator
  • Custom blank config encoder support: Implement an encoder for any data format and have a blank config generated in it
  • Type conversion support: Your config has string "5" but you want an int 5? No problem
  • Comes pre-baked with JSON, TOML file support and Environmental Variables
  • Initialise from structs: Pass a struct (optionally with values) to register keys and set defaults painlessly
  • Unmarshal straight to structs: Unmarshall the entire tree or sub-tree directly into a struct
  • Satisfies github.com/yourheropaul/inj:Datasource: Allows you to bypass the manual wiring of config values to struct properties (see below)

Built for a project at HomeMade Digital, configrs primary goal was to eliminate user error when deploying projects with heavy configuration needs. The inclusion of required key support, value validators, descriptions and blank config generator allowed us to reduce pain for seperated client ops teams when deploying our apps. Our secondary goal was flexible configuration sources be it pulling from Mongo Document, DynamoDB Table, JSON or TOML files.

Example

Register From and Unmarshal To Structs
type Email struct {
	FromAddress string `configr:"From,required"`
	Subject     string
	MaxRetries  int
	RetryOnFail bool
}

defaultEmailConfig := Email{
	RetryOnFail: true,
	MaxRetries: 3,
}
configr.RegisterFromStruct(&defaultEmailConfig)

// {
// 	"Subject": "Hello world!",
// 	"From": "configr@github.com",
// }
configr.AddSource(someSourceLikeJSON)
if err := configr.Parse(); err != nil {
	...
}

email := Email{}
if err := configr.Unmarshal(&email); err != nil {
	...
}
fmt.Println(email.RetryOnFail) // true
fmt.Println(email.MaxRetries)  // 3
fmt.Println(email.FromAddress) // configr@github.com
fmt.Println(email.Subject)     // Hello world!
Simple JSON File

Pre register some keys to expect from your configuration sources (typically via init() for small projects, or via your own object initiation system for larger projects):

configr.RequireKey("email.fromAddress", "Email from address")
configr.RequireKey("email.subject", "Email subject")
configr.RegisterKey("email.retryOnFail", "Retry sending email if it fails", false)
configr.RegisterKey("email.maxRetries", "How many times to retry email resending", 3)

Create some configuration:

{
	"email": {
		"fromAddress": "my@email.com",
		"subject": "A Subject",
		"retryOnFail": true,
		"maxRetries": 5
	}
}

Add a source:

configr.AddSource(configr.NewFile("/tmp/config.json"))

Parse your config:

if err := configr.Parse(); err != nil {
	...
}

And use at your own leisure:

fromAddress, err := configr.String("email.fromAddress")
if err != nil {
	...
}
Inj Datasource

Continuing from the simple JSON example above, you can use http://github.com/yourheropaul/inj to auto-wire in your configuration values, bypassing much of the typical config wiring boilerplate:

Pre register keys:

configr.RequireKey("email.fromAddress", "Email from address")
configr.RequireKey("email.subject", "Email subject")
configr.RegisterKey("email.retryOnFail", "Retry sending email if it fails", false)
configr.RegisterKey("email.maxRetries", "How many times to retry email resending", 3)

Add the relevant inj struct tags with their corresponding key paths:

type Email struct {
	FromAddress string `inj:"email.fromAddress"`
	Subject     string `inj:"email.subject"`
	MaxRetries  int    `inj:"email.maxRetries"`
	RetryOnFail bool   `inj:"email.retryOnFail"`
}

Add and setup your source (assume we're using the same config json as above):

configr.AddSource(configr.NewFile("/tmp/config.json"))

Parse your config:

if err := configr.Parse(); err != nil {
	...
}

Setup inj with configr as its Datasource and commence the magic:

email := Email{}
inj.Provide(&email) // Informs inj to perform DI on given instance
inj.AddDatasource(configr.GetConfigr()) // Provides inj with a datasource to query

if valid, errors := inj.Assert(); !valid { // Triggers the inj DI process
	...
}

Marvel at the ease of auto-wiring:

fmt.Println("> Email Address:", email.FromAddress)
fmt.Println("> Subject:", email.Subject)
fmt.Println("> Max Retries:", email.MaxRetries)
fmt.Println("> Retry on Fail:", email.RetryOnFail)

> Email Address: my@email.com
> Subject: A Subject
> Max Retries: 5
> Retry on Fail: true

More examples can be found in the examples/ dir.

Changes

v0.6.0

  • Add support to register and require keys directly from structs
  • Add support to unmarshal partial and full config tree into struct

v0.5.0

  • Remove KeysToUnmarshal from the Source interface, broke adapter and generally was unncessary. Functionality has been squashed into the Unmarhsal method. Any existing sources will need to be updated to accept the 2 new arguments on Unmarshal.
  • Validation errors are now wrapped so you can see the offending key
  • Moved EnvVar source into the sources/ dir

v0.4.0

  • Added new method KeysToUnmarshal to the Source interface, allows configr to tell your source what keys to expect, it also passes a key splitter func along so you can deconstruct nested keys to do as you please. See ./env_vars.go for an example. Expected to be used in instances where a source doesn't have scan like functionality and needs to know the keys to search for in advance when it unmarshals.
  • API Change: Added a new method KeysToUnmarshal to the Source interface, you'll need to add the method to any existing sources you have created, but it doesn't have to do anything. See ./file.go for an example.

v0.3.0

  • File source now supports registering encoders/decoders at a distance, check out the json and toml packages for examples
  • API Change: NewFileSource() -> NewFile()

v0.2.0

  • Add support for inj datasource

TODO:

  • Concurrent safety, particularly in multi Parse()'ing systems and when adding sources (will allow for hot reloads)
  • FileSource needs to be refactored to reduce dependency needs, something similar to sql package with a central register and blank importing the flavour you need
  • More available sources, Env vars, Flags... etc
  • Decide wether or not to ditch errors on the key getter methods (String, Get, Bool...). Alternative solution is to provide a 'Errored() bool' and 'Errors() []error or chan error' methods to Config interface. Arguments for: - Simpler interface when all you want is values Arguments against: - Error swallowing, decoupling of cause and effect (try to fetch key that cannot be converted to type ("aaa" -> Int()), user never checks configr for errors, system starts behaving weirdly) - Internal error managing will get funky in a concurrent environment, would have to use an error channel to pump the errors into, wouldn't be able to guarentee ordering or sacrafice performance for co-ordination
  • Wrap validation errors
  • Provide all primary types as getter methods
  • Add 'Keys' method to Source interface to accept keys and key name splitting func as parameters, provides keys for lookup for Sources that don't have 'scan' style interfaces, and potential performance improvements

Documentation

Overview

Configr provides an abstraction above configuration sources, allowing you to use a single interface to get all your configuration values

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrKeyNotFound          = errors.New("configr: Key not found")
	ErrParseHasntBeenCalled = errors.New("configr: Trying to get values before calling Parse()")
	ErrNoRegisteredValues   = errors.New("configr: No registered values to generate")
)
View Source 
var (
	RegisteredFileEncoders = make(map[string]Encoder)
	RegisteredFileDecoders = make(map[string]FileDecoder)
	ExtensionToDecoderName = make(map[string]string)
	ExtensionToEncoderName = make(map[string]string)

	ErrUnknownEncoding = errors.New("configr: Unable to determine file encoding, please set manually")
)

Functions

func AddSource 

func AddSource(p Source)

AddSource registers Sources with the Configr instance to Unmarshal() when Parse() is called. Sources are parsed in a FILO order, meaning the first source added is considered the highest priority, and any keys from lower priority sources that are present in a higher will be overwritten

func Bool 

func Bool(key string) (bool, error)

Bool wraps Get() and will attempt to cast the resulting value to a bool or error

func Float64 

func Float64(key string) (float64, error)

Float64 wraps Get() and will attempt to cast the resulting value to a float64 or error

func GenerateBlank 

func GenerateBlank(e Encoder) ([]byte, error)

GenerateBlank generates a 'blank' configuration using the passed Encoder, it will honour nested keys, use default values where possible and when not fall back to placing the description as the value.

func Get 

func Get(key string) (interface{}, error)

Get can only be called after a Parse() has been done. Keys support the nested notation format: 

"user.age.month"

If a key is not found but has been registered with a default, the default will be returned

func Int 

func Int(key string) (int, error)

Int wraps Get() and will attempt to cast the resulting value to a int or error

func MustParse 

func MustParse()

MustParse wraps Parse() and will panic if there are any resulting errors

func Parse 

func Parse() error

Parse calls Unmarshal on all registered sources, and caches the subsequent key/value's. Additional calls to Parse can be made to add additional config from sources.

Sources are called in a FILO order, meaning the first source added is considered the highest priority, any keys set from lower priority sources found in higher priority will be overwritten.

func Parsed 

func Parsed() bool

Parsed lets the caller know if a Parse() call has been made or not

func RegisterFileDecoder added in v0.3.0 

func RegisterFileDecoder(name string, source FileDecoder, fileExtensions ...string)

func RegisterFileEncoder added in v0.3.0 

func RegisterFileEncoder(name string, encoder Encoder, fileExtensions ...string)

func RegisterFromStruct added in v0.6.0 

func RegisterFromStruct(structPtr interface{}, fieldToKeyFunc ...NameToKeyFunc) error

func RegisterKey 

func RegisterKey(name, description string, defaultVal interface{}, validators ...Validator)

RegisterKey registers a configuration key (name) along with a description of what the configuration key is for, a default value and optional validators

name supports nested notation in the form of '.' delimitered keys (unless changed) e.g. 

"user.age.month"

func RequireKey 

func RequireKey(name, description string, validators ...Validator)

RequireValue wraps the RegisterValue() call but upon parsing sources, if the configuration key (name) is not found, Parse() will return a ErrRequiredValuesMissing error

func SetConfigr 

func SetConfigr(c *Configr)

func String 

func String(key string) (string, error)

String wraps Get() and will attempt to cast the resulting value to a string or error

func ToLowerCamelCase added in v0.6.0 

func ToLowerCamelCase(s string) string

func Unmarshal 

func Unmarshal(destination interface{}) error

Unmarshals all parsed values into struct, uses `configr` struct tag for alternative property name. e.g. 

type Example struct {
    property1 string `configr:"myproperty1"`
}

func UnmarshalKey added in v0.6.0 

func UnmarshalKey(key string, destination interface{}) error

UnmarshalKey unmarshals a subtree of parsed values from Key into a struct. Key follows the same rules as Get.

Types

type Config 

type Config interface {
	Parse() error
	Parsed() bool
	MustParse()

	Get(string) (interface{}, error)

	String(string) (string, error)
	Bool(string) (bool, error)
	Int(string) (int, error)
	Float64(string) (float64, error)

	Unmarshal(interface{}) error
	UnmarshalKey(string, interface{}) error
}

type Configr 

type Configr struct {
	// contains filtered or unexported fields
}

func GetConfigr 

func GetConfigr() *Configr

func New 

func New() *Configr

func (*Configr) AddSource 

func (c *Configr) AddSource(p Source)

func (*Configr) Bool 

func (c *Configr) Bool(key string) (bool, error)

func (*Configr) Float64 

func (c *Configr) Float64(key string) (float64, error)

func (*Configr) GenerateBlank 

func (c *Configr) GenerateBlank(e Encoder) ([]byte, error)

func (*Configr) Get 

func (c *Configr) Get(key string) (interface{}, error)

func (*Configr) Int 

func (c *Configr) Int(key string) (int, error)

func (*Configr) MustParse 

func (c *Configr) MustParse()

func (*Configr) Parse 

func (c *Configr) Parse() error

func (*Configr) Parsed 

func (c *Configr) Parsed() bool

func (*Configr) Read added in v0.2.0 

func (c *Configr) Read(key string) (interface{}, error)

Satisfies github.com/yourheropaul/inj:DatasourceReader interface, not intended for regular configr usage. Use `configr.Get(string)` instead.

func (*Configr) RegisterFromStruct added in v0.6.0 

func (c *Configr) RegisterFromStruct(structPtr interface{}, fieldToKeyFunc ...NameToKeyFunc) error

func (*Configr) RegisterKey 

func (c *Configr) RegisterKey(name, description string, defaultVal interface{}, validators ...Validator)

func (*Configr) RequireKey 

func (c *Configr) RequireKey(name, description string, validators ...Validator)

func (*Configr) SetDescriptionWrapper 

func (c *Configr) SetDescriptionWrapper(wrapper string)

func (*Configr) SetIsCaseSensitive 

func (c *Configr) SetIsCaseSensitive(isCaseSensitive bool)

func (*Configr) SetKeyPathDelimeter 

func (c *Configr) SetKeyPathDelimeter(delimeter string)

func (*Configr) String 

func (c *Configr) String(key string) (string, error)

func (*Configr) Unmarshal added in v0.6.0 

func (c *Configr) Unmarshal(destination interface{}) error

func (*Configr) UnmarshalKey added in v0.6.0 

func (c *Configr) UnmarshalKey(key string, destination interface{}) error

type Encoder 

type Encoder interface {
	Marshal(interface{}) ([]byte, error)
}

Encoder would be used to encode registered and required values (along with their defaults or descriptions) into bytes.

type EncoderAdapter added in v0.2.0 

type EncoderAdapter func(interface{}) ([]byte, error)

EncoderAdapter allows you to convert a func: 

func(interface{}) ([]byte, error)

into a type that satisfies the Encoder interface

func (EncoderAdapter) Marshal added in v0.2.0 

func (f EncoderAdapter) Marshal(v interface{}) ([]byte, error)

type ErrRequiredKeysMissing 

type ErrRequiredKeysMissing []string

func (ErrRequiredKeysMissing) Error 

func (e ErrRequiredKeysMissing) Error() string

type File added in v0.3.0 

type File struct {
	// contains filtered or unexported fields
}

func NewFile added in v0.3.0 

func NewFile(path string) *File

func (*File) Marshal added in v0.3.0 

func (f *File) Marshal(v interface{}) ([]byte, error)

func (*File) Path added in v0.3.0 

func (f *File) Path() string

func (*File) SetEncodingName added in v0.3.0 

func (f *File) SetEncodingName(name string)

func (*File) SetPath added in v0.3.0 

func (f *File) SetPath(path string)

func (*File) Unmarshal added in v0.3.0 

func (f *File) Unmarshal(_ []string, _ KeySplitter) (map[string]interface{}, error)

type FileDecoder added in v0.3.0 

type FileDecoder interface {
	Unmarshal([]byte, interface{}) error
}

type FileDecoderAdapter added in v0.3.0 

type FileDecoderAdapter func([]byte, interface{}) error

func (FileDecoderAdapter) Unmarshal added in v0.3.0 

func (f FileDecoderAdapter) Unmarshal(b []byte, v interface{}) error

type InvalidTypeError added in v0.6.0 

type InvalidTypeError struct {
	Type reflect.Type
}

func (InvalidTypeError) Error added in v0.6.0 

func (e InvalidTypeError) Error() string

type KeySplitter added in v0.4.0 

type KeySplitter func(string) []string

KeySplitter is a function that takes a key path and splits it into its sub-parts: 

In: "person.height.inches"
Out: []string("person", "height", "inches")

func NewKeySplitter added in v0.5.0 

func NewKeySplitter(delimeter string) KeySplitter

type Manager 

type Manager interface {
	RegisterFromStruct(interface{}, ...NameToKeyFunc) error

	RegisterKey(string, string, interface{}, ...Validator)
	RequireKey(string, string, ...Validator)

	AddSource(Source)

	GenerateBlank(Encoder) ([]byte, error)
	SetIsCaseSensitive(bool)
}

type NameToKeyFunc added in v0.6.0 

type NameToKeyFunc func(string) string

Converts a Struct Name or Field into an appropriate key name: 

In: "FromAddress"
Out: "fromAddress"

type Source 

type Source interface {
	Unmarshal([]string, KeySplitter) (map[string]interface{}, error)
}

Source is a source of configuration keys and values, calling unmarshal should return a map[string]interface{} of all key/value pairs (nesting is supported) with multiple types. First arg is a slice of all expected keys.

type SourceAdapter 

type SourceAdapter func([]string, KeySplitter) (map[string]interface{}, error)

SourceAdapter allows you to convert a func: 

func() (map[string]interface{}, error)

into a type that satisfies the Source interface

func (SourceAdapter) Unmarshal 

func (f SourceAdapter) Unmarshal(keys []string, keySplitterFn KeySplitter) (map[string]interface{}, error)

type ValidationError added in v0.5.0 

type ValidationError struct {
	Key string
	Err error
}

func NewValidationError added in v0.5.0 

func NewValidationError(key string, err error) ValidationError

func (ValidationError) Error added in v0.5.0 

func (v ValidationError) Error() string

type Validator 

type Validator func(interface{}) error

Validator is a validation function which would be coupled with a configuration key, anytime the config key is found in a Source it's value is validated.

Source Files

View all Source files

Directories

Path Synopsis
examples
from-struct
simple-envvar
simple-json
simple-json-inj
file/json
file/toml

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.