config

package
v1.0.0-beta2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 10, 2017 License: MIT Imports: 16 Imported by: 0

README

Configuration Package

At a high level, configuration is any data that is used in an application but not part of the application itself. Any reasonably complex system needs to have knobs to tune, and not everything can have intelligent defaults.

In UberFx, we try very hard to make configuring UberFx convenient. Users can:

  • Get components working with minimal configuration
  • Override any field if the default doesn't make sense for their use case

Nesting

The configuration system wraps a set of providers that each know how to get values from an underlying source:

  • Static YAML configuration
  • Environment variables

So by stacking these providers, we can have a priority system for defining configuration that can be overridden by higher priority providers. For example, the static YAML configuration would be the lowest priority and those values should be overridden by values specified as environment variables.

As an example, imagine a YAML config that looks like:

foo:
  bar:
    boo: 1
    baz: hello

stuff:
  server:
    port: 8081
    greeting: Hello There!

UberFx Config allows direct key access, such as foo.bar.baz:

cfg := svc.Config()
if value := cfg.Get("foo.bar.baz"); value.HasValue() {
  fmt.Printf("Say %s", value.AsString()) // "Say hello"
}

Or via a strongly typed structure, even as a nest value, such as:

type myStuff struct {
  Port     int    `yaml:"port" default:"8080"`
  Greeting string `yaml:"greeting"`
}

// ....

target := &myStuff{}
cfg := svc.Config()
if err := cfg.Get("stuff.server").PopulateStruct(target); err != nil {
  // fail, we didn't find it.
}

fmt.Printf("Port is: %v", target.Port)

Prints Port is 8081

This model respects priority of providers to allow overriding of individual values. In this example, we override the server port via an environment variable:

export CONFIG__stuff__server__port=3000

Then running the above example will result in Port is 3000

Provider

Provider is the interface for anything that can provide values. We provide a few reference implementations (environment and YAML), but you are free to register your own providers via config.RegisterProviders() and config.RegisterDynamicProviders.

Static configuration providers

Static configuration providers conform to the Provider interface and are bootstrapped first. Use these for simple providers such as file-backed or environment-based configuration providers.

Dynamic Configuration Providers

Dynamic configuration providers frequently need some bootstrap configuration to be useful, so UberFx treats them specially. Dynamic configuration providers conform to the Provider interface, but they're instantiated after the Static Providers on order to read bootstrap values.

For example, if you were to implement a ZooKeeper-backed Provider, you'd likely need to specify (via YAML or environment variables) where your ZooKeeper nodes live.

Value

Value is the return type of every configuration providers' Get(key string) method. Under the hood, we use the empty interface (interface{}) since we don't necessarily know the structure of your configuration ahead of time.

You can use a Value for two main purposes:

  • Get a single value out of configuration.

For example, if we have a YAML configuration like so:

one:
  two: hello

You could access the value using "dotted notation":

foo := provider.Get("one.two").AsString()
fmt.Println(foo)
// Output: hello

To get an access to the root element use config.Root:

root := provider.Get(config.Root).AsString()
fmt.Println(root)
// Output: map[one:map[two:hello]]
  • Populate a struct (PopulateStruct(&myStruct))

The As* method has two variants: TryAs* and As*. The former is a two-value return, similar to a type assertion, where the user checks if the second bool is true before using the value.

The As* methods are similar to the Must* pattern in the standard library. If the underlying value cannot be converted to the requested type, As* will panic.

PopulateStruct

PopulateStruct is akin to json.Unmarshal() in that it takes a pointer to a custom struct and fills in the fields. It returns a true if the requested fields were found and populated properly, and false otherwise.

For example, say we have the following YAML file:

hello:
  world: yes
  number: 42

We could deserialize into our custom type with the following code:

type myConfig struct {
  World  string
  Number int
}

m := myConfig{}
provider.Get("hello").Populate(&m)
fmt.Println(m.World)
// Output: yes

Note that any fields you wish to deserialize into must be exported, just like json.Unmarshal and friends.

Benchmarks

Current performance benchmark data:

BenchmarkYAMLCreateSingleFile-8                    	   50000	     31345 ns/op	   11144 B/op	     122 allocs/op
BenchmarkYAMLCreateMultiFile-8                     	   30000	     51927 ns/op	   20080 B/op	     207 allocs/op
BenchmarkYAMLSimpleGetLevel1-8                     	50000000	        26.8 ns/op	       0 B/op	       0 allocs/op
BenchmarkYAMLSimpleGetLevel3-8                     	50000000	        27.3 ns/op	       0 B/op	       0 allocs/op
BenchmarkYAMLSimpleGetLevel7-8                     	50000000	        26.2 ns/op	       0 B/op	       0 allocs/op
BenchmarkYAMLPopulateStruct-8                      	 1000000	      1598 ns/op	     576 B/op	      16 allocs/op
BenchmarkYAMLPopulateStructNested-8                	  300000	      3972 ns/op	    1056 B/op	      42 allocs/op
BenchmarkYAMLPopulateStructNestedMultipleFiles-8   	  300000	      5013 ns/op	    1248 B/op	      52 allocs/op
BenchmarkYAMLPopulateNestedTextUnmarshaler-8       	  100000	     18209 ns/op	    3368 B/op	     211 allocs/op

Documentation

Overview

Package config is the Configuration Package.

At a high level, configuration is any data that is used in an application but not part of the application itself. Any reasonably complex system needs to have knobs to tune, and not everything can have intelligent defaults.

In UberFx, we try very hard to make configuring UberFx convenient. Users can:

• Get components working with minimal configuration

• Override any field if the default doesn't make sense for their use case

Nesting

The configuration system wraps a set of *providers* that each know how to get values from an underlying source:

• Static YAML configuration

• Environment variables

So by stacking these providers, we can have a priority system for defining configuration that can be overridden by higher priority providers. For example, the static YAML configuration would be the lowest priority and those values should be overridden by values specified as environment variables.

As an example, imagine a YAML config that looks like: 

foo:
  bar:
    boo: 1
    baz: hello

stuff:
  server:
    port: 8081
    greeting: Hello There!

UberFx Config allows direct key access, such as foo.bar.baz: 

cfg := svc.Config()
if value := cfg.Get("foo.bar.baz"); value.HasValue() {
  fmt.Printf("Say %s", value.AsString()) // "Say hello"
}

Or via a strongly typed structure, even as a nest value, such as: 

type myStuff struct {
  Port     int    `yaml:"port" default:"8080"`
  Greeting string `yaml:"greeting"`
}

// ....

target := &myStuff{}
cfg := svc.Config()
if err := cfg.Get("stuff.server").PopulateStruct(target); err != nil {
  // fail, we didn't find it.
}

fmt.Printf("Port is: %v", target.Port)

Prints **Port is 8081**

This model respects priority of providers to allow overriding of individual values. In this example, we override the server port via an environment variable: 

export CONFIG__stuff__server__port=3000

Then running the above example will result in **Port is 3000**

Provider

Provider is the interface for anything that can provide values. We provide a few reference implementations (environment and YAML), but you are free to register your own providers via config.RegisterProviders() and config.RegisterDynamicProviders.

Static configuration providers

Static configuration providers conform to the Provider interface and are bootstrapped first. Use these for simple providers such as file-backed or environment-based configuration providers.

Dynamic Configuration Providers

Dynamic configuration providers frequently need some bootstrap configuration to be useful, so UberFx treats them specially. Dynamic configuration providers conform to the Provider interface, but they're instantiated **after** the Static Providers on order to read bootstrap values.

For example, if you were to implement a ZooKeeper-backed Provider, you'd likely need to specify (via YAML or environment variables) where your ZooKeeper nodes live.

Value

Value is the return type of every configuration providers' Get(key string) method. Under the hood, we use the empty interface ( interface{}) since we don't necessarily know the structure of your configuration ahead of time.

You can use a Value for two main purposes:

• Get a single value out of configuration.

For example, if we have a YAML configuration like so: 

one:
  two: hello

You could access the value using "dotted notation": 

foo := provider.Get("one.two").AsString()
fmt.Println(foo)
// Output: hello

To get an access to the root element use config.Root: 

root := provider.Get(config.Root).AsString()
fmt.Println(root)
// Output: map[one:map[two:hello]]

• Populate a struct (PopulateStruct(&myStruct))

The As* method has two variants: TryAs* and As*. The former is a two-value return, similar to a type assertion, where the user checks if the second bool is true before using the value.

The As* methods are similar to the Must* pattern in the standard library. If the underlying value cannot be converted to the requested type, As* will panic.

PopulateStruct

PopulateStruct is akin to json.Unmarshal() in that it takes a pointer to a custom struct and fills in the fields. It returns a true if the requested fields were found and populated properly, and false otherwise.

For example, say we have the following YAML file: 

hello:
  world: yes
  number: 42

We could deserialize into our custom type with the following code: 

type myConfig struct {
  World  string
  Number int
}

m := myConfig{}
provider.Get("hello").Populate(&m)
fmt.Println(m.World)
// Output: yes

Note that any fields you wish to deserialize into must be exported, just like json.Unmarshal and friends.

Benchmarks

Current performance benchmark data: 

BenchmarkYAMLCreateSingleFile-8                    	   50000	     31345 ns/op	   11144 B/op	     122 allocs/op
BenchmarkYAMLCreateMultiFile-8                     	   30000	     51927 ns/op	   20080 B/op	     207 allocs/op
BenchmarkYAMLSimpleGetLevel1-8                     	50000000	        26.8 ns/op	       0 B/op	       0 allocs/op
BenchmarkYAMLSimpleGetLevel3-8                     	50000000	        27.3 ns/op	       0 B/op	       0 allocs/op
BenchmarkYAMLSimpleGetLevel7-8                     	50000000	        26.2 ns/op	       0 B/op	       0 allocs/op
BenchmarkYAMLPopulateStruct-8                      	 1000000	      1598 ns/op	     576 B/op	      16 allocs/op
BenchmarkYAMLPopulateStructNested-8                	  300000	      3972 ns/op	    1056 B/op	      42 allocs/op
BenchmarkYAMLPopulateStructNestedMultipleFiles-8   	  300000	      5013 ns/op	    1248 B/op	      52 allocs/op
BenchmarkYAMLPopulateNestedTextUnmarshaler-8       	  100000	     18209 ns/op	    3368 B/op	     211 allocs/op

Index

Constants

View Source 
const (
	// ServiceNameKey is the config key of the service name
	ServiceNameKey = "name"
	// ServiceDescriptionKey is the config key of the service
	// description
	ServiceDescriptionKey = "description"
	// ServiceOwnerKey is the config key for a service owner
	ServiceOwnerKey = "owner"
)
View Source 
const Root = ""

Root marks the root node in a Provider

Variables

This section is empty.

Functions

func AppRoot 

func AppRoot() string

AppRoot returns the root directory of your application. UberFx developers can edit this via the APP_ROOT environment variable. If the environment variable is not set then it will fallback to the current working directory.

func Environment 

func Environment() string

Environment returns current environment setup for the service

func EnvironmentKey 

func EnvironmentKey() string

EnvironmentKey returns environment variable key name

func EnvironmentPrefix 

func EnvironmentPrefix() string

EnvironmentPrefix returns environment prefix for the application

func Path 

func Path() string

Path returns path to the yaml configurations

func RegisterDynamicProviders 

func RegisterDynamicProviders(dynamicProviderFuncs ...DynamicProviderFunc)

RegisterDynamicProviders registers dynamic config providers for the global config Dynamic provider initialization needs access to Provider for accessing necessary information for bootstrap, such as port number,keys, endpoints etc.

func RegisterProviders 

func RegisterProviders(providerFuncs ...ProviderFunc)

RegisterProviders registers configuration providers for the global config

func ResolvePath 

func ResolvePath(relative string) (string, error)

ResolvePath returns an absolute path derived from AppRoot and the relative path. If the input parameter is already an absolute path it will be returned immediately.

func SetEnvironmentPrefix 

func SetEnvironmentPrefix(envPrefix string)

SetEnvironmentPrefix sets environment prefix for the application

func UnregisterProviders 

func UnregisterProviders()

UnregisterProviders clears all the default providers

Types

type ChangeCallback 

type ChangeCallback func(key string, provider string, data interface{})

ChangeCallback is called for updates of configuration data

type DynamicProviderFunc 

type DynamicProviderFunc func(config Provider) (Provider, error)

DynamicProviderFunc is used to create config providers on configuration initialization

type EnvironmentValueProvider 

type EnvironmentValueProvider interface {
	Get(key string) (string, bool)
}

An EnvironmentValueProvider provides configuration from your environment

type FileResolver 

type FileResolver interface {
	Resolve(file string) io.ReadCloser
}

A FileResolver resolves references to files

func NewRelativeResolver 

func NewRelativeResolver(paths ...string) FileResolver

NewRelativeResolver returns a file resolver relative to the given paths

type Provider 

type Provider interface {
	// the Name of the provider (YAML, Env, etc)
	Name() string
	// Get pulls a config value
	Get(key string) Value
	Scope(prefix string) Provider

	// A RegisterChangeCallback provides callback registration for config providers.
	// These callbacks are nop if a dynamic provider is not configured for the service.
	RegisterChangeCallback(key string, callback ChangeCallback) error
	UnregisterChangeCallback(token string) error
}

A Provider provides a unified interface to accessing configuration systems.

func Load 

func Load() Provider

Load creates a Provider for use in a service

func NewEnvProvider 

func NewEnvProvider(prefix string, provider EnvironmentValueProvider) Provider

NewEnvProvider creates a configuration provider backed by an environment

func NewProviderGroup 

func NewProviderGroup(name string, providers ...Provider) Provider

NewProviderGroup creates a configuration provider from a group of backends

func NewScopedProvider 

func NewScopedProvider(prefix string, provider Provider) Provider

NewScopedProvider creates a child provider given a prefix

func NewStaticProvider 

func NewStaticProvider(data map[string]interface{}) Provider

NewStaticProvider should only be used in tests to isolate config from your environment It is not race free, because underlying objects can be accessed with Value().

func NewYAMLProviderFromBytes 

func NewYAMLProviderFromBytes(yamls ...[]byte) Provider

NewYAMLProviderFromBytes creates a config provider from a byte-backed YAML blobs. As above, all the objects are going to be merged and arrays/values overridden in the order of the yamls.

func NewYAMLProviderFromFiles 

func NewYAMLProviderFromFiles(mustExist bool, resolver FileResolver, files ...string) Provider

NewYAMLProviderFromFiles creates a configuration provider from a set of YAML file names. All the objects are going to be merged and arrays/values overridden in the order of the files.

func NewYAMLProviderFromReader 

func NewYAMLProviderFromReader(readers ...io.ReadCloser) Provider

NewYAMLProviderFromReader creates a configuration provider from a list of `io.ReadClosers`. As above, all the objects are going to be merged and arrays/values overridden in the order of the files.

type ProviderFunc 

type ProviderFunc func() (Provider, error)

ProviderFunc is used to create config providers on configuration initialization

func EnvProvider 

func EnvProvider() ProviderFunc

EnvProvider returns function to create environment based config provider

func Providers 

func Providers() []ProviderFunc

Providers should only be used during tests

func StaticProvider 

func StaticProvider(data map[string]interface{}) ProviderFunc

StaticProvider returns function to create StaticProvider during configuration initialization

func YamlProvider 

func YamlProvider() ProviderFunc

YamlProvider returns function to create Yaml based configuration provider

type RelativeResolver 

type RelativeResolver struct {
	// contains filtered or unexported fields
}

A RelativeResolver resolves files relative to the given paths

func (RelativeResolver) Resolve 

func (rr RelativeResolver) Resolve(file string) io.ReadCloser

Resolve finds a reader relative to the given resolver

type Value 

type Value struct {
	Timestamp time.Time
	Type      ValueType
	// contains filtered or unexported fields
}

A Value holds the value of a configuration

func NewValue 

func NewValue(
	provider Provider,
	key string,
	value interface{},
	found bool,
	t ValueType,
	timestamp *time.Time,
) Value

NewValue creates a configuration value from a provider and a set of parameters describing the key

func (Value) AsBool 

func (cv Value) AsBool() bool

AsBool returns the configuration value as an bool, or panics if not bool-able

func (Value) AsFloat 

func (cv Value) AsFloat() float64

AsFloat returns the configuration value as an float64, or panics if not float64-able

func (Value) AsInt 

func (cv Value) AsInt() int

AsInt returns the configuration value as an int, or panics if not int-able

func (Value) AsString 

func (cv Value) AsString() string

AsString returns the configuration value as a string, or panics if not string-able

func (Value) ChildKeys 

func (cv Value) ChildKeys() []string

ChildKeys returns the child keys TODO(ai) what is this and do we need to keep it?

func (Value) HasValue 

func (cv Value) HasValue() bool

HasValue returns whether the configuration has a value that can be used

func (Value) IsDefault 

func (cv Value) IsDefault() bool

IsDefault returns whether the return value is the default.

func (Value) LastUpdated 

func (cv Value) LastUpdated() time.Time

LastUpdated returns when the configuration value was last updated

func (Value) PopulateStruct 

func (cv Value) PopulateStruct(target interface{}) error

PopulateStruct fills in a struct from configuration TODO(alsam) now we can populate not only structs. Provide a generic function.

func (Value) Source 

func (cv Value) Source() string

Source returns a configuration provider's name

func (Value) String 

func (cv Value) String() string

String prints out underline value in Value with fmt.Srpintf.

func (Value) TryAsBool 

func (cv Value) TryAsBool() (bool, bool)

TryAsBool attempts to return the configuration value as a bool

func (Value) TryAsFloat 

func (cv Value) TryAsFloat() (float64, bool)

TryAsFloat attempts to return the configuration value as a float

func (Value) TryAsInt 

func (cv Value) TryAsInt() (int, bool)

TryAsInt attempts to return the configuration value as an int

func (Value) TryAsString 

func (cv Value) TryAsString() (string, bool)

TryAsString attempts to return the configuration value as a string

func (Value) Value 

func (cv Value) Value() interface{}

Value returns the underlying configuration's value

func (Value) WithDefault 

func (cv Value) WithDefault(value interface{}) Value

WithDefault creates a shallow copy of the current configuration value and sets its default.

type ValueType 

type ValueType int

A ValueType is a type-description of a configuration value 

const (
	// Invalid represents an unset or invalid config type
	Invalid ValueType = iota
	// String is, well, you know what it is
	String
	// Integer holds numbers without decimals
	Integer
	// Bool is, well... go check Wikipedia. It's complicated.
	Bool
	// Float is an easy one. They don't sink in pools.
	Float
	// Slice will cut you.
	Slice
	// Dictionary contains words and their definitions
	Dictionary
)

func GetType 

func GetType(value interface{}) ValueType

GetType returns GO type of the provided object

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.