README
¶
distconf
Framework for configuration variables and an abbreviation of distributed configuration.
Consider using this framework if you:
- Want to configure your application from multiple sources
- Want access to configuration to be quick (atomic reads)
- Want to know when configuration changes, without restarting your application
The library comes with basic configuration Readers, but most advanced environments will want to read from something like Redis, Consul, or Zookeeper as well.
Core Concepts
Distconf is made up of an array of Readers that are in priority of configuration preference. When users want to get a configuration value, the readers are searched for the value in order and the first one is returned. The returned value has a Get() function that atomically returns the previous value and will return a new configuration value whenever one changes. It is ok to call Get() inside hot path code. Even if the original config value was fetched from a remote service, like Redis, the call to Get() will be an atomic memory access each time.
Example use case
The example I currently use is to have an array of Readers{ Environment, config file, consul (defaults), consul (production) } This allows me to override configuration from the env or a config file during development, use consul for dynamic configuration, have a default consul location for configuration shared across deployments, and have a specific consul location for production, canary, or internal development configuration.
Example for a user
As a user of distconf, a distconf.Distconf object will already be created and passed to you. You can get string, int64, time.Duration, boolean, or float objects from this Distconf. When you ask for them, you specify a default value that is used if for some reason all backings are down. You should ask for this value once, then get the configuration value from this distconf holder in your app. Generally, you create a config object that is Loaded in main and used in your program.
// app.go
type Config struct {
QueueSize *distconf.Int
}
func (c *Config) Load(dconf *distconf.Distconf) {
c.QueueSize = dconf.Int("app.queue_size", 100)
}
type App struct {
Config Config
}
func (a *App) Start() {
for {
x := sqs.Load(a.Config.QueueSize.Get())
// ...
}
}
// main.go
func main() {
dconf := loadDistConf()
c := app.Config{}
c.Load(dconf)
q := app.App {
Config: c,
}
q.Start()
}
Example creating a distconf
jconf := JSONConfig {}
jconf.RefreshFile("config.json")
dconf := Distconf {
Readers: []Reader {
&Env{}, &jconf,
},
}
// The variable should only be fetched a single time. Then, store qsize somewhere and call Get() to fetch the
// current value of queue.size
qsize := dconf.Int("queue.size", 10)
for {
fmt.Println(qsize.Get())
}
Readers
The included readers only depend upon the core libraries. If you want to implement your own Reader, it should satisfy the following interface.
// Reader can get a []byte value for a config key
type Reader interface {
// Get should return the given config value. If the value does not exist, it should return nil, nil.
Get(key string) ([]byte, error)
Close()
}
If your reader is dynamic (the values contained can change while the application is running), then it should also implement the Dynamic interface.
// A Dynamic config can change what it thinks a value is over time.
type Dynamic interface {
// Watch should execute callback function whenever the key changes. The parameter to callback should be the
// key's name.
Watch(key string, callback func(string)) error
}
Debugging
You can expose an expvar of all the configuration keys and values with Var() and just the keys with Keys().
LICENSE
Apache 2.0
Originally forked from https://github.com/signalfx/golib/tree/master/distconf at 3b7d7c75a7219b2f7b8ca5dd119251254f6a2a06
Documentation
¶
Index ¶
- func ReaderCacheNotify(s *ReaderCache, key string, newVal []byte)
- func ReaderCacheRefresh(s *ReaderCache, r Reader, OnFetchErr func(err error, key string))
- type Bool
- type CachedReader
- func (m *CachedReader) Close()
- func (m *CachedReader) Get(key string) ([]byte, error)
- func (m *CachedReader) ListConfig() map[string][]byte
- func (m *CachedReader) ReadFrom(r io.Reader) (n int64, err error)
- func (m *CachedReader) StoreConfig(toStore map[string][]byte)
- func (m *CachedReader) Watch(key string, callback func(string)) error
- func (m *CachedReader) WriteTo(w io.Writer) (int64, error)
- type ComboRefresher
- type CommandLine
- type Distconf
- func (c *Distconf) Bool(key string, defaultVal bool) *Bool
- func (c *Distconf) Close()
- func (c *Distconf) Duration(key string, defaultVal time.Duration) *Duration
- func (c *Distconf) Float(key string, defaultVal float64) *Float
- func (c *Distconf) Int(key string, defaultVal int64) *Int
- func (c *Distconf) Keys() []string
- func (c *Distconf) Str(key string, defaultVal string) *Str
- func (c *Distconf) Struct(key string, defaultVal interface{}) *Struct
- func (c *Distconf) Var() expvar.Var
- type Duration
- type Dynamic
- type DynamicReader
- type Env
- type Float
- type InMemory
- func (m *InMemory) Close()
- func (m *InMemory) Get(key string) ([]byte, error)
- func (m *InMemory) ListConfig() map[string][]byte
- func (m *InMemory) StoreConfig(toStore map[string][]byte)
- func (m *InMemory) Watch(key string, callback func(string)) error
- func (m *InMemory) Write(key string, value []byte) error
- type Int
- type JSONConfig
- type Logger
- type Reader
- type ReaderCache
- type ReaderWriter
- type Refreshable
- type Refresher
- type Str
- type Struct
- type Writer
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ReaderCacheNotify ¶
func ReaderCacheNotify(s *ReaderCache, key string, newVal []byte)
ReaderCacheNotify notifies a readercache that a value changed. This function isn't public on the ReaderCache so structs can embed a ReaderCache directly without exposing notifyWatchers.
func ReaderCacheRefresh ¶
func ReaderCacheRefresh(s *ReaderCache, r Reader, OnFetchErr func(err error, key string))
ReaderCacheRefresh calls a Get on every value watched by the ReaderCache
Types ¶
type Bool ¶
type Bool struct {
// contains filtered or unexported fields
}
Bool is a Boolean type config inside a Config. It uses strconv.ParseBool to parse the conf contents as either true for false
func (*Bool) MarshalJSON ¶
MarshalJSON converts the boolean into the JSON encoding of the boolean
type CachedReader ¶
type CachedReader struct {
Fallback DynamicReader
// contains filtered or unexported fields
}
CachedReader allows bulk loading values and caching the result
func (*CachedReader) Get ¶
func (m *CachedReader) Get(key string) ([]byte, error)
Get will try fetching it from source. If you can, update in memory. If you can't, try to find it in memory and return that instead.
func (*CachedReader) ListConfig ¶
func (m *CachedReader) ListConfig() map[string][]byte
ListConfig returns all the cached values
func (*CachedReader) ReadFrom ¶
func (m *CachedReader) ReadFrom(r io.Reader) (n int64, err error)
ReadFrom loads a stored cache
func (*CachedReader) StoreConfig ¶
func (m *CachedReader) StoreConfig(toStore map[string][]byte)
StoreConfig overwrites the stored config
type ComboRefresher ¶
type ComboRefresher []Refreshable
ComboRefresher can refresh from multiple sources
func (ComboRefresher) Refresh ¶
func (c ComboRefresher) Refresh()
Refresh calls all refreshes at once
type CommandLine ¶
CommandLine gets distconf values from the command line
type Distconf ¶
Distconf gets configuration data from the first backing that has it
func (*Distconf) Bool ¶
Bool object that can be referenced to get boolean values from a backing config
func (*Distconf) Close ¶
func (c *Distconf) Close()
Close this config framework's readers. Config variable results are undefined after this call.
func (*Distconf) Duration ¶
Duration returns a duration object that calls ParseDuration() on the given key
func (*Distconf) Float ¶
Float object that can be referenced to get float values from a backing config
func (*Distconf) Int ¶
Int object that can be referenced to get integer values from a backing config
type Duration ¶
type Duration struct {
// contains filtered or unexported fields
}
Duration is a duration type config inside a Config.
func (*Duration) MarshalJSON ¶
MarshalJSON converts the duration into the JSON encoding of the duration
type Dynamic ¶
type Dynamic interface {
// Watch should execute callback function whenever the key changes. The parameter to callback should be the
// key's name.
Watch(key string, callback func(string)) error
}
A Dynamic config can change what it thinks a value is over time.
type DynamicReader ¶
DynamicReader is a reader that can also change
type Env ¶
type Env struct {
// Prefix is a forced prefix in front of config variables
Prefix string
// OsGetenv is a stub for os.Getenv()
OsGetenv func(string) string
}
Env allows fetching configuration from the env
type Float ¶
type Float struct {
// contains filtered or unexported fields
}
Float is an float type config inside a Config.
func (*Float) MarshalJSON ¶
MarshalJSON converts the float into the JSON encoding of the float
type InMemory ¶
type InMemory struct {
// contains filtered or unexported fields
}
InMemory stores configuration in memory
func (*InMemory) Close ¶
func (m *InMemory) Close()
Close does nothing and exists to satisfy an interface
func (*InMemory) ListConfig ¶
ListConfig returns a copy of the currently stored config values
func (*InMemory) StoreConfig ¶
StoreConfig stores into memory the given values, only if there currently is not a copy
type Int ¶
type Int struct {
// contains filtered or unexported fields
}
Int is an integer type config inside a Config.
func (*Int) MarshalJSON ¶
MarshalJSON converts the integer into the JSON encoding of the integer
type JSONConfig ¶
type JSONConfig struct {
// contains filtered or unexported fields
}
JSONConfig reads configuration from a JSON stream
func (*JSONConfig) Close ¶
func (j *JSONConfig) Close()
Close does nothing and exists to satisfy an interface
func (*JSONConfig) Get ¶
func (j *JSONConfig) Get(key string) ([]byte, error)
Get returns the key's value as read by JSON
func (*JSONConfig) Refresh ¶
func (j *JSONConfig) Refresh(input io.Reader) error
Refresh loads the configuration from a Reader
func (*JSONConfig) RefreshFile ¶
func (j *JSONConfig) RefreshFile(filename string) error
RefreshFile reloads the configuration from a file
type Reader ¶
type Reader interface {
// Get should return the given config value. If the value does not exist, it should return nil, nil.
Get(key string) ([]byte, error)
Close()
}
Reader can get a []byte value for a config key
type ReaderCache ¶
type ReaderCache struct {
// contains filtered or unexported fields
}
ReaderCache is a type of distconf reader that simply caches previous Get() values and updates watchers when a value changes
func (*ReaderCache) Watch ¶
func (s *ReaderCache) Watch(key string, callback func(string)) error
Watch registers a distconf watcher
func (*ReaderCache) WatchedKeys ¶
func (s *ReaderCache) WatchedKeys() []string
WatchedKeys copies the internal watched keys map's keys
type ReaderWriter ¶
A ReaderWriter can both read and write configuration information
type Refreshable ¶
type Refreshable interface {
Refresh()
}
Refreshable is any configuration that needs to be periodically refreshed
type Refresher ¶
type Refresher struct {
WaitTime *Duration
ToRefresh Refreshable
// contains filtered or unexported fields
}
A Refresher can refresh the values inside any Refreshable object
type Str ¶
type Str struct {
// contains filtered or unexported fields
}
Str is a string type config inside a Config.
func (*Str) MarshalJSON ¶
MarshalJSON converts the string into the JSON encoding of the string
type Struct ¶
type Struct struct {
// contains filtered or unexported fields
}
Struct allows distconf to atomically update sets of variables. The structure provided must be represented as a json object in the backend. The DefaultValue provided sets the type that the backend josn will be unmarshelled into.
func (*Struct) Get ¶
func (s *Struct) Get() interface{}
Get the current value. For structs this *must* be cast to the same type as the DefaultValue provided. Not matching the type will result in a runtime panic.
func (*Struct) MarshalJSON ¶
MarshalJSON converts the struct into the JSON encoding of the struct
Source Files