README
¶
D3config 
⚠ This library was formerly called
configdb. You may need to do the following adjustments:
- Change your import paths and
go.modentry fromgithub.com/Dadido3/configdbtogithub.com/Dadido3/D3config.- Change any package name from
configdbtoconfig.- Change any struct tag key from
cdbtoconf.
This is a small library for handling hierarchical configuration values. The main principle is that the configuration values are loaded from storage objects, like YAML or JSON files. If there are multiple storage objects, their hierarchies are merged into a single tree that you can easily read from and written to.
Configuration values can be modified at runtime, either from the outside by editing the source files, or from within an application. In the latter case, the library writes the changes back to the first storage object you defined.
You can implement your own storage type by implementing the Storage interface.
Features
- Marshal & unmarshal any structures or types.
- Support of encoding.TextMarshaler and encoding.TextUnmarshaler interfaces.
- Can handle multiple configuration files. They are merged into one tree prioritized by order. (e.g. user settings, default, ...)
- Has several storage types (JSON files, YAML files), and you can implement your own storage types.
- Changes are saved to disk automatically, and changes on disk are loaded automatically.
- Listeners for tree/value changes can be registered.
- Safe against power loss while writing files to disk.
- Thread-safe by design.
Current state
The library is feature complete, but as it is really new and not much tested (Beside the unit tests) i can't guarantee that everything will work correctly. If you encounter a bug, or some undocumented behavior, open an issue.
Usage
To add this library to your go.mod file use:
go get github.com/Dadido3/D3config
Initialize
// The upper storage objects have higher priority as the lower ones.
// So the properties/values of the upper will overwrite the ones in the lower entries.
// One special case is the storage object at index 0, this is the one that changes are written into.
storages := []config.Storage{
config.UseJSONFile("testfiles/json/userconfig.json"),
config.UseYAMLFile("testfiles/yaml/custom.yml"),
config.UseJSONFile("testfiles/json/default.json"),
}
c, err := config.New(storages)
if err != nil {
fmt.Fatal(err)
}
defer c.Close()
Alternatively, you can define config.UseDummyStorage("", nil) as the first storage source.
In this case any modification of the values are only temporary and will be forgotten when the program ends.
Read value
var f float32
// Pass a pointer to any object you want to read from the internal tree at the given path ".box.width".
err := c.Get(".box.width", &f)
if err != nil {
t.Error(err)
}
This will write 123.456 into f, with json data like:
{
"box": {
"width": 123.456,
"height": 654.321
}
}
Read structure
// You can use tags to change the names, or exclude fields with "omit".
var str struct {
Width float64 `conf:"width"`
Height float64 `conf:"height"`
PlsIgnore string `conf:",omit"`
}
// Pass a pointer to any object you want to read from the internal tree at the given path ".box".
err := c.Get(".box", &str)
if err != nil {
t.Error(err)
}
fmt.Printf("%+v", str)
With the same json data as above, this will result in:
{Width:123.456 Height:654.321 PlsIgnore:}
Read slices, maps and more
// It also works with slices/arrays.
// They can be any type, even arrays of arrays.
var s []string
err := c.Get(".box.names", &s)
if err != nil {
t.Error(err)
}
fmt.Printf("%#v\n", s)
// Maps have the limitation that the key has to be a string.
// But the value type can be anything.
var m map[string]interface{}
err = c.Get(".box", &m)
if err != nil {
t.Error(err)
}
fmt.Printf("%#v\n", m)
// The lib supports all objects that support text (un)marshaller interface.
var ti time.Time
err = c.Get(".back.toTheFuture", &ti)
if err != nil {
t.Error(err)
}
fmt.Printf("%v\n", ti)
Will result in:
[]string{"Sam Sung", "Saad Maan", "Chris P. Bacon"}
map[string]interface {}{"height":"654.321", "names":[]interface {}{"Sam Sung", "Saad Maan", "Chris P. Bacon"}, "width":"123.456"}
1985-10-26 01:21:00 +0000 UTC
Write value
b := true
// Pass a boolean to be written at the path ".todo.WriteCode".
err := c.Set(".todo.WriteCode", b)
if err != nil {
t.Error(err)
}
ti := time.Date(2019, 7, 24, 14, 46, 24, 124, time.UTC)
// Pass time object to be written at the path ".time.WriteCodeAt".
err = c.Set(".time.WriteCodeAt", ti)
if err != nil {
t.Error(err)
}
This will write the changes to disk immediately, but the internal tree may be updated later.
Therefore a Get() directly following a Set() may still result in old data.
In these cases it's better to rely on the event mechanism, which is explained a few steps below.
If config was created with testfiles/json/userconfig.json being the first file, the following content will be added to it:
{
"todo": {
"WriteCode": true
},
"time": {
"WriteCodeAt": "2019-07-24T14:46:24.000000124Z"
},
}
Write structure
str := struct {
Eat, Sleep bool
}{true, false}
// Pass an object to be written at the path ".todo".
err := c.Set(".todo", str)
if err != nil {
t.Error(err)
}
Which will result in the file testfiles/json/userconfig.json to look like: (Assuming that "WriteCode": true was already present)
{
"todo": {
"Eat": true,
"Sleep": false,
"WriteCode": true
}
}
Write nil
// You can also overwrite anything with nil.
err := c.Set(".todo", nil)
if err != nil {
t.Error(err)
}
Which will result in:
{
"todo": null
}
This can be used to overwrite and disable any defaults from other storage objects.
Reset element
// Resets the element at the path ".todo".
// This will restore any defaults, if there are any present in lower priority storage objects.
err := c.Reset(".todo")
if err != nil {
t.Error(err)
}
// This will reset everything to default.
// It has the same effect as deleting the highest priority file.
err = c.Reset("")
if err != nil {
t.Error(err)
}
Register and unregister event callback
// Register callback to listen for events.
// Once registered, the callback is called once to update the listener with the current state of the tree.
id := c.RegisterCallback(nil, func(c *config.Config, modified, added, removed []string) {
fmt.Printf("All m: %v, a: %v, r:%v\n", modified, added, removed)
})
// Use the result id to unregister later.
defer c.UnregisterCallback(id)
// Register callback to listen for events, but only inside the path ".something.to.watch".
// This includes modifications to ".something.to.watch" itself.
id = c.RegisterCallback([]string{".something.to.watch"}, func(c *config.Config, modified, added, removed []string) {
fmt.Printf("Filtered m: %v, a: %v, r:%v\n", modified, added, removed)
})
// Use the result id to unregister later.
defer c.UnregisterCallback(id)
// Test the callback.
err := c.Set(".something.to.watch.for", 125)
if err != nil {
t.Error(err)
}
// The event may not be sent immediately, wait a bit before terminating the program.
time.Sleep(100 * time.Millisecond)
The output could look like this:
All m: [], a: [.back .back.toTheFuture .box .box.width .box.height .box.names .slicedNodes .something .something.to .something.to.watch .something.to.watch.for], r:[]
Filtered m: [], a: [.something.to.watch .something.to.watch.for], r:[]
Filtered m: [.something.to.watch.for], a: [], r:[]
All m: [.something.to.watch.for], a: [], r:[]
When you register a new listener, there will be one initial call to your callback.
The parameters are lists of paths (strings) that have either been modified, added or deleted from the tree. In most cases these lists can be ignored and are only needed for more advanced tasks.
A whitelist of paths can be defined to filter events. This way only paths that are included in the whitelist (or that are child elements of whitelisted paths) will trigger a callback. You can use this to restart a web server on configuration changes.
Additionally it is made sure that the tree is in sync with the changes. It's safe to use c.Get() or even c.Set()/c.Reset() inside the callback.
Custom storage objects
// Implement Storage interface.
type CustomStorage struct {
}
func (f *CustomStorage) Read() (tree.Node, error) {
return tree.Node{
"SomethingPermanent": tree.Node{
"foo": tree.Number("123"),
"bar": tree.Number("-123.456"),
},
}, nil
}
func (f *CustomStorage) Write(t tree.Node) error {
return fmt.Errorf("Can't write into this storage object")
}
func (f *CustomStorage) RegisterWatcher(changeChan chan<- struct{}) error {
return nil
}
func TestCustomStorage(t *testing.T) {
// Use the custom made storage object along with others.
// Be aware, that if you have a non writable storage at the top, the tree can't be modified anymore.
storages := []config.Storage{
config.UseJSONFile("testfiles/json/userconfig.json"),
&CustomStorage{},
config.UseJSONFile("testfiles/json/default.json"),
}
c, err := config.New(storages)
if err != nil {
t.Fatal(err)
}
defer c.Close()
}
FAQ
What are valid element names?
Any character except period . is allowed. Also empty names are valid too.
How to address elements of an array or slice with a path?
You can't. Paths can only address map elements or structure fields.
But you can use Get() to read any slice or array.
If you need to register a callback on something inside an array or slice, you have to point on the array/slice itself.
E.g. .someField.slice will also trigger an event when some element or value several levels deep inside of that slice is modified.
Is it really not possible to address elements inside arrays or slices?
With a trick it is:
- Import
"github.com/Dadido3/D3config/tree" - Use the following snippet:
var nodes []tree.Node
// Get a list of tree.Node objects.
// That will copy a subtree into the variable nodes.
err := c.Get(".slicedNodes", &nodes)
if err != nil {
t.Fatal(err)
}
// Read value of that subtree.
result := nodes[0].GetInt64(".something", 0)
fmt.Println(result)
Any edits you do on nodes have no effect on the main tree.
You need to use c.Set(path, nodes) to write it back.
This way you can also create copies to work with while the configuration is being modified.
Documentation
¶
Index ¶
- type Config
- func (c *Config) Close()
- func (c *Config) Get(path string, object interface{}) error
- func (c *Config) RegisterCallback(paths []string, callback func(c *Config, modified, added, removed []string)) int
- func (c *Config) Reset(path string) error
- func (c *Config) Set(path string, object interface{}) error
- func (c *Config) UnregisterCallback(id int)
- type DummyStorage
- type JSONFile
- type Storage
- type YAMLFile
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// contains filtered or unexported fields
}
Config contains the hierarchical configuration data.
Create this by using New(). Use the Set(), Reset() and Get() methods to interact with that tree. Changes made to the config are immediately stored in the the defined storage.
func New ¶
New returns a new Config object.
It takes a list of Storage objects that can be created with UseJSONFile(path) and similar functions. All storage objects in the list are merged into one big configuration tree. The higher the index of an storage object in that list, the lower its content's priority. Higher priority properties will overwrite lower priority ones.
If a file is changed on disk, it is reloaded, and merged with all other files/storages automatically. Changes in the configuration tree will be broadcasted to any listener.
If any of these storage objects couldn't be read from, this function will return an error. On the other hand, if any storage object fails to read later, nothing will reload.
func NewOrPanic ¶
NewOrPanic returns a new Config object. It's similar to New(), but it panics instead of returning an error.
func (*Config) RegisterCallback ¶
func (c *Config) RegisterCallback(paths []string, callback func(c *Config, modified, added, removed []string)) int
RegisterCallback will add the given callback to the internal listener list. A list of paths can be defined to ignore all events that are not inside the given paths.
An integer is returned, that can be used to Unregister() the callback.
func (*Config) Reset ¶
Reset will remove the element at the given path. Lower priority properties will be visible again, if available.
func (*Config) Set ¶
Set changes the element at the given path.
It's possible to modify the root node, with the path "", if the passed object is a map or a structure.
Changes are written immediately to the to the storage object.
func (*Config) UnregisterCallback ¶
UnregisterCallback removes a callback from the internal listener list.
type DummyStorage ¶
type DummyStorage struct {
// contains filtered or unexported fields
}
DummyStorage is a virtual storage container. The data is only kept in RAM, so anything stored in it will be lost once it is freed.
func (*DummyStorage) Read ¶
func (f *DummyStorage) Read() (tree.Node, error)
Read returns the tree representation of its content.
func (*DummyStorage) RegisterWatcher ¶
func (f *DummyStorage) RegisterWatcher(changeChan chan<- struct{}) error
RegisterWatcher takes a channel that is used to signal changes/modifications of the data. Only one channel can be registered at a time.
A nil value can be passed to unregister the listener.
type JSONFile ¶
type JSONFile struct {
// contains filtered or unexported fields
}
JSONFile represents a json file on disk.
func (*JSONFile) RegisterWatcher ¶
RegisterWatcher takes a channel that is used to signal changes/modifications of the data. Only one channel can be registered at a time.
A nil value can be passed to unregister the listener.
type Storage ¶
type Storage interface {
Read() (tree.Node, error)
Write(t tree.Node) error
RegisterWatcher(changeChan chan<- struct{}) error
}
Storage interface provides arbitrary ways to store/read hierarchical data.
func UseDummyStorage ¶
UseDummyStorage returns a virtual storage container that only stores data in the RAM. Use it if you want the ability to overwrite settings, without writing those settings somewhere permanently.
The storage can be initialized with a data structure that will be written at initPath.
type YAMLFile ¶
type YAMLFile struct {
// contains filtered or unexported fields
}
YAMLFile represents a json file on disk.
func (*YAMLFile) RegisterWatcher ¶
RegisterWatcher takes a channel that is used to signal changes/modifications of the data. Only one channel can be registered at a time.
A nil value can be passed to unregister the listener.
Source Files
Directories