konf

README

The simplest config loader for Go

konf offers an(other) opinion on how Go programs can read configuration without becoming coupled to a particular configuration source.

Features

• konf.Unmarshal for reading configuration to any type of object.
• konf.OnChange for registering callbacks while configuration changes.
• konf.Explain for understanding where the configuration is loaded from.
• Various providers for loading configuration from major clouds, AWS, Azure, and GCP with Notifier for notifying the changes of configuration.
• Zero dependencies in core module which supports loading configuration from environment variables,flags, and embed file system.

Benchmarks

The following benchmarks compare the performance of konf with spf13/viper and knadh/koanf, which are inspiration of konf.

	Unmarshal (ns/op)	Unmarshal (allocs/op)	Get[^1] (ns/op)	Get (allocs/op)
Konf	41.09	4	16.71	1
Viper	614.8	22	104.9	3
Koanf	15949	657	7.898	1

[^1]: Comparing to Get in both Viper and Koanf only can get the primitive types, Get in Konf can get any type since it's just a wrapper of Unmarshal. So for complex struct, it only needs single Konf Get but may needs multiple Get(s) in Viper and Koanf.

Usage

Somewhere, early in an application's life, it will make a decision about which configuration source(s) (implementation) it actually wants to use. Something like:

//go:embed config
var config embed.FS

func main() {
    var config konf.Config

    // Load configuration from embed file system.
    if err := config.Load(fs.New(config, "config/config.json")); err != nil {
        // Handle error here.
    }
    // Load configuration from environment variables.
    if err := config.Load(env.New(env.WithPrefix("server"))); err != nil {
        // Handle error here.
    }

    // Watch the changes of configuration.
    go func() {
      if err := config.Watch(ctx); err != nil {
        // Handle error here.
      }
    }()

    konf.SetDefault(config)

    // ... other setup code ...
}

Outside of this early setup, no other packages need to know about the choice of configuration source(s). They read configuration in terms of functions in package konf:

func (app *appObject) Run() {
    // Server configuration with default values.
    serverConfig := struct {
        Host string
        Port int
    }{
        Host: "localhost",
        Port: "8080",
    }
    // Read the server configuration.
    if err := konf.Unmarshal("server", &serverConfig);  err != nil {
        // Handle error here.
    }

    // Register callbacks while server configuration changes.
    konf.OnChange(func() {
      // Reconfig the application object.
    }, "server")

    // ... use cfg in app code ...
}

Design

It contains two APIs with two different sets of users:

• The Config type is intended for application authors. It provides a relatively small API which can be used everywhere you want to read configuration. It defers the actual configuration loading to the Loader interface.
• The Loader and Watcher interface is intended for configuration source library implementers. They are pure interfaces which can be implemented to provide the actual configuration.

This decoupling allows application developers to write code in terms of *konf.Config while the configuration source(s) is managed "up stack" (e.g. in or near main()). Application developers can then switch configuration sources(s) as necessary.

Change Notification

The providers for loading configuration from clouds periodically poll the configuration source. It also supports watching the changes of configuration using corresponding notifier. For example, the sns notifier notifies the changes of appconfig and s3 provider:

notifier := sns.NewNotifier("konf-test")
notifier.Register(s3Loader, appConfigLoader)
go func() {
  if err := notifier.Start(ctx); err != nil {
    // handle error
  }
}()

Understand the configuration

While the configuration is loaded from multiple sources, static like environments or dynamic like AWS AppConfig, it's hard to understand where a final value comes from. The Config.Explain method provides information about how Config resolve each value from loaders for the given path. One example explanation is like:

config.nest has value [map] is loaded by map.
Here are other value(loader)s:
  - env(env)

Even more, the Config.Explain blurs sensitive information (e.g. password, secret, api keys).

Observability

For watching the changes of configuration, it uses slog.Default() for logging. You can change the logger via option konf.WithLogHandler. Furthermore, you also can register onStatus via option konf.WithOnStatus to monitor the status of configuration loading/watching, e.g. recording metrics.

Configuration Providers

There are providers for the following configuration sources.

Loader	Load From	Watch Changes	Notifier
env	environment variables
fs	fs.FS
file	file	✓
flag	flag
pflag	spf13/pflag
appconfig	AWS AppConfig	✓	sns
s3	AWS S3	✓	sns
parameterstore	AWS ParameterStore	✓	sns
azappconfig	Azure App Configuration	✓	azservicebus
azblob	Azure Blob Storage	✓	azservicebus
secretmanager	GCP Secret Manager	✓	pubsub
gcs	GCP Cloud Storage	✓	pubsub

cobra is supported through the pflag loader, with the pflag.WithFlagSet option:

config.Load(kflag.New(&config, kflag.WithFlagSet(yourCobraCmd.Flags())))

Custom Configuration Providers

You can Custom provider by implementing the Loader for static configuration loader (e.g fs) or both Loader and Watcher for dynamic configuration loader (e.g. appconfig).

Documentation

Overview

Package konf provides a general-purpose configuration API and abstract interfaces to back that API. Packages in the Go ecosystem can depend on this package, while callers can load configuration from whatever source is appropriate.

It defines a type, Config, which provides a method Config.Unmarshal for loading configuration under the given path into the given object.

Each Config is associated with multiple Loader(s), Which load configuration from a source, such as file, environment variables etc. There is a default Config accessible through top-level functions (such as Unmarshal and Get) that call the corresponding Config methods.

Configuration is hierarchical, and the path is a sequence of keys that separated by delimiter. The default delimiter is `.`, which makes configuration path like `parent.child.key`.

Load Configuration

After creating a Config, you can load configuration from multiple Loader(s) using Config.Load. Each loader takes precedence over the loaders before it. As long as the configuration has been loaded, it can be used in following code to get or unmarshal configuration, even for loading configuration from another source. For example, it can read config file path from environment variables, and then use the file path to load configuration from file system.

Watch Changes

Config.Watch watches and updates configuration when it changes, which leads Config.Unmarshal always returns latest configuration. You may use Config.OnChange to register a callback if the value of any path have been changed. It could push the change into application objects instead pulling the configuration periodically.

Field Tags

When decoding to a struct, konf will use the field name by default to perform the mapping. For example, if a struct has a field "Username" then konf will look for a key in the source value of "username" (case insensitive).

type User struct {
    Username string
}

You can change the behavior of konf by using struct tags. The default struct tag that konf looks for is "konf" but you can customize it using DecoderConfig.

Renaming Fields

To rename the key that konf looks for, use the "konf" tag and set a value directly. For example, to change the "username" example above to "user":

type User struct {
    Username string `konf:"user"`
}

Embedded Structs and Squashing

Embedded structs are treated as if they're another field with that name. By default, the two structs below are equivalent when decoding with konf:

type Person struct {
    Name string
}

type Friend struct {
    Person
}

type Friend struct {
    Person Person
}

This would require an input that looks like below:

map[string]interface{}{
    "person": map[string]interface{}{"name": "alice"},
}

If your "person" value is NOT nested, then you can append ",squash" to your tag value and konf will treat it as if the embedded struct were part of the struct directly. Example:

type Friend struct {
    Person `konf:",squash"`
}

Now the following input would be accepted:

map[string]interface{}{
    "name": "alice",
}

Unexported fields

Since unexported (private) struct fields cannot be set outside the package where they are defined, the decoder will simply skip them.

For this output type definition:

type Exported struct {
    private string // this unexported field will be skipped
    Public string
}

Using this map as input:

map[string]interface{}{
    "private": "I will be ignored",
    "Public":  "I made it through!",
}

The following struct will be decoded:

type Exported struct {
    private: "" // field is left with an empty string (zero value)
    Public: "I made it through!"
}

Index

• func Explain(path string) string
• func Get[T any](path string) T
• func OnChange(onChange func(), paths ...string)
• func SetDefault(config *Config)
• func Unmarshal(path string, target any) error
• type Config
  • func New(opts ...Option) *Config
  • func (c *Config) Exists(path []string) bool
  • func (c *Config) Explain(path string) string
  • func (c *Config) Load(loader Loader) error
  • func (c *Config) OnChange(onChange func(*Config), paths ...string)
  • func (c *Config) Unmarshal(path string, target any) error
  • func (c *Config) Watch(ctx context.Context) error
• type Loader
• type Option
  • func WithCaseSensitive() Option
  • func WithDecodeHook[F, T any, FN func(F) (T, error) | func(F, T) error](hook FN) Option
  • func WithDelimiter(delimiter string) Option
  • func WithLogHandler(handler slog.Handler) Option
  • func WithMapKeyCaseSensitive() Option
  • func WithOnStatus(onStatus func(loader Loader, changed bool, err error)) Option
  • func WithTagName(tagName string) Option
• type Statuser
• type Watcher

Examples

• Get
• SetDefault
• Unmarshal

Functions

func Explain

func Explain(path string) string

Explain provides information about how default Config resolve each value from loaders for the given path. It blur sensitive information. The path is case-insensitive unless konf.WithCaseSensitive is set.

func Get

func Get[T any](path string) T

Get retrieves the value under the given path from the default Config. It returns the zero value of the expected type if there is an error. The path is case-insensitive unless konf.WithCaseSensitive is set.

Example

package main

import (
	"embed"
	"fmt"

	"github.com/nil-go/konf"
	"github.com/nil-go/konf/provider/env"

	kfs "github.com/nil-go/konf/provider/fs"
)

func main() {
	ExampleSetDefault()

	fmt.Print(konf.Get[string]("server.host"))
}

//go:embed testdata
var testdata embed.FS

func ExampleSetDefault() {
	var config konf.Config
	if err := config.Load(kfs.New(testdata, "testdata/config.json")); err != nil {

		panic(err)
	}
	if err := config.Load(env.New(env.WithPrefix("server"))); err != nil {

		panic(err)
	}
	konf.SetDefault(&config)

}

Output:

example.com

func OnChange

func OnChange(onChange func(), paths ...string)

OnChange registers a callback function that is executed when the value of any given path in the default Config changes. The paths are case-insensitive unless konf.WithCaseSensitive is set.

The register function must be non-blocking and usually completes instantly. If it requires a long time to complete, it should be executed in a separate goroutine.

This method is concurrent-safe.

func SetDefault

func SetDefault(config *Config)

SetDefault sets the given Config as the default Config. After this call, the konf package's top functions (e.g. konf.Get) will interact with the given Config.

Example

package main

import (
	"embed"

	"github.com/nil-go/konf"
	"github.com/nil-go/konf/provider/env"

	kfs "github.com/nil-go/konf/provider/fs"
)

//go:embed testdata
var testdata embed.FS

func main() {
	var config konf.Config
	if err := config.Load(kfs.New(testdata, "testdata/config.json")); err != nil {
		// Handle error here.
		panic(err)
	}
	if err := config.Load(env.New(env.WithPrefix("server"))); err != nil {
		// Handle error here.
		panic(err)
	}
	konf.SetDefault(&config)
}

func Unmarshal

func Unmarshal(path string, target any) error

Unmarshal reads configuration under the given path from the default Config and decodes it into the given object pointed to by target. The path is case-insensitive unless konf.WithCaseSensitive is set.

Example

package main

import (
	"embed"
	"fmt"

	"github.com/nil-go/konf"
	"github.com/nil-go/konf/provider/env"

	kfs "github.com/nil-go/konf/provider/fs"
)

func main() {
	ExampleSetDefault()

	cfg := struct {
		Host string
		Port int
	}{
		Host: "localhost",
		Port: 8080,
	}

	if err := konf.Unmarshal("server", &cfg); err != nil {
		// Handle error here.
		panic(err)
	}
	fmt.Printf("%s:%d\n", cfg.Host, cfg.Port)
}

//go:embed testdata
var testdata embed.FS

func ExampleSetDefault() {
	var config konf.Config
	if err := config.Load(kfs.New(testdata, "testdata/config.json")); err != nil {

		panic(err)
	}
	if err := config.Load(env.New(env.WithPrefix("server"))); err != nil {

		panic(err)
	}
	konf.SetDefault(&config)

}

Output:

example.com:8080

Types

type Config

type Config struct {
	// contains filtered or unexported fields
}

Config reads configuration from appropriate sources.

To create a new Config, call New.

func New

func New(opts ...Option) *Config

New creates a new Config with the given Option(s).

func (*Config) Exists

func (c *Config) Exists(path []string) bool

Exists tests if the given path exist in the configuration.

It's used by the loader to check if the configuration has been set by other loaders.

func (*Config) Explain

func (c *Config) Explain(path string) string

Explain provides information about how Config resolve each value from loaders for the given path. It blur sensitive information. The path is case-insensitive unless konf.WithCaseSensitive is set.

func (*Config) Load

func (c *Config) Load(loader Loader) error

Load loads configuration from the given loader. Each loader takes precedence over the loaders before it.

This method is concurrent-safe.

func (*Config) OnChange

func (c *Config) OnChange(onChange func(*Config), paths ...string)

OnChange registers a callback function that is executed when the value of any given path in the Config changes. It requires Config.Watch has been called first. The paths are case-insensitive unless konf.WithCaseSensitive is set.

The register function must be non-blocking and usually completes instantly. If it requires a long time to complete, it should be executed in a separate goroutine.

This method is concurrent-safe.

func (*Config) Unmarshal

func (c *Config) Unmarshal(path string, target any) error

Unmarshal reads configuration under the given path from the Config and decodes it into the given object pointed to by target. The path is case-insensitive unless konf.WithCaseSensitive is set.

func (*Config) Watch

func (c *Config) Watch(ctx context.Context) error

Watch watches and updates configuration when it changes. It blocks until ctx is done, or the service returns an error.

It only can be called once. Call after first has no effects.

type Loader

type Loader interface {
	Load() (map[string]any, error)
}

Loader is the interface that wraps the Load method.

Load loads the latest configuration and returns it as a nested map[string]any. The keys should be nested like `{parent: {child: {key: 1}}}`.

type Option

type Option func(*options)

Option configures a Config with specific options.

func WithCaseSensitive

func WithCaseSensitive() Option

WithCaseSensitive enables the case sensitivity of the configuration keys.

func WithDecodeHook

func WithDecodeHook[F, T any, FN func(F) (T, error) | func(F, T) error](hook FN) Option

WithDecodeHook provides the decode hook for decoding. The decode hook is a function that can customize how configuration are decoded.

It can be either `func(F) (T, error)` which returns the converted value, or `func(F, T) error` which sets the converted value inline.

By default, it composes string to time.Duration, string to []string split by `,` and string to encoding.TextUnmarshaler.

func WithDelimiter

func WithDelimiter(delimiter string) Option

WithDelimiter provides the delimiter used when specifying config paths. The delimiter is used to separate keys in the path.

For example, with the default delimiter `.`, a config path might look like `parent.child.key`.

func WithLogHandler

func WithLogHandler(handler slog.Handler) Option

WithLogHandler provides the slog.Handler for logs from watch.

By default, it uses handler from slog.Default().

func WithMapKeyCaseSensitive

func WithMapKeyCaseSensitive() Option

WithMapKeyCaseSensitive enables the case sensitivity of the map keys.

func WithOnStatus

func WithOnStatus(onStatus func(loader Loader, changed bool, err error)) Option

WithOnStatus provides the callback for monitoring status of configuration loading/watching.

func WithTagName

func WithTagName(tagName string) Option

WithTagName provides the tag name that reads for field names. The tag name is used when decoding configuration into structs.

For example, with the default tag name `konf`, it would look for `konf` tags on struct fields.

type Statuser

type Statuser interface {
	Status(onStatus func(changed bool, err error))
}

Statuser is the interface that wraps the Status method.

Status enables providers report the status of configuration watching.

type Watcher

type Watcher interface {
	Watch(ctx context.Context, onChange func(map[string]any)) error
}

Watcher is the interface that wraps the Watch method.

Watch watches the configuration and triggers the register callback with the latest full configurations as a nested map[string]any when it changes. It blocks until ctx is done, or the watching returns an error.