yagcl

README

yagcl

This libraries aim is to provide a powerful and dynamic way to provide configuration for your application.

Why

The thing that other libraries were lacking is the ability to parse different formats, allow merging them (for example override a setting via environment variables). Additionally I wanna be able to specify certain things via a reduced set and golang defaults.

The aim is to support all standard datatypes and allow nested structs with specified sub prefixes as well as one main prefix.

Additionally it is planned for the consumer of the library to be able to validate a struct, essentially making sure it does't contain nonsensical combinations of tags.

For example, the following wouldn't really make sense, since defining a key for an ignored field has no effect and will therefore result in an error:

type Configuration struct {
    Field string `key:"field" ignore:"true"`
}

Modules

Name	Repo	Docs	Status	Test Coverage
Main	You are here already!		WIP
Environment Variables	GitHub		WIP
JSON	GitHub		WIP
.env	-	-	Planned	-

Also check out the Roadmap for more detailed information.

Contribution

This library is separated into multiple modules. The main module and additional modules for each supported source. This allows you to only specify certain sources in your go-mod, keeping your dependency tree small. Additionally it makes navigating the code base easier.

If you wish to contribute a new source, please create a corresponding submodule.

Examples

An example configuration usage may look something like this:

import (
    yagcl_env "github.com/Bios-Marcel/yagcl-env"
    yagcl_json "github.com/Bios-Marcel/yagcl-json"
)

type Configuration struct {
    // The `key` here is used to define the JSON name for example. But the
    // environment variable names are the same, but uppercased.
    Host string `key:"host" required:"true"`
    Post int    `key:"port" required:"true"`
    // If you don't wish to export a field, you have to ignore it.
    // If it isn't ignored and doesn't have an explicit key, you'll
    // get an error, as this indicates a bug. The reason we don't
    // auto-generate a key is that this could result in unstable promises
    // as the variable name could change and break loading of old files.
    DontLoad    int               `ignore:"true"`
    // Nested structs are special, as they may not be part of your actual
    // configuration in case you are using environment variables, but will
    // be if you are using a JSON file. Either way, these also require the
    // key tag, as we are otherweise unable to build the names for its fields.
    KafkaServer KafkaServerConfig `key:"kafka"`
}

type KafkaServerConfig struct {
    //Alternatively you can define them explicitly. The same goes for json names.
    Host              string        `json:"host" env:"HOST" required:"true"`
    Port              int           `json:"port" env:"PORT" required:"true"`
    ConnectionTimeout time.Duration `json:"connection_timeout" env:"CONNECTION_TIMEOUT" required:"false"`
}

func LoadConfig() error {
    //Defaults should simply be defined on struct creation.
    configuration := Configuration{
        KafkaServer: KafkaServerConfig{
            Host:              "localhost",
            Port:              1234,
            ConnectionTimeout: time.Second * 10,
        },
    }
    err := yagcl.
        New[Configuration]()
        //This allows ordering when using override, so you can have something like this.
        Add(yagcl_json.Source().Path("/etc/myapp/config.json").Must()).
        Add(yagcl_env.Source().Prefix("MY_APP_")).
        Add(yagcl_json.Source().Path("~/.config/config.json")).
        AllowOverride().
        Parse(&configuration)
    return err
}

The configuration loaded by this would look like this:

{
    "host": "localhost",
    "port": 1234,
    "kafka": {
        "host": "123.123.123.123",
        "port": 9092,
        "connection_timeout": "10s"
    }
}

Or this when loading environment variables:

MY_APP_HOST=localhost
MY_APP_PORT=1234
MY_APP_KAFKA_HOST=123.123.123.123
MY_APP_KAFKA_PORT=9092
MY_APP_KAFKA_CONNECTION_TIMEOUT=10s

Usage

This library isn't stable / feature complete yet, even if it mostly works. The API might change any second ;)

If you want to try it out anyway, simply go get the desired modules.

For example:

go get github.com/Bios-Marcel/yagcl-env

Roadmap

• Basic API
• General Features
  • Honor required tags
  • Validation of configuration struct
  • Functioning Override mechanism where a whole source is optional or only some fields

  While overriding in general works, we'll error as soon as we are missing one required value in any of the sources.

• Read JSON
  • Honor key tags
  • Honor jsontags
  • Honor ignore tags
  • Type support
    • int / uint
    • float
    • bool
    • string
    • struct
    • pointer
    • time.Duration
    • array (Tests missing)
    • map (Tests missing)
• Read Environment variables
  • Honor key tags
  • Honor env tags
  • Honor ignore tags
  • Type support
    • int / uint
    • float
    • bool
    • string
    • struct
    • pointer
    • time.Duration
    • array
    • map
• Read .env files

  Will share code with environment variables and should have the same progression.

Non Goals

• High performance

  The code makes use of reflection and generally isn't written with efficiency in mind. However, this lib is supposed to do a one-time parse of configuration sources and therefore it shouldn't matter

Documentation

Index

• Constants
• Variables
• type ParsingCompanion
• type Source
• type YAGCL
  • func New[T any]() YAGCL[T]

Constants

const DefaultKeyTagName = "key"

DefaultKeyTagName is the go annotation key name for specifying the default fieldname. The value is to be expected to use only lowercase letters and underscores, as this is deemed the best default for readability. Sources can then convert this to kebapCase, UPPER_CASE or whatever else the put their faith in.

Variables

var (
	// ErrExpectAtLeastOneSource indicates that no configuration can be loaded
	// if there's not at least one source.
	ErrExpectAtLeastOneSource = errors.New("please define at least one source")
	// ErrSourceNotFound implies that a source has been added, but for example
	// the file it is supposed to read can't be found.
	ErrSourceNotFound = errors.New("the source could not find its target")
	// ErrInvalidConfiguraionPointer TODO
	ErrInvalidConfiguraionPointer = errors.New("invalid configuration pointer passed")

	// ErrExportedFieldMissingKey implies that a field is exported, but
	// doesn't define the key required for parsing it from a source. This
	// error can be omitted by setting `ignore:"true"`
	ErrExportedFieldMissingKey = errors.New("exported field is missing key definition")
	// ErrValueNotSet implies that no value could be found for a field, even
	// though it is required. The reason is either that there's no value, no
	// default value or a user error has occurred (e.g. a typo in the key).
	ErrValueNotSet = errors.New("required not set and no non-zero default value was found")
	// ErrParseValue implies that the value we attempted to parse is not in
	// the correct format to be assigned to its corresponding field. This is
	// most likely a user error.
	ErrParseValue = errors.New("value not parsable as type specified by field")
	// ErrUnsupportedFieldType implies that this library does NOT YET support
	// a certain field type.
	ErrUnsupportedFieldType = errors.New("unsupported field type")
)

Types

type ParsingCompanion

type ParsingCompanion interface {
	// IncludeField determines whether the field should be included in parsing.
	// This defines the standard for YAGCL and should be used by all sources.
	// A source may support additional rules that may even overwrite this ruleset.
	IncludeField(reflect.StructField) bool
	// ExtractFieldKey defines which identifier should be used for the given
	// field. This defines the standard identifier defined by YAGCL if no
	// specific identifier has been found for your source.
	ExtractFieldKey(reflect.StructField) string
}

type Source

type Source interface {
	// Parse attempts retrieving data from the source and parsing it. This
	// should return true if anything was loaded. If nothing was loaded, but we
	// expected to load data, we should return an error. A realistic scenario
	// here would be that a Source is required instead of "load if found".
	Parse(ParsingCompanion, any) (bool, error)
	// KeyTag defines the golang struct field tag that defines a specific tag
	// for this source. This allows having one generic key, but overrides for
	// specific sources.
	KeyTag() string
}

Source represents a source for configuration values. This might be backed by files of different formats, network access, environment variables or even the Windows registry. While YAGCL offers default sources, you can easily integrate your own source.

type YAGCL

type YAGCL[T any] interface {
	// Add adds a single source to read configuration from. This method can
	// be called multiple times, adding multiple ordered sources. Whatever is
	// added first is preferred. If AllowOverride() is called, all source will be
	// parsed in the defined order.
	Add(source Source) YAGCL[T]
	// AllowOverride allows YAGCL to read from multiple sources. For example more
	// than one JSON file or a JSON file and the environment.
	AllowOverride() YAGCL[T]
	// InferFieldKeys activates automtic generation of a field key if none has
	// been defined by the programmers.
	InferFieldKeys() YAGCL[T]
	// AdditionalKeyTags defines tags other than `key`, which will then be used
	// by ParsingCompanion.ExtractFieldKey.
	AdditionalKeyTags(tags ...string) YAGCL[T]

	// Parse expects a pointer to a struct, which it'll attempt loading the
	// configuration into. Note that you'll first have to specify any type
	// type of configuration to be loaded.
	Parse(configurationStruct *T) error
}

YAGCL defines the setup interface for YAGCL.

func New

func New[T any]() YAGCL[T]

New creates a fresh instance of YAGCL. This is an alternative to using the global instance accessible via Global().