config

Documentation

Overview

A strongly-typed config library to parse configs from PFlags, Env Vars and Config files. Config package enables consumers to access (readonly for now) strongly typed configs without worrying about mismatching keys or casting to the wrong type. It supports basic types (e.g. int, string) as well as more complex structures through json encoding/decoding.

Config package introduces the concept of Sections. Each section should be given a unique section key. The binary will not load if there is a conflict. Each section should be represented as a Go struct and registered at startup before config is loaded/parsed.

Sections can be nested too. A new config section can be registered as a sub-section of an existing one. This allows dynamic grouping of sections while continuing to enforce strong-typed parsing of configs.

Config data can be parsed from supported config file(s) (yaml, prop, toml), env vars, PFlags or a combination of these Precedence is (flags, env vars, config file, defaults). When data is read from config files, a file watcher is started to monitor for changes in those files. If the registrant of a section subscribes to changes then a handler is called when the relevant section has been updated. Sections within a single config file will be invoked after all sections from that particular config file are parsed. It follows that if there are inter-dependent sections (e.g. changing one MUST be followed by a change in another), then make sure those sections are placed in the same config file.

A convenience tool is also provided in cli package (pflags) that generates an implementation for PFlagProvider interface based on json names of the fields.

Example

// This example demonstrates basic usage of config sections.

//go:generate pflags OtherComponentConfig

type OtherComponentConfig struct {
	DurationValue Duration `json:"duration-value"`
	URLValue      URL      `json:"url-value"`
	StringValue   string   `json:"string-value"`
}

// Each component should register their section in package init() or as a package var
section := MustRegisterSection("other-component", &OtherComponentConfig{})

// Override configpath to look for a custom location.
configPath := filepath.Join("testdata", "config.yaml")

// Initialize an accessor.
var accessor Accessor
// e.g.
// accessor = viper.NewAccessor(viper.Options{
//		StrictMode:   true,
//		SearchPaths: []string{configPath, configPath2},
// })

// Optionally bind to Pflags.
// accessor.InitializePflags(flags)

// Parse config from file or pass empty to rely on env variables and PFlags
err := accessor.UpdateConfig(context.Background())
if err != nil {
	fmt.Printf("Failed to validate config from [%v], error: %v", configPath, err)
	return
}

// Get parsed config value.
parsedConfig := section.GetConfig().(*OtherComponentConfig)
fmt.Printf("Config: %v", parsedConfig)

Example (Nested)

// This example demonstrates registering nested config sections dynamically.

//go:generate pflags OtherComponentConfig

type OtherComponentConfig struct {
	DurationValue Duration `json:"duration-value"`
	URLValue      URL      `json:"url-value"`
	StringValue   string   `json:"string-value"`
}

// Each component should register their section in package init() or as a package var
Section := MustRegisterSection("my-component", &MyComponentConfig{})

// Other packages can register their sections at the root level (like the above line) or as nested sections of other
// sections (like the below line)
NestedSection := Section.MustRegisterSection("nested", &OtherComponentConfig{})

// Override configpath to look for a custom location.
configPath := filepath.Join("testdata", "nested_config.yaml")

// Initialize an accessor.
var accessor Accessor
// e.g.
// accessor = viper.NewAccessor(viper.Options{
//		StrictMode:   true,
//		SearchPaths: []string{configPath, configPath2},
// })

// Optionally bind to Pflags.
// accessor.InitializePflags(flags)

// Parse config from file or pass empty to rely on env variables and PFlags
err := accessor.UpdateConfig(context.Background())
if err != nil {
	fmt.Printf("Failed to validate config from [%v], error: %v", configPath, err)
	return
}

// Get parsed config value.
parsedConfig := NestedSection.GetConfig().(*OtherComponentConfig)
fmt.Printf("Config: %v", parsedConfig)

Index

• Constants
• Variables
• func AllConfigsAsMap(root Section) (m map[string]interface{}, err error)
• func DeepEqual(config1, config2 Config) bool
• func NewConfigCommand(accessorProvider AccessorProvider) *cobra.Command
• type Accessor
• type AccessorProvider
• type Config
  • func DeepCopyConfig(config Config) (Config, error)
• type Duration
  • func (d Duration) MarshalJSON() ([]byte, error)
  • func (d *Duration) UnmarshalJSON(b []byte) error
• type Options
• type PFlagProvider
• type Port
  • func (p Port) MarshalJSON() ([]byte, error)
  • func (p Port) String() string
  • func (p *Port) UnmarshalJSON(b []byte) error
• type Regexp
  • func (r Regexp) MarshalJSON() ([]byte, error)
  • func (r *Regexp) UnmarshalJSON(b []byte) error
• type Section
  • func GetRootSection() Section
  • func GetSection(key SectionKey) Section
  • func MustRegisterSection(key SectionKey, configSection Config) Section
  • func MustRegisterSectionWithUpdates(key SectionKey, configSection Config, updatesFn SectionUpdated) Section
  • func NewRootSection() Section
  • func NewSection(configSection Config, updatesFn SectionUpdated) Section
  • func RegisterSection(key SectionKey, configSection Config) (Section, error)
  • func RegisterSectionWithUpdates(key SectionKey, configSection Config, updatesFn SectionUpdated) (Section, error)
• type SectionKey
• type SectionMap
• type SectionUpdated
• type URL
  • func (d URL) MarshalJSON() ([]byte, error)
  • func (d *URL) UnmarshalJSON(b []byte) error

Examples

• Package
• Package (Nested)

Constants

const (
	PathFlag        = "file"
	StrictModeFlag  = "strict"
	CommandValidate = "validate"
	CommandDiscover = "discover"
)

Variables

var (
	ErrStrictModeValidation       = fmt.Errorf("failed strict mode check")
	ErrChildConfigOverridesConfig = fmt.Errorf("child config attempts to override an existing native config property")
)

Functions

func AllConfigsAsMap

func AllConfigsAsMap(root Section) (m map[string]interface{}, err error)

Builds a generic map out of the root section config and its sub-sections configs.

func DeepEqual

func DeepEqual(config1, config2 Config) bool

func NewConfigCommand

func NewConfigCommand(accessorProvider AccessorProvider) *cobra.Command

Types

type Accessor

type Accessor interface {
	// Gets a friendly identifier for the accessor.
	ID() string

	// Initializes the config parser with golang's default flagset.
	InitializeFlags(cmdFlags *flag.FlagSet)

	// Initializes the config parser with pflag's flagset.
	InitializePflags(cmdFlags *pflag.FlagSet)

	// Parses and validates config file(s) discovered then updates the underlying config section with the results.
	// Exercise caution when calling this because multiple invocations will overwrite each other's results.
	UpdateConfig(ctx context.Context) error

	// Gets path(s) to the config file(s) used.
	ConfigFilesUsed() []string
}

Provides a simple config parser interface.

type AccessorProvider

type AccessorProvider func(options Options) Accessor

type Config

type Config = interface{}

func DeepCopyConfig

func DeepCopyConfig(config Config) (Config, error)

Uses Json marshal/unmarshal to make a deep copy of a config object.

type Duration

type Duration struct {
	time.Duration
}

A wrapper around time.Duration that enables Json Marshalling capabilities

func (Duration) MarshalJSON

func (d Duration) MarshalJSON() ([]byte, error)

func (*Duration) UnmarshalJSON

func (d *Duration) UnmarshalJSON(b []byte) error

type Options

type Options struct {
	// Instructs parser to fail if any section/key in the config file read do not have a corresponding registered section.
	StrictMode bool

	// Search paths to look for config file(s). If not specified, it searches for config.yaml under current directory as well
	// as /etc/flyte/config directories.
	SearchPaths []string

	// Defines the root section to use with the accessor.
	RootSection Section
}

Options used to initialize a Config Accessor

type PFlagProvider

type PFlagProvider interface {
	GetPFlagSet(prefix string) *pflag.FlagSet
}

A section can optionally implements this interface to add its fields as cmdline arguments.

type Port

type Port struct {
	Port int `json:"port,omitempty"`
}

A common port struct that supports Json marshal/unmarshal into/from simple strings/floats.

func (Port) MarshalJSON

func (p Port) MarshalJSON() ([]byte, error)

func (Port) String

func (p Port) String() string

func (*Port) UnmarshalJSON

func (p *Port) UnmarshalJSON(b []byte) error

type Regexp

type Regexp struct {
	regexp.Regexp
}

A regexp.Regexp wrapper that can marshal and unmarshal into simple regexp string.

func (Regexp) MarshalJSON

func (r Regexp) MarshalJSON() ([]byte, error)

func (*Regexp) UnmarshalJSON

func (r *Regexp) UnmarshalJSON(b []byte) error

type Section

type Section interface {
	// Gets a cloned copy of the Config registered to this section. This config instance does not account for any child
	// section registered.
	GetConfig() Config

	// Gets a function pointer to call when the config has been updated.
	GetConfigUpdatedHandler() SectionUpdated

	// Sets the config and sets a bit indicating whether the new config is different when compared to the existing value.
	SetConfig(config Config) error

	// Gets a value indicating whether the config has changed since the last call to GetConfigChangedAndClear and clears
	// the changed bit. This operation is atomic.
	GetConfigChangedAndClear() bool

	// Retrieves the loaded values for section key if one exists, or nil otherwise.
	GetSection(key SectionKey) Section

	// Gets all child config sections.
	GetSections() SectionMap

	// Registers a section with the config manager. Section keys are case insensitive and must be unique.
	// The section object must be passed by reference since it'll be used to unmarshal into. It must also support json
	// marshaling. If the section registered gets updated at runtime, the updatesFn will be invoked to handle the propagation
	// of changes.
	RegisterSectionWithUpdates(key SectionKey, configSection Config, updatesFn SectionUpdated) (Section, error)

	// Registers a section with the config manager. Section keys are case insensitive and must be unique.
	// The section object must be passed by reference since it'll be used to unmarshal into. It must also support json
	// marshaling. If the section registered gets updated at runtime, the updatesFn will be invoked to handle the propagation
	// of changes.
	MustRegisterSectionWithUpdates(key SectionKey, configSection Config, updatesFn SectionUpdated) Section

	// Registers a section with the config manager. Section keys are case insensitive and must be unique.
	// The section object must be passed by reference since it'll be used to unmarshal into. It must also support json
	// marshaling.
	RegisterSection(key SectionKey, configSection Config) (Section, error)

	// Registers a section with the config manager. Section keys are case insensitive and must be unique.
	// The section object must be passed by reference since it'll be used to unmarshal into. It must also support json
	// marshaling.
	MustRegisterSection(key SectionKey, configSection Config) Section
}

func GetRootSection

func GetRootSection() Section

Gets the global root section.

func GetSection

func GetSection(key SectionKey) Section

Retrieves the loaded values for section key if one exists, or nil otherwise.

func MustRegisterSection

func MustRegisterSection(key SectionKey, configSection Config) Section

func MustRegisterSectionWithUpdates

func MustRegisterSectionWithUpdates(key SectionKey, configSection Config, updatesFn SectionUpdated) Section

func NewRootSection

func NewRootSection() Section

func NewSection

func NewSection(configSection Config, updatesFn SectionUpdated) Section

func RegisterSection

func RegisterSection(key SectionKey, configSection Config) (Section, error)

Registers a section with the config manager. Section keys are case insensitive and must be unique. The section object must be passed by reference since it'll be used to unmarshal into. It must also support json marshaling.

func RegisterSectionWithUpdates

func RegisterSectionWithUpdates(key SectionKey, configSection Config, updatesFn SectionUpdated) (Section, error)

Registers a section with the config manager. Section keys are case insensitive and must be unique. The section object must be passed by reference since it'll be used to unmarshal into. It must also support json marshaling. If the section registered gets updated at runtime, the updatesFn will be invoked to handle the propagation of changes.

type SectionKey

type SectionKey = string

type SectionMap

type SectionMap map[SectionKey]Section

type SectionUpdated

type SectionUpdated func(ctx context.Context, newValue Config)

type URL

type URL struct {
	url.URL
}

A url.URL wrapper that can marshal and unmarshal into simple URL strings.

func (URL) MarshalJSON

func (d URL) MarshalJSON() ([]byte, error)

func (*URL) UnmarshalJSON

func (d *URL) UnmarshalJSON(b []byte) error