config

package
v1.4.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 12, 2023 License: MIT Imports: 28 Imported by: 1

README

config package

The config package wraps the common viper methods and types but adds some custom processing.

This probably could be done using custom decode hooks for viper, but I have not had time to understand them in detail.

Using a wrapper also allows custom methods on config for further functionality.

Unless the file format is given to LoadConfig then all the viper formats are supported.

Expansion of values

The following package functions and type methods are overridden to add expansion of embedded strings of the form ${name} and $name:

The string values are passed to ExpandString which supports a range of data substitutions.

Documentation

Overview

Package config adds support for value expansion over viper based configurations.

A number of the most common access methods from viper are replaced with local versions that add support for ExpandString. Additionally, there are a number of functions to simplify programs including LoadConfig.

Package config adds support for value expansion over viper based configurations.

A number of the most common access methods from viper are replaced with local versions that add support for ExpandString. Additionally, there are a number of functions to simplify programs including LoadConfig.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Checksum 

func Checksum(r io.Reader) (c uint32, err error)

func ChecksumFile 

func ChecksumFile(path string) (c uint32, err error)

func ChecksumString 

func ChecksumString(in string) (c uint32, err error)

func EncodePasswordPrompt added in v1.4.1 

func EncodePasswordPrompt(keyfile string, expandable bool) (out string, err error)

EncodePasswordPrompt prompts the user for a password and again to verify, offering up to three attempts until the password match. When the two match the plaintext is encoded using the supplied keyfile. If expandable is true then the encoded password is returned in a format useable by the Expand function and includes a path to the keyfile.

func EncodeWithKey added in v1.4.1 

func EncodeWithKey(plaintext []byte, key []byte) (out string, err error)

EncodeWithKey encodes the plaintext using the AES key in the byte slice given. The encoded password is returned in `Geneos AES256` format, with the `+encs+` prefix.

func EncodeWithKeyReader added in v1.4.1 

func EncodeWithKeyReader(plaintext []byte, r io.Reader) (out string, err error)

EncodeWithKey encodes the plaintext using the AES key read from the io.Reader given. The encoded password is returned in `Geneos AES256` format, with the `+encs+` prefix.

func EncodeWithKeyfile added in v1.4.1 

func EncodeWithKeyfile(plaintext []byte, keyfile string, expandable bool) (out string, err error)

EncodeWithKey encodes the plaintext using the AES key read from the file given. The encoded password is returned in `Geneos AES256` format, with the `+encs+` prefix, unless expandable is set to true in which case it is returned in a format that can be used with the Expand function and includes a reference to the keyfile.

If the keyfile is located under the user's configuration directory, as defined by UserConfigDir, then the function will replace any home directory prefix with `~/' to shorten the keyfile path.

func Expand added in v1.4.1 

func Expand(input string, options ...ExpandOptions) (value []byte)

Expand behaves like ExpandString but returns a byte slice.

This should be used where the return value may contain sensitive data and an immutable string cannot be destroyed after use.

func ExpandString added in v1.3.0 

func ExpandString(input string, options ...ExpandOptions) (value string)

ExpandString returns the input with all occurrences of the form ${name} replaced using an os.Expand-like function (but without support for bare names) for the built-in and optional formats (in the order of priority) below. The caller can use options to define additional expansion functions based on a "prefix:", disabled external lookups and also to pass in lookup tables referred to as value maps. 

${enc:keyfile[|keyfile...]:encodedvalue}

   The item "encodedvalue" is an AES256 ciphertext in Geneos format
   - or a reference to one - which will be decoded using the key
   file(s) given. Each "keyfile" must be one of either an absolute
   path, a path relative to the working directory of the program, or
   if prefixed with "~/" then relative to the home directory of the
   user running the program. The first valid decode (see below) is
   returned.

   The "encodedvalue" must be either prefixed "+encs+" to align with
   Geneos or will otherwise be looked up using the forms of any of
   the other references below, but without the surrounding
   dollar-brackets "${...}".

   To minimise (but not wholly eliminate) any false-positive decodes
   that occur in some circumstances when using the wrong key file,
   the decoded value is only returned if it is a valid UTF-8 string
   as per [utf8.Valid].

   Examples:

   * password: ${enc:~/.keyfile:+encs+9F2C3871E105EC21E4F0D5A7921A937D}
   * password: ${enc:/etc/geneos/keyfile.aes:env:ENCODED_PASSWORD}
   * password: ${enc:~/.config/geneos/keyfile1.aes:app.password}
   * password: ${enc:~/.keyfile.aes:config:mySecret}

${config:key} or ${path.to.config}

   Fetch the "key" configuration value (for single layered
   configurations, where a sub-level dot cannot be used) or if any
   value containing one or more dots "." will be looked-up in the
   existing configuration that the method is called on. The
   underlying configuration is not changed and values are resolved
   each time ExpandString() is called. No locking of the
   configuration is done.

 ${key}

   "key" will be substituted with the value of the first matching
   key from the maps "values...", in the order passed to the
   function. If no "values" are passed (as opposed to the key not
   being found in any of the maps) then name is looked up
   as an environment variable, as 4. below.

 ${env:name}

   "name" will be substituted with the contents of the environment
   variable of the same name.

 The prefixes below are optional and enabled by default. They can be
 enabled or disabled using the option [config.ExternalLookups()]
 option. The are enabled by default.

 ${~/file} or ${/path/to/file} or ${file://path/to/file} or ${file:~/path/to/file}

   The contents of the referenced file will be read. Multiline files
   are used as-is; this can, for example, be used to read PEM
   certificate files or keys. If the path is prefixed with "~/" (or
   as an addition to a standard file url, if the first "/" is
   replaced with a tilde "~") then the path is relative to the home
   directory of the user running the process.

   Examples:

   * certfile ${file://etc/ssl/cert.pem}
   * template: ${file:~/templates/autogen.gotmpl}

 ${https://host/path} or ${http://host/path}

   The contents of the URL are fetched and used similarly as for
   local files above. The URL is passed to [http.Get] and supports
   proxies, embedded Basic Authentication and other features from
   that function.

 The prefix below can be enabled with the [config.Expressions()] option.

 ${expr:EXPRESSION}

   EXPRESSION is evaluated using [github.com/maja42/goval]. Inside
   the expression all configuration items are available as variables
   with the top level map `env` set to the environment variables
   available. All results are returned as strings. An empty string
   may mean there was an error in evaluating the expression.

Additional custom lookup prefixes can be added with the config.Prefix option.

The bare form "$name" is NOT supported, unlike os.Expand as this can unexpectedly match values containing valid literal dollar signs.

Expansion is not recursive. Configuration values are read and stored as literals and are expanded each time they are used. For each substitution any leading and trailing whitespace are removed. External sources are fetched each time they are used and so there may be a performance impact as well as the value unexpectedly changing during a process lifetime.

Any errors (particularly from substitutions from external files or remote URLs) may result in an empty or corrupt string being returned. Error returns are intentionally discarded and an empty string substituted. Where a value contains multiple expandable items processing will continue even after an error for one of them.

It is not currently possible to escape the syntax supported by ExpandString and if it is necessary to have a configuration value be a literal of the form "${name}" then you can set an otherwise unused item to the value and refer to it using the dotted syntax, e.g. for YAML 

config:
  real: ${config.literal}
  literal: "${unchanged}"

In the above a reference to ${config.real} will return the literal string ${unchanged} as there is no recursive lookups.

func GetByteSlice added in v1.4.1 

func GetByteSlice(s string, options ...ExpandOptions) []byte

GetByteSlice functions like viper.GetString but additionally calls Expand with the configuration value, passing any "values" maps and returning a byte slice

func GetInt added in v1.4.3 

func GetInt(s string, options ...ExpandOptions) int

GetInt functions like viper.GetInt but additionally calls ExpandString with the configuration value, passing any "values" maps. If the conversion fails then the value returned will be the one from strconv.ParseInt - typically 0 but can be the maximum integer value

func GetInt64 added in v1.4.3 

func GetInt64(s string, options ...ExpandOptions) int64

GetInt64 functions like viper.GetInt but additionally calls ExpandString with the configuration value, passing any "values" maps. If the conversion fails then the value returned will be the one from strconv.ParseInt - typically 0 but can be the maximum integer value

func GetString 

func GetString(s string, options ...ExpandOptions) string

GetString functions like viper.GetString but additionally calls ExpandString with the configuration value, passing any "values" maps

func GetStringMapString 

func GetStringMapString(s string, options ...ExpandOptions) map[string]string

GetStringMapString functions like viper.GetStringMapString but additionally calls ExpandString on each value element of the map, passing any "values" maps

func GetStringSlice 

func GetStringSlice(s string, options ...ExpandOptions) []string

GetStringSlice functions like viper.GetStringSlice but additionally calls ExpandString on each element of the slice, passing any "values" maps

func ReadPasswordFile added in v1.4.1 

func ReadPasswordFile(path string) []byte

func ReadPasswordPrompt added in v1.4.1 

func ReadPasswordPrompt(prompt ...string) []byte

func SetStringMapString added in v1.4.3 

func SetStringMapString(m string, vals map[string]string)

SetMap iterates over a map[string]string and sets each key to the value given. Viper's Set() doesn't support maps until the configuration is written to and read back from a file.

func UserConfigDir added in v1.3.2 

func UserConfigDir(username ...string) (path string, err error)

Types

type AESValues 

type AESValues struct {
	Key []byte
	IV  []byte
}

An AESValues structure contains the values required to create a Geneos Gateway AES key file and then to encode and decode AES passwords in configurations

func NewAESValues 

func NewAESValues() (a AESValues, err error)

NewAESValues returns a new AESValues structure or an error

func ReadAESValues 

func ReadAESValues(r io.Reader) (a AESValues, err error)

ReadAESValues returns an AESValues struct populated with the contents read from r. The caller must close the Reader on return.

func ReadAESValuesFile 

func ReadAESValuesFile(path string) (a AESValues, err error)

ReadAESValuesFile returns an AESValues struct populated with the contents of the file passed as path.

func (*AESValues) Checksum 

func (a *AESValues) Checksum() (c uint32, err error)

Checksum returns the CRC32 checksum of the AESValue it is called on.

func (AESValues) DecodeAES 

func (a AESValues) DecodeAES(in []byte) (out []byte, err error)

DecodeAES returns the decoded value of in bytes using the AESValues given as the method receiver. Any prefix of "+encs+" is trimmed before decode. If decoding fails out is empty and error will contain the reason.

func (AESValues) DecodeAESString 

func (a AESValues) DecodeAESString(in string) (out string, err error)

DecodeAESString returns a plain text of the input or an error

func (AESValues) EncodeAES 

func (a AESValues) EncodeAES(in []byte) (out []byte, err error)

func (AESValues) EncodeAESBytes added in v1.4.1 

func (a AESValues) EncodeAESBytes(in []byte) (out []byte, err error)

func (AESValues) EncodeAESString 

func (a AESValues) EncodeAESString(in string) (out string, err error)

func (AESValues) String 

func (a AESValues) String() string

String method for AESValues

The output is in the format for suitable for use as a gateway key file for secure passwords as described in: https://docs.itrsgroup.com/docs/geneos/current/Gateway_Reference_Guide/gateway_secure_passwords.htm

func (AESValues) WriteAESValues 

func (a AESValues) WriteAESValues(w io.Writer) error

WriteAESValues writes the AESValues structure to the io.Writer. Each fields acts as if it were being marshalled with an ",omitempty" tag.

type Config 

type Config struct {
	*viper.Viper
	Type string
	// contains filtered or unexported fields
}

Config embeds Viper and also exposes the config type used

func GetConfig 

func GetConfig() *Config

GetConfig returns the global Config instance

func LoadConfig 

func LoadConfig(configName string, options ...Options) (c *Config, err error)

LoadConfig loads configuration files from internal defaults, external defaults and the given configuration file(s). The configuration file(s) can be passed as an option. Each layer is only loaded once, if given. Internal defaults are passed as a byte slice - this is typically loaded from an embedded file but can be supplied from any source. External defaults, which have a `.defaults` suffix before the file extension, and the main configuration file are passed as ordered slices of strings. The first match is loaded unless the MergeSettings() option is passed, in which case all defaults are merged and then all non-defaults are merged in the order they were given. 

LoadConfig("geneos")

//go:embed somefile.json
var myDefaults []byte
LoadConfig("geneos", config.SetDefaults(myDefaults, "json"), config.SetConfigFile(configPath))

Options can be passed to change the default behaviour and to pass any embedded defaults or an existing viper.

for defaults see: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

TBD: windows equiv of above

func New 

func New() *Config

New returns a Config instance initialised with a new viper instance

func ReadHOCONFile added in v1.4.1 

func ReadHOCONFile(path string) (cf *Config, err error)

func (*Config) DefaultExpandOptions added in v1.4.1 

func (c *Config) DefaultExpandOptions(options ...ExpandOptions)

DefaultExpandOptions sets defaults to all subsequent calls to functions that perform configuration expansion. These defaults can be reset by calling DefaultExpandOptions with no arguments.

func (*Config) Expand added in v1.4.1 

func (c *Config) Expand(input string, options ...ExpandOptions) (value []byte)

Expand behaves like the ExpandString method but returns a byte slice.

func (*Config) ExpandAllSettings 

func (c *Config) ExpandAllSettings(options ...ExpandOptions) (all map[string]interface{})

ExpandAllSettings returns all the settings from c applying ExpandString() to all string values and all string slice values. Further types may be added over time.

func (*Config) ExpandString 

func (c *Config) ExpandString(input string, options ...ExpandOptions) (value string)

ExpandString works just like the package level ExpandString but on a specific config instance.

func (*Config) GetByteSlice added in v1.4.1 

func (c *Config) GetByteSlice(s string, options ...ExpandOptions) []byte

GetByteSlice functions like viper.GetString on a Config instance, but additionally calls Expand with the configuration value, passing any "values" maps and returning a byte slice

func (*Config) GetInt added in v1.4.3 

func (c *Config) GetInt(s string, options ...ExpandOptions) (i int)

GetInt functions like viper.GetInt on a Config instance, but additionally calls ExpandString with the configuration value, passing any "values" maps, before converting the result to an int. If the conversion fails then the value returned will be the one from strconv.ParseInt - typically 0 but can be the maximum integer value

func (*Config) GetInt64 added in v1.4.3 

func (c *Config) GetInt64(s string, options ...ExpandOptions) (i int64)

GetInt64 functions like viper.GetInt on a Config instance, but additionally calls ExpandString with the configuration value, passing any "values" maps, before converting the result to an int. If the conversion fails then the value returned will be the one from strconv.ParseInt - typically 0 but can be the maximum integer value

func (*Config) GetString 

func (c *Config) GetString(s string, options ...ExpandOptions) string

GetString functions like viper.GetString on a Config instance, but additionally calls ExpandString with the configuration value, passing any "values" maps

func (*Config) GetStringMapString 

func (c *Config) GetStringMapString(s string, options ...ExpandOptions) (m map[string]string)

GetStringMapString functions like viper.GetStringMapString on a Config instance but additionally calls ExpandString on each value element of the map, passing any "values" maps

func (*Config) GetStringSlice 

func (c *Config) GetStringSlice(s string, options ...ExpandOptions) (slice []string)

GetStringSlice functions like viper.GetStringSlice on a Config instance but additionally calls ExpandString on each element of the slice, passing any "values" maps

func (*Config) MergeHOCONConfig added in v1.4.1 

func (cf *Config) MergeHOCONConfig(conf string) (err error)

MergeHOCONConfig parses the HOCON configuration in conf and merges the results into the cf *config.Config object

func (*Config) MergeHOCONFile added in v1.4.1 

func (cf *Config) MergeHOCONFile(path string) (err error)

MergeHOCONFile reads a HOCON configuration file in path and merges the settings into the cf *config.Config object

func (*Config) SetStringMapString added in v1.4.3 

func (c *Config) SetStringMapString(m string, vals map[string]string)

SetMap iterates over a map[string]string and sets each key to the value given. Viper's Set() doesn't support maps until the configuration is written to and read back from a file.

func (*Config) Sub 

func (c *Config) Sub(key string) *Config

Sub returns a Config instance for the sub-key passed

type ExpandOptions added in v1.4.1 

type ExpandOptions func(*expandOptions)

ExpandOptions control the way configuration options undergo string expansion through the underlying ExpandString functions. ExpandOptions can be passed to any of the normal lookup functions that are provided to override viper versions, such as GetString.

e.g. 

s := config.GetString("config.value", ExternalLookups(false), LookupTable(configMap), Prefix("myconf", myFunc))

func Default added in v1.4.3 

func Default(value any) ExpandOptions

Default sets a default value to be returned if the resulting expansion of the whole config value is empty (after any optional trimming of leading and trailing spaces). This includes cases where external lookups fail or a configuration item is not found. If TrimSpace is false and the returned value consists wholly of whitespace then this is returned and not the default given here.

func Expressions added in v1.4.1 

func Expressions(yes bool) ExpandOptions

Expressions enables or disables the built-in expansion for expressions via the `github.com/maja42/goval` package. The default is false.

func ExternalLookups added in v1.4.1 

func ExternalLookups(yes bool) ExpandOptions

ExternalLookups enables or disables the built-in expansion options that fetch data from outside the program, such as URLs and file paths. The default is true.

func LookupTable added in v1.4.1 

func LookupTable(values map[string]string) ExpandOptions

LookupTable adds a lookup map to the Expand functions. If there are no maps defined then `${item}` is looked up as an environment variable. When string expansion is done to a plain word, ie. without a prefix, then `${item}` is looked up in each map, in the order the LookupTable options are given, and first match, if any, wins. If there is no match in any of the lookup maps then a nil value is returned and the environment variables are not checked.

func Prefix added in v1.4.1 

func Prefix(prefix string, fn func(*Config, string, bool) string) ExpandOptions

Prefix defines a custom mapping for the given prefix to an expand-like function. The prefix should not include the terminating ":". If the configuration prefix matches during expansion then the function is called with the config data and the contents of the expansion including the prefix (for URLs) but stripped of the opening `${` and closing `}`

func TrimPrefix added in v1.4.1 

func TrimPrefix() ExpandOptions

TrimPrefix enables the removal of the prefix from the string passed to expansion functions. If this is not set then URLs can be passed as-is since the prefix is part of the URL. If set then URLs would need the schema explicitly added after the prefix. Using this option allows standard function like strings.ToUpper to be used without additional wrappers.

func TrimSpace added in v1.4.3 

func TrimSpace(yes bool) ExpandOptions

TrimSpace enables the removal of leading and trailing spaces on all values in an expansion. The default is `true`. If a default value is given using the Default() then this is never trimmed.

type Options 

type Options func(*configOptions)

func AddConfigDirs 

func AddConfigDirs(paths ...string) Options

AddConfigDirs adds one or more directories to search for the configuration and defaults files. Directories are searched in order, so any directories set with this option are checked before the built-in list. This option can be given multiple times as each call appends to the existing list..

func Global added in v1.4.1 

func Global() Options

Global tells LoadConfig to set values in the global configuration structure instead of creating a new one. The global configuration is returned by LoadConfig.

func IgnoreSystemDir 

func IgnoreSystemDir() Options

IgnoreSystemDir tells LoadConfig() not to search in the system configuration directory. This only applies on UNIX-like systems and is normally `/etc` and a sub-directory of AppName.

func IgnoreUserConfDir 

func IgnoreUserConfDir() Options

IgnoreUserConfDir tells LoadConfig not to search under the user config directory (The user configuration directory is as per os.UserConfDir and a sub-directory of AppName)

func IgnoreWorkingDir 

func IgnoreWorkingDir() Options

IgnoreWorkingDir tells LoadConfig not to search the working directory for configuration files. This should be used when the caller may be running from an unknown or untrusted location.

func MergeSettings added in v1.4.1 

func MergeSettings() Options

MergeSettings change the default behaviour of LoadConfig which is to load the first configuration file found, instead loading each configuration file found and merges the settings together. Merging is done using viper.MergeConfigMap and should result in the last definition of each configuration item being used.

MergeSettings applies to both default and main settings, but separately.

func SetAppName 

func SetAppName(name string) Options

SetAppName overrides to use of the LoadConfig `configName` argument as the application name, `AppName`, which is used for sub-directories while `configName“ is used as the prefix for files in those directories.

func SetConfigFile 

func SetConfigFile(path string) Options

SetConfigFile forces LoadConfig to load only the configuration at the given path. This path must include the file extension. Defaults are still loaded using the normal directories unless [IgnoreDefaults] is also passed as an option.

func SetDefaults 

func SetDefaults(defaults []byte, format string) Options

SetDefaults takes a []byte slice and a format type to set configuration defaults. This can be used in conjunction with `embed` to set embedded default configuration values so that a program can function without a configuration file, e.g. 

//go:embed "defaults.yaml"
var defaults []byte
...
c, err := config.LoadConfig("appname", config.SetDefaults(defaults, "yaml"))

func UseDefaults added in v1.4.1 

func UseDefaults(b bool) Options

UseDefaults tells LoadConfig whether to load defaults or not. The default is true.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.