config

README

config

• Project README
• Release/Build Status
• Changelog
• License
• Contribution Guidelines
• Code of Conduct
• Secure Coding Guidelines

Table of Contents

1. Description
2. Structure and Organisation
3. Class Diagram
4. Functionality
5. Data Types
6. Testing
7. Proposed Functionality/Requirements
8. References

Specification

Description

This package contains all configuration related code such as reading config file and functions to configure at runtime.

proposed There are two sides to configuration:

1. default configuration, which has to be loaded for the fresh installation of new dms;
2. dynamic configuration, which can be changed by a user that has access to the DMS; this dynamic configuration may need to be persistent (or not).

proposed Default configuration

Default configuration should be included into DMS distribution as a config.yaml in the root directory. The following is loosely based on general practice of passing yaml configuration to Go programs (see e.g. A clean way to pass configs in a Go application). DMS would parse this file during onboarding and populate the internal.config.Config variable that will be imported to other packages and used accordingly.

proposed Dynamic configuration

Dynamic configuration would use the same internal.config.Config variable, but would allow for adding new values or changing configuration by an authorized DMS user -- via DMS CLI or REST API calls.

The mechanism of dynamic configuration will enable to override or change default values. For enabling this functionality, the internal.config.Config variable will have a synchronized copy in the local DMS database, defined with db package.

Structure and Organisation

Here is quick overview of the contents of this pacakge:

• README: Current file which is aimed towards developers who wish to use and modify the package functionality.

• config: This file contains data structures for this package.

• load: This file establishes a configuration loader using Viper, supports loading JSON files from various locations, applies defaults, and exposes functions to manage and obtain the loaded configuration.

Class Diagram

Source

config class diagram

Rendered from source file

!$rootUrlGitlab = "https://gitlab.com/nunet/device-management-service/-/raw/main"
!$packageRelativePath = "/internal/config"
!$packageUrlGitlab = $rootUrlGitlab + $packageRelativePath
 
!include $packageUrlGitlab/specs/class_diagram.puml

Functionality

The methods of this package are explained below:

getViper

• signature: getViper() *viper.Viper
  

• input: None
  

• output: A pointer to a viper.Viper struct

getViper function creates a new viper.Viper instance used for configuration management with search paths.

setDefaultConfig

• signature: setDefaultConfig() *viper.Viper
  

• input: None
  

• output: A pointer to a viper.Viper struct

Sets default values for various configuration options in a viper.Viper instance.

LoadConfig

• signature: LoadConfig()
  

• input: None
  

• output: None

LoadConfig loads the configuration from the search paths.

SetConfig

• signature: SetConfig(key string, value interface{})
  

• input #1 : key for the configuration value
  

• input #2 : value corresponding to the key provided
  

• output: None

SetConfig Sets a specific configuration value using key value pair provided. In case of any error, the deafult configuration values are applied.

GetConfig

• signature: GetConfig() *Config
  

• input: None
  

• output: internal.config.Config

GetConfig returns the loaded configuration data.

findConfig

• signature: findConfig(paths []string, filename string) ([]byte, error)
  

• input # 1: list of search paths
  

• input # 1: name of the configuration file
  

• output: contents of the configuration file

• output(error): error message

findConfig Searches for the configuration file in specified paths and returns content or error.

removeComments

• signature: removeComments(configBytes []byte) []byte
  

• input: The byte array containing the configuration file content
  

• output: byte array with comments removed

removeComments Removes comments from the configuration file content using a regular expression.

Data Types

• internal.config.Config: holds the overall configuration with nested structs for specific sections

type Config struct {
	General `mapstructure:"general"`
	Rest    `mapstructure:"rest"`
	P2P     `mapstructure:"p2p"`
	Job     `mapstructure:"job"`
}

• internal.config.General: Configuration related to general application behavior (data paths, debug mode)

type General struct {
	MetadataPath string `mapstructure:"metadata_path"`
	DataDir      string `mapstructure:"data_dir"`
	Debug        bool   `mapstructure:"debug"`
}

• internal.config.Rest: Configuration for the REST API (port number)

type Rest struct {
	Port int `mapstructure:"port"`
}

• internal.config.P2P: Configuration for the P2P network

type P2P struct {
	ListenAddress  []string `mapstructure:"listen_address"`
	BootstrapPeers []string `mapstructure:"bootstrap_peers"`
}

• internal.config.Job: Configuration for background tasks (log update interval, target peer for deployments, container cleanup interval)

type Job struct {
	LogUpdateInterval int    `mapstructure:"log_update_interval"` // in minutes
	TargetPeer        string `mapstructure:"target_peer"`         // specific peer to send deployment requests to - XXX probably not a good idea. Remove after testing stage.
	CleanupInterval   int    `mapstructure:"cleanup_interval"` // docker container and images clean up interval in days
}

Testing

proposed Unit tests for each functionality are defined in files with *_test.go naming convention.

Proposed Functionality / Requirements

List of issues

All issues that are related to the implementation of internal package can be found below. These include any proposals for modifications to the package or new functionality needed to cover the requirements of other packages.

• internal package implementation TBD

proposed Functionalities

Following Gherkin feature files describe the proposed functionality for config package.

1. Load default DMS configuration: see scenario definition

2. Restore default DMS configuration: see scenario definition

3. Load existing DMS configuration: see scenario definition

References

Documentation

Index

• func CreateConfigFileIfNotExists(fs afero.Fs) error
• func FileExists(fs afero.Fs) (bool, error)
• func Get(key string) (interface{}, error)
• func GetPath() string
• func LoadConfig() error
• func Set(fs afero.Fs, key string, value interface{}) error
• func SetupConfig()
• type APM
• type Config
  • func GetConfig() *Config
• type General
• type Job
• type Observability
• type P2P
• type Profiler
• type Rest

Functions

func CreateConfigFileIfNotExists

func CreateConfigFileIfNotExists(fs afero.Fs) error

func FileExists

func FileExists(fs afero.Fs) (bool, error)

func Get

func Get(key string) (interface{}, error)

func GetPath

func GetPath() string

func LoadConfig

func LoadConfig() error

func Set

func Set(fs afero.Fs, key string, value interface{}) error

func SetupConfig

func SetupConfig()

Types

type APM

type APM struct {
	ServerURL   string `mapstructure:"server_url" json:"server_url"`
	ServiceName string `mapstructure:"service_name" json:"service_name"`
	Environment string `mapstructure:"environment" json:"environment"`
	APIKey      string `mapstructure:"api_key" json:"api_key"`
}

type Config

type Config struct {
	Profiler      `mapstructure:"profiler" json:"profiler"`
	General       `mapstructure:"general" json:"general"`
	Rest          `mapstructure:"rest" json:"rest"`
	P2P           `mapstructure:"p2p" json:"p2p"`
	Job           `mapstructure:"job" json:"job"`
	Observability `mapstructure:"observability" json:"observability"`
	APM           `mapstructure:"apm" json:"apm"`
}

func GetConfig

func GetConfig() *Config

type General

type General struct {
	UserDir                string `mapstructure:"user_dir" json:"user_dir"`
	WorkDir                string `mapstructure:"work_dir" json:"work_dir"`
	DataDir                string `mapstructure:"data_dir" json:"data_dir"`
	Debug                  bool   `mapstructure:"debug" json:"debug"`
	HostCountry            string `mapstructure:"host_country" json:"host_country"`
	HostCity               string `mapstructure:"host_city" json:"host_city"`
	HostContinent          string `mapstructure:"host_continent" json:"host_continent"`
	PortAvailableRangeFrom int    `mapstructure:"port_available_range_from" json:"port_available_range_from"`
	PortAvailableRangeTo   int    `mapstructure:"port_available_range_to" json:"port_available_range_to"`
}

type Job

type Job struct {
	AllowPrivilegedDocker bool `mapstructure:"allow_privileged_docker" json:"allow_privileged_docker"`
}

type Observability

type Observability struct {
	LogLevel             string `mapstructure:"log_level" json:"log_level"`
	LogFile              string `mapstructure:"log_file" json:"log_file"`
	MaxSize              int    `mapstructure:"max_size" json:"max_size"` // in megabytes
	MaxBackups           int    `mapstructure:"max_backups" json:"max_backups"`
	MaxAge               int    `mapstructure:"max_age" json:"max_age"` // in days
	ElasticsearchURL     string `mapstructure:"elasticsearch_url" json:"elasticsearch_url"`
	ElasticsearchIndex   string `mapstructure:"elasticsearch_index" json:"elasticsearch_index"`
	FlushInterval        int    `mapstructure:"flush_interval" json:"flush_interval"`               // in seconds
	ElasticsearchEnabled bool   `mapstructure:"elasticsearch_enabled" json:"elasticsearch_enabled"` // disable elastic logging
	ElasticsearchAPIKey  string `mapstructure:"elasticsearch_api_key" json:"elasticsearch_api_key"`
	InsecureSkipVerify   bool   `mapstructure:"insecure_skip_verify" json:"insecure_skip_verify"` // allow insecure TLS connections
}

type P2P

type P2P struct {
	ListenAddress   []string `mapstructure:"listen_address" json:"listen_address"`
	BootstrapPeers  []string `mapstructure:"bootstrap_peers" json:"bootstrap_peers"`
	Memory          int      `mapstructure:"memory" json:"memory"`
	FileDescriptors int      `mapstructure:"fd" json:"fd"`
}

type Profiler

type Profiler struct {
	Enabled bool   `mapstructure:"enabled" json:"enabled"`
	Addr    string `mapstructure:"addr" json:"addr"`
	Port    uint32 `mapstructure:"port" json:"port"`
}

type Rest

type Rest struct {
	Addr string `mapstructure:"addr" json:"addr"`
	Port uint32 `mapstructure:"port" json:"port"`
}