config

package
v0.2.0-beta Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 25, 2022 License: GPL-3.0 Imports: 8 Imported by: 0

Documentation

Overview

Config controls the overall configuration of the application.

It is generated by first attempting to read a configuration file and then overwriting those values with anything found in environment variables. Environment variables always come last and have the highest priority. As per (https://12factor.net/config).

All environment variables are prefixed with "GOFER". Ex: GOFER_DEBUG=true

Most envvar configuration abides by common envvar string=string formatting: Ex. `export GOFER_DEBUG=true`. Complex envvars like "Triggers" for example take a json string as value. 

Example: export GOFER_TRIGGERS="{"name":"test"},{"name":"test2"}"

You can print out a current description of current environment variable configuration by using the cli command: 

`gofer service printenv`

Note: Even though this package uses the envconfig package it is incorrect to use the 'default' struct tags as that will cause incorrect overwriting of user defined configurations.

Note: Because of the idiosyncrasies of how hcl conversion works certain advanced types like `time.Duration` need to have a sister variable that we read in through hcl via another type and convert to the actual wanted type.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func PrintAPIEnvs 

func PrintAPIEnvs() error

func PrintCLIEnvs 

func PrintCLIEnvs() error

Types

type API 

type API struct {
	// Controls the ability to trigger runs. This setting can be toggled while the server is running.
	IgnorePipelineRunEvents bool `split_words:"true" hcl:"ignore_pipeline_run_events,optional"`

	// Controls how long Gofer will hold onto events before discarding them. This is important factor in disk space
	// and memory footprint.
	//
	// Example: Rough math on a 5,000 pipeline Gofer instance with a full 6 months of retention
	//  puts the memory and storage footprint at about 9GB.
	EventLogRetention time.Duration `split_words:"true"`

	// EventLogRetentionHCL is the HCL compatible counter part to EventLogRetention. It allows the parsing of a string
	// to a time.Duration since HCL does not support parsing directly into a time.Duration.
	EventLogRetentionHCL string `ignored:"true" hcl:"event_log_retention,optional"`

	// Key used for encryption of secret values specific to Gofer operation. Ex. API tokens, trigger keys, etc
	EncryptionKey string `split_words:"true" hcl:"encryption_key,optional"`

	// How often the background process for pruning events should run.
	PruneEventsInterval time.Duration `split_words:"true"`

	// PruneEventsIntervalHCL is the HCL compatible counter part to PruneEventsInterval. It allows the parsing of a string
	// to a time.Duration since HCL does not support parsing directly into a time.Duration.
	PruneEventsIntervalHCL string `ignored:"true" hcl:"prune_events_interval,optional"`

	// URL for the server to bind to. Ex: localhost:8080
	Host string `hcl:"host,optional"`

	// Log level affects the entire application's logs including launched triggers.
	LogLevel string `split_words:"true" hcl:"log_level,optional"`

	// The total amount of runs before logs of the oldest log starts being deleted.
	RunLogExpiry int `split_words:"true" hcl:"run_log_expiry,optional"`

	// Directory to store task run log files.
	TaskRunLogsDir string `split_words:"true" hcl:"task_run_logs_dir,optional"`

	// TaskRunStopTimeout controls the time the scheduler will wait for a normal user container(non-trigger containers)
	// to stop. When the timeout is reached the container will be forcefully terminated.
	// You can use a negative duration("-1s") to convey that no timeout should be specified and the scheduler
	// should wait however long it takes the container to respond to the terminate signal.
	// This is usually passed to the scheduler when a request to cancel a task run is being made.
	TaskRunStopTimeout time.Duration `split_words:"true"`

	// TaskRunStopTimeoutHCL is the HCL compatible counter part to TaskRunStopTimeout. It allows the parsing of a string
	// to a time.Duration since HCL does not support parsing directly into a time.Duration.
	TaskRunStopTimeoutHCL string `ignored:"true" hcl:"task_run_stop_timeout,optional"`

	ExternalEventsAPI *ExternalEventsAPI `split_words:"true" hcl:"external_events_api,block"`
	Database          *Database          `hcl:"database,block"`
	ObjectStore       *ObjectStore       `hcl:"object_store,block"`
	SecretStore       *SecretStore       `hcl:"secret_store,block"`
	Scheduler         *Scheduler         `hcl:"scheduler,block"`
	Server            *Server            `hcl:"server,block"`
	Triggers          *Triggers          `hcl:"triggers,block"`
	Notifiers         *Notifiers         `hcl:"notifiers,block"`
}

API defines config settings for the gofer server

func DefaultAPIConfig 

func DefaultAPIConfig() *API

func InitAPIConfig 

func InitAPIConfig(userDefinedPath string) (*API, error)

Get the final configuration for the server. This involves correctly finding and ordering different possible paths for the configuration file.

1) The function is intended to be called with paths gleaned from the -config flag 2) Then combine that with possible other config locations that the user might store a config file. 3) Then try to see if the user has set an envvar for the config file, which overrides all previous config file paths. 4) Finally, pass back whatever is deemed the final config path from that process.

We then use that path data to find the config file and read it in via HCL parsers. Once that is finished we then take any configuration from the environment and superimpose that on top of the final config struct.

func (*API) FromBytes 

func (c *API) FromBytes(content []byte) error

FromBytes attempts to parse a given HCL configuration.

func (*API) FromEnv 

func (c *API) FromEnv() error

FromEnv parses environment variables into the config object based on envconfig name

func (*API) FromFile 

func (c *API) FromFile(path string) error

type BoltDB 

type BoltDB struct {
	Path string `hcl:"path,optional"` // file path for database file
}

BoltDB: https://pkg.go.dev/go.etcd.io/bbolt

type BoltDBSecret added in v0.0.3 

type BoltDBSecret struct {
	Path string `hcl:"path,optional"` // file path for database file
	// EncryptionKey is a 32-bit random string of characters used to encrypt data at rest.
	EncryptionKey string `split_words:"true" hcl:"encryption_key,optional"`
}

BoltDBSecret: https://pkg.go.dev/go.etcd.io/bbolt

type CLI 

type CLI struct {
	Namespace string `split_words:"true" hcl:"namespace,optional"`
	Detail    bool   `hcl:"detail,optional"`
	Format    string `hcl:"format,optional"`
	Host      string `hcl:"host,optional"`
	NoColor   bool   `split_words:"true" hcl:"no_color,optional"`
	Token     string `hcl:"token,optional"`
}

func DefaultCLIConfig 

func DefaultCLIConfig() *CLI

DefaultCLIConfig returns a pre-populated configuration struct that is used as the base for super imposing user configuration settings.

func InitCLIConfig 

func InitCLIConfig(flagPath string) (*CLI, error)

Get the final configuration for the CLI. This involves correctly finding and ordering different possible paths for the configuration file.

1) The function is intended to be called with paths gleaned from the -config flag 2) Then combine that with possible other config locations that the user might store a config file. 3) Then try to see if the user has set an envvar for the config file, which overrides all previous config file paths. 4) Finally, pass back whatever is deemed the final config path from that process.

We then use that path data to find the config file and read it in via HCL parsers. Once that is finished we then take any configuration from the environment and superimpose that on top of the final config struct.

func (*CLI) FromBytes 

func (c *CLI) FromBytes(content []byte) error

FromBytes attempts to parse a given HCL configuration.

func (*CLI) FromEnv 

func (c *CLI) FromEnv() error

FromEnv parses environment variables into the config object based on envconfig name

func (*CLI) FromFile 

func (c *CLI) FromFile(path string) error

type Database 

type Database struct {
	// The database engine used by the backend
	// possible values are: bolt
	Engine string `hcl:"engine,optional"`

	// MaxResultsLimit defines the total number of results the database can return in one call to any "GETALL" endpoint.
	MaxResultsLimit int     `split_words:"true" hcl:"max_results_limit,optional"`
	BoltDB          *BoltDB `hcl:"boltdb,block"`
}

Database defines config settings for gofer database

func DefaultDatabaseConfig 

func DefaultDatabaseConfig() *Database

type Docker 

type Docker struct {
	// Prune runs a reoccuring `docker system prune` job to avoid filling the local disk with docker images.
	Prune bool `hcl:"prune,optional"`

	// The period of time in between runs of `docker system prune`
	PruneInterval time.Duration `split_words:"true"`

	// PruneIntervalHCL is the HCL compatible counter part to PruneInterval. It allows the parsing of a string
	// to a time.Duration since HCL does not support parsing directly into a time.Duration.
	PruneIntervalHCL string `ignored:"true" hcl:"prune_interval,optional"`
}

func DefaultDockerConfig 

func DefaultDockerConfig() *Docker

type ExternalEventsAPI 

type ExternalEventsAPI struct {
	Enable bool `hcl:"enable,optional"`

	// URL for the server to bind to. Ex: localhost:8080
	Host string `hcl:"host,optional"`
}

ExternalEventsAPI controls how the settings around the HTTP service that handles external trigger events.

func DefaultExternalEventsAPIConfig 

func DefaultExternalEventsAPIConfig() *ExternalEventsAPI

type Frontend 

type Frontend struct {
	Enable bool `hcl:"enable,optional"`
}

Frontend represents configuration for frontend basecoat

type Notifier 

type Notifier struct {
	// The name for a trigger this should be alphanumerical and can't contain spaces.
	Kind string `json:"kind" hcl:"kind,label" storm:"id"`

	// The docker repository and image name of the trigger: Ex. docker.io/library/hello-world:latest
	Image string `json:"image" hcl:"image"`

	// The user id for the docker repository; if needed.
	User string `json:"user" hcl:"user,optional"`

	// The password for the docker repository; if needed.
	Pass string `json:"pass" hcl:"pass,optional"`

	// Environment variables to pass to the trigger container. This is used to pass runtime settings to the container.
	EnvVars map[string]string `json:"env_vars" hcl:"env_vars,optional"`
}

Notifier represents the settings for all notifiers within Gofer.

type Notifiers 

type Notifiers struct {
	// RegisteredNotifiers represents the notifiers that Gofer will attempt to startup with.
	RegisteredNotifiers RegisteredNotifiers `split_words:"true" hcl:"registered_notifiers,block"`
}

Notifiers represents the configuration for Gofer Notifiers. Notifiers are used to perform some action upon the completion of a run.

func DefaultNotifiersConfig 

func DefaultNotifiersConfig() *Notifiers

type ObjectStore 

type ObjectStore struct {
	// The ObjectStore engine used by the backend.
	// Possible values are: bolt
	Engine string `hcl:"engine,optional"`

	BoltDB *BoltDB `hcl:"boltdb,block"`

	// Pipeline Objects last forever but are limited in number. This is the total amount of items that can be stored
	// per pipeline before gofer starts deleting objects.
	PipelineObjectLimit int `split_words:"true" hcl:"pipeline_object_limit,optional"`

	// Objects stored at the run level are unlimited in number, but only last for a certain number of runs.
	// The number below controls how many runs until the run objects for the oldest run will be deleted.
	// Ex. an object stored on run number #5 with an expiry of 2 will be deleted on run #7 regardless of run
	// health.
	RunObjectExpiry int `split_words:"true" hcl:"run_object_expiry,optional"`
}

ObjectStore defines config settings for gofer ObjectStore. The ObjectStore stores temporary objects for pipelines and runs.

func DefaultObjectStoreConfig 

func DefaultObjectStoreConfig() *ObjectStore

type RegisteredNotifiers 

type RegisteredNotifiers []Notifier

Notifiers represents the list of notifiers that Gofer will attempt to startup with and use.

func (*RegisteredNotifiers) Set 

func (t *RegisteredNotifiers) Set(value string) error

Set is a method that implements the ability for envconfig to unfurl a notifier mentioned as an environment variable. Basically the notifier is just wrapped up as a json blurb and set will unwrap it into the proper struct.

type RegisteredTriggers 

type RegisteredTriggers []Trigger

RegisteredTrigger represents the list of triggers that Gofer will attempt to startup with and use.

func (*RegisteredTriggers) Set 

func (t *RegisteredTriggers) Set(value string) error

Set is a method that implements the ability for envconfig to unfurl a trigger mentioned as an environment variable. Basically the trigger is just wrapped up as a json blurb and set will unwrap it into the proper struct.

type Scheduler 

type Scheduler struct {
	// The database engine used by the scheduler
	// possible values are: docker
	Engine string  `hcl:"engine,optional"`
	Docker *Docker `hcl:"docker,block"`
}

Scheduler defines config settings for gofer scheduler. The scheduler is the backend for how containers are run.

func DefaultSchedulerConfig 

func DefaultSchedulerConfig() *Scheduler

type SecretStore added in v0.0.3 

type SecretStore struct {
	// The ObjectStore engine used by the backend.
	// Possible values are: bolt
	Engine string `hcl:"engine,optional"`

	BoltDB *BoltDBSecret `hcl:"boltdb,block"`
}

SecretStore defines the configuration for Gofer's secret backend.

func DefaultSecretStoreConfig added in v0.0.3 

func DefaultSecretStoreConfig() *SecretStore

type Server 

type Server struct {
	// DevMode turns on humanized debug messages, extra debug logging for the webserver and other
	// convenient features for development. Usually turned on along side LogLevel=debug.
	DevMode bool `hcl:"dev_mode,optional"`

	// How long the GRPC service should wait on in-progress connections before hard closing everything out.
	ShutdownTimeout time.Duration `split_words:"true"`

	// ShutdownTimeoutHCL is the HCL compatible counter part to ShutdownTimeout. It allows the parsing of a string
	// to a time.Duration since HCL does not support parsing directly into a time.Duration.
	ShutdownTimeoutHCL string `ignored:"true" hcl:"shutdown_timeout,optional"`

	TLSCertPath string `split_words:"true" hcl:"tls_cert_path,optional"`
	TLSKeyPath  string `split_words:"true" hcl:"tls_key_path,optional"`

	// Temporary storage for downloaded pipeline configs.
	TmpDir string `split_words:"true" hcl:"tmp_dir,optional"`
}

Server respresents lower level HTTP/GRPC server settings.

func DefaultServerConfig 

func DefaultServerConfig() *Server

DefaultServerConfig returns a pre-populated configuration struct that is used as the base for super imposing user configuration settings.

type Trigger 

type Trigger struct {
	// The name for a trigger this should be alphanumerical and can't contain spaces.
	Kind string `json:"kind" hcl:"kind,label" storm:"id"`

	// The docker repository and image name of the trigger: Ex. docker.io/library/hello-world:latest
	Image string `json:"image" hcl:"image"`

	// The user id for the docker repository; if needed.
	User string `json:"user" hcl:"user,optional"`

	// The password for the docker repository; if needed.
	Pass string `json:"pass" hcl:"pass,optional"`

	// Environment variables to pass to the trigger container. This is used to pass runtime settings to the container.
	EnvVars map[string]string `json:"env_vars" hcl:"env_vars,optional"`
}

Trigger represents the settings for all triggers within Gofer.

type Triggers 

type Triggers struct {
	// StopTimeout controls the time the scheduler will wait for a trigger container to stop. After this period
	// Gofer will attempt to force stop the trigger container.
	StopTimeout time.Duration `split_words:"true"`

	// StopTimeoutHCL is the HCL compatible counter part to TriggerStopTimeout. It allows the parsing of a string
	// to a time.Duration since HCL does not support parsing directly into a time.Duration.
	StopTimeoutHCL string `ignored:"true" hcl:"stop_timeout,optional"`

	// HealthcheckInterval defines the period of time between attempted GRPC connections to all triggers. Triggers
	// are healthchecked to ensure proper operation.
	HealthcheckInterval time.Duration `split_words:"true"`

	// HealthcheckInternalHCL is the HCL compatible counter part to TriggerHealthcheck. It allows the parsing of a string
	// to a time.Duration since HCL does not support parsing directly into a time.Duration.
	HealthcheckIntervalHCL string `ignored:"true" hcl:"healthcheck_interval,optional"`

	// TLSCertPath is the file path of the trigger TLS certificate.
	TLSCertPath string `split_words:"true" hcl:"tls_cert_path,optional"`

	// TLSKeyPath is the file path of the trigger TLS key.
	TLSKeyPath string `split_words:"true" hcl:"tls_key_path,optional"`

	// RegisteredTriggers represents the triggers that Gofer will attempt to startup with.
	RegisteredTriggers RegisteredTriggers `split_words:"true" hcl:"registered_triggers,block"`
}

Triggers represents the configuration for Gofer Triggers. Triggers are used to generate events in which pipelines should run.

func DefaultTriggersConfig 

func DefaultTriggersConfig() *Triggers

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.