configuration

package
v0.342.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 16, 2024 License: Apache-2.0 Imports: 41 Imported by: 0

Documentation

Overview

Package configuration is the FTL configuration and secret management API.

The full design is documented here.

A Manager is the high-level interface to storing, listing, and retrieving secrets and configuration. A [Resolver] is the next layer, mapping names to a storage location key such as environment variables, keychain, etc. The Provider is the final layer, responsible for actually storing and retrieving values in concrete storage.

A constructed Manager and its providers are parametric on either secrets or configuration and thus cannot be used interchangeably.

Index

Constants

This section is empty.

Variables

View Source 
var ErrNotFound = errors.New("not found")

ErrNotFound is returned when a configuration entry is not found or cannot be resolved.

Functions

func ConfigFromEnvironment added in v0.201.0 

func ConfigFromEnvironment() []string

func ContextWithConfig added in v0.138.0 

func ContextWithConfig(ctx context.Context, configManager *Manager[Configuration]) context.Context

ContextWithConfig adds a configuration manager to the given context.

func ContextWithSecrets added in v0.138.0 

func ContextWithSecrets(ctx context.Context, secretsManager *Manager[Secrets]) context.Context

ContextWithSecrets adds a secrets manager to the given context.

func ProviderKeyForAccessor added in v0.281.0 

func ProviderKeyForAccessor(accessor *url.URL) string

Types

type ASM added in v0.236.0 

type ASM struct {
	// contains filtered or unexported fields
}

ASM implements a Provider for AWS Secrets Manager (ASM). Only supports loading "string" secrets, not binary secrets.

One controller is elected as the leader and is responsible for syncing the cache of secrets from ASM (see asmLeader). Others get secrets from the leader via AdminService (see asmFollower).

func NewASM added in v0.266.0 

func NewASM(ctx context.Context, secretsClient *secretsmanager.Client, advertise *url.URL, leaser leases.Leaser) *ASM

func (*ASM) Delete added in v0.236.0 

func (a *ASM) Delete(ctx context.Context, ref Ref) error

func (ASM) Key added in v0.236.0 

func (ASM) Key() string

func (ASM) Role added in v0.236.0 

func (ASM) Role() Secrets

func (*ASM) Store added in v0.236.0 

func (a *ASM) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

Store and if the secret already exists, update it.

func (*ASM) Sync added in v0.290.0 

func (a *ASM) Sync(ctx context.Context, entries []Entry, values *xsync.MapOf[Ref, SyncedValue]) error

func (*ASM) SyncInterval added in v0.290.0 

func (a *ASM) SyncInterval() time.Duration

type AsynchronousProvider added in v0.290.0 

type AsynchronousProvider[R Role] interface {
	Provider[R]

	SyncInterval() time.Duration

	// Sync is called periodically to update the cache with the latest values.
	//
	// SyncInterval() provides the expected time between syncs.
	// If Sync() returns an error, sync will be retried with an exponential backoff.
	//
	// Values should be updated by Sync().
	// An array of known entries from the router is provided in case it is helpful, but the provider can store any values it wants.
	Sync(ctx context.Context, entries []Entry, values *xsync.MapOf[Ref, SyncedValue]) error
}

AsynchronousProvider is an interface for providers that support syncing values. This is recommended if the provider allows batch access, or is expensive to load.

type Configuration added in v0.138.0 

type Configuration struct{}

func (Configuration) String added in v0.145.0 

func (Configuration) String() string

type DBConfigProvider added in v0.231.0 

type DBConfigProvider struct {
	// contains filtered or unexported fields
}

DBConfigProvider is a configuration provider that stores configuration in its key.

func NewDBConfigProvider added in v0.231.0 

func NewDBConfigProvider(dal DBConfigProviderDAL) DBConfigProvider

func (DBConfigProvider) Delete added in v0.231.0 

func (d DBConfigProvider) Delete(ctx context.Context, ref Ref) error

func (DBConfigProvider) Key added in v0.231.0 

func (DBConfigProvider) Key() string

func (DBConfigProvider) Load added in v0.231.0 

func (d DBConfigProvider) Load(ctx context.Context, ref Ref, key *url.URL) ([]byte, error)

func (DBConfigProvider) Role added in v0.231.0 

func (DBConfigProvider) Role() Configuration

func (DBConfigProvider) Store added in v0.231.0 

func (d DBConfigProvider) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

type DBConfigProviderDAL added in v0.231.0 

type DBConfigProviderDAL interface {
	GetModuleConfiguration(ctx context.Context, module optional.Option[string], name string) ([]byte, error)
	SetModuleConfiguration(ctx context.Context, module optional.Option[string], name string, value []byte) error
	UnsetModuleConfiguration(ctx context.Context, module optional.Option[string], name string) error
}

type DBConfigResolver added in v0.231.0 

type DBConfigResolver struct {
	// contains filtered or unexported fields
}

DBConfigResolver loads values a project's configuration from the given database.

func NewDBConfigResolver added in v0.231.0 

func NewDBConfigResolver(db DBConfigResolverDAL) DBConfigResolver

func (DBConfigResolver) Get added in v0.231.0 

func (d DBConfigResolver) Get(ctx context.Context, ref Ref) (*url.URL, error)

func (DBConfigResolver) List added in v0.231.0 

func (d DBConfigResolver) List(ctx context.Context) ([]Entry, error)

func (DBConfigResolver) Role added in v0.231.0 

func (d DBConfigResolver) Role() Configuration

func (DBConfigResolver) Set added in v0.231.0 

func (d DBConfigResolver) Set(ctx context.Context, ref Ref, key *url.URL) error

func (DBConfigResolver) Unset added in v0.231.0 

func (d DBConfigResolver) Unset(ctx context.Context, ref Ref) error

type DBConfigResolverDAL added in v0.231.0 

type DBConfigResolverDAL interface {
	ListModuleConfiguration(ctx context.Context) ([]sql.ModuleConfiguration, error)
}

type DBSecretResolver added in v0.276.5 

type DBSecretResolver struct {
	// contains filtered or unexported fields
}

DBSecretResolver loads values a project's secrets from the given database.

func NewDBSecretResolver added in v0.276.5 

func NewDBSecretResolver(db DBSecretResolverDAL) DBSecretResolver

func (DBSecretResolver) Get added in v0.276.5 

func (d DBSecretResolver) Get(ctx context.Context, ref Ref) (*url.URL, error)

func (DBSecretResolver) List added in v0.276.5 

func (d DBSecretResolver) List(ctx context.Context) ([]Entry, error)

func (DBSecretResolver) Role added in v0.276.5 

func (d DBSecretResolver) Role() Secrets

func (DBSecretResolver) Set added in v0.276.5 

func (d DBSecretResolver) Set(ctx context.Context, ref Ref, key *url.URL) error

func (DBSecretResolver) Unset added in v0.276.5 

func (d DBSecretResolver) Unset(ctx context.Context, ref Ref) error

type DBSecretResolverDAL added in v0.276.5 

type DBSecretResolverDAL interface {
	GetModuleSecretURL(ctx context.Context, module optional.Option[string], name string) (string, error)
	ListModuleSecrets(ctx context.Context) ([]dal.ModuleSecret, error)
	SetModuleSecretURL(ctx context.Context, module optional.Option[string], name string, url string) error
	UnsetModuleSecret(ctx context.Context, module optional.Option[string], name string) error
}

type Entry 

type Entry struct {
	Ref
	Accessor *url.URL
}

Entry in the configuration store.

type EnvarProvider 

type EnvarProvider[R Role] struct{}

EnvarProvider is a configuration provider that reads secrets or configuration from environment variables.

func (EnvarProvider[R]) Delete 

func (e EnvarProvider[R]) Delete(ctx context.Context, ref Ref) error

func (EnvarProvider[R]) Key 

func (EnvarProvider[R]) Key() string

func (EnvarProvider[R]) Load 

func (e EnvarProvider[R]) Load(ctx context.Context, ref Ref, key *url.URL) ([]byte, error)

func (EnvarProvider[R]) Role added in v0.145.0 

func (EnvarProvider[R]) Role() R

func (EnvarProvider[R]) Store 

func (e EnvarProvider[R]) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

type InlineProvider 

type InlineProvider[R Role] struct{}

InlineProvider is a configuration provider that stores configuration in its key.

func (InlineProvider[R]) Delete 

func (InlineProvider[R]) Delete(ctx context.Context, ref Ref) error

func (InlineProvider[R]) Key 

func (InlineProvider[R]) Key() string

func (InlineProvider[R]) Load 

func (InlineProvider[R]) Load(ctx context.Context, ref Ref, key *url.URL) ([]byte, error)

func (InlineProvider[R]) Role added in v0.145.0 

func (InlineProvider[R]) Role() R

func (InlineProvider[R]) Store 

func (InlineProvider[R]) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

type KeychainProvider 

type KeychainProvider struct{}

func (KeychainProvider) Delete 

func (k KeychainProvider) Delete(ctx context.Context, ref Ref) error

func (KeychainProvider) Key 

func (k KeychainProvider) Key() string

func (KeychainProvider) Load 

func (k KeychainProvider) Load(ctx context.Context, ref Ref, key *url.URL) ([]byte, error)

func (KeychainProvider) Role added in v0.145.0 

func (KeychainProvider) Role() Secrets

func (KeychainProvider) Store 

func (k KeychainProvider) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

type Manager 

type Manager[R Role] struct {
	// contains filtered or unexported fields
}

Manager is a high-level configuration manager that abstracts the details of the Router and Provider interfaces.

func ConfigFromContext added in v0.138.0 

func ConfigFromContext(ctx context.Context) *Manager[Configuration]

ConfigFromContext retrieves the configuration.Manager previously added to the context with ContextWithConfig.

func New 

func New[R Role](ctx context.Context, router Router[R], providers []Provider[R]) (*Manager[R], error)

New configuration manager.

func NewConfigurationManager added in v0.138.0 

func NewConfigurationManager(ctx context.Context, router Router[Configuration]) (*Manager[Configuration], error)

NewConfigurationManager creates a new configuration manager with the default configuration providers.

func NewDefaultConfigurationManagerFromConfig added in v0.201.0 

func NewDefaultConfigurationManagerFromConfig(ctx context.Context, config string) (*Manager[Configuration], error)

NewDefaultConfigurationManagerFromConfig creates a new configuration manager from the project config found in the config paths.

func NewDefaultSecretsManagerFromConfig added in v0.201.0 

func NewDefaultSecretsManagerFromConfig(ctx context.Context, config string, opVault string) (*Manager[Secrets], error)

NewDefaultSecretsManagerFromConfig creates a new secrets manager from the project config found in the config paths.

func NewSecretsManager added in v0.138.0 

func NewSecretsManager(ctx context.Context, router Router[Secrets], opVault string, config string) (*Manager[Secrets], error)

NewSecretsManager creates a new secrets manager with the default secret providers.

func SecretsFromContext added in v0.138.0 

func SecretsFromContext(ctx context.Context) *Manager[Secrets]

SecretsFromContext retrieves the secrets configuration.Manager previously added to the context with ContextWithConfig.

func (*Manager[R]) Get 

func (m *Manager[R]) Get(ctx context.Context, ref Ref, value any) error

Get a configuration value from the active providers.

"value" must be a pointer to a Go type that can be unmarshalled from JSON.

func (*Manager[R]) List 

func (m *Manager[R]) List(ctx context.Context) ([]Entry, error)

func (*Manager[R]) MapForModule added in v0.184.2 

func (m *Manager[R]) MapForModule(ctx context.Context, module string) (map[string][]byte, error)

MapForModule combines all configuration values visible to the module. Local values take precedence.

func (*Manager[R]) Set 

func (m *Manager[R]) Set(ctx context.Context, pkey string, ref Ref, value any) error

Set a configuration value, encoding "value" as JSON before storing it.

func (*Manager[R]) SetJSON added in v0.255.4 

func (m *Manager[R]) SetJSON(ctx context.Context, pkey string, ref Ref, value json.RawMessage) error

SetJSON sets a configuration value using raw JSON data.

func (*Manager[R]) Unset 

func (m *Manager[R]) Unset(ctx context.Context, pkey string, ref Ref) error

Unset a configuration value in all providers.

type ManualSyncProvider added in v0.296.6 

type ManualSyncProvider[R Role] struct {
	// contains filtered or unexported fields
}

ManualSyncProvider prevents normal syncs by returning a very high sync interval when syncAndWait() is called, it starts returning a 0 sync interval and then then blocks until sync completes. See why we didn't use mock clocks to schedule syncs here: https://github.com/TBD54566975/ftl/issues/2092

func NewManualSyncProvider added in v0.296.6 

func NewManualSyncProvider[R Role](provider AsynchronousProvider[R]) *ManualSyncProvider[R]

func (*ManualSyncProvider[R]) Delete added in v0.296.6 

func (a *ManualSyncProvider[R]) Delete(ctx context.Context, ref Ref) error

func (*ManualSyncProvider[R]) Key added in v0.296.6 

func (a *ManualSyncProvider[R]) Key() string

func (*ManualSyncProvider[R]) Role added in v0.296.6 

func (a *ManualSyncProvider[R]) Role() R

func (*ManualSyncProvider[R]) Store added in v0.296.6 

func (a *ManualSyncProvider[R]) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

func (*ManualSyncProvider[R]) Sync added in v0.296.6 

func (a *ManualSyncProvider[R]) Sync(ctx context.Context, entries []Entry, values *xsync.MapOf[Ref, SyncedValue]) error

func (*ManualSyncProvider[R]) SyncAndWait added in v0.296.6 

func (a *ManualSyncProvider[R]) SyncAndWait() error

func (*ManualSyncProvider[R]) SyncInterval added in v0.296.6 

func (a *ManualSyncProvider[R]) SyncInterval() time.Duration

type Obfuscator added in v0.277.0 

type Obfuscator struct {
	// contains filtered or unexported fields
}

Obfuscator hides and reveals a value, but does not provide real security instead the aim of this Obfuscator is to make values not easily human readable

Obfuscation is done by XOR-ing the input with the AES key. Length of key must be 16, 24 or 32 bytes (corresponding to AES-128, AES-192 or AES-256 keys).

func (Obfuscator) Obfuscate added in v0.277.0 

func (o Obfuscator) Obfuscate(input []byte) ([]byte, error)

Obfuscate takes a value and returns an obfuscated value (encoded in base64)

func (Obfuscator) Reveal added in v0.277.0 

func (o Obfuscator) Reveal(input []byte) ([]byte, error)

Reveal takes an obfuscated value and de-obfuscates the base64 encoded value

type ObfuscatorProvider added in v0.277.0 

type ObfuscatorProvider interface {
	// contains filtered or unexported methods
}

type OnePasswordProvider 

type OnePasswordProvider struct {
	Vault       string
	ProjectName string
}

OnePasswordProvider is a configuration provider that reads passwords from 1Password vaults via the "op" command line tool.

func (OnePasswordProvider) Delete 

func (o OnePasswordProvider) Delete(ctx context.Context, ref Ref) error

func (OnePasswordProvider) Key 

func (o OnePasswordProvider) Key() string

func (OnePasswordProvider) Role added in v0.145.0 

func (OnePasswordProvider) Role() Secrets

func (OnePasswordProvider) Store 

func (o OnePasswordProvider) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

Store will save the given secret in 1Password via the `op` command.

op does not support "create or update" as a single command. Neither does it support specifying an ID on create. Because of this, we need check if the item exists before creating it, and update it if it does.

func (OnePasswordProvider) Sync added in v0.290.0 

func (o OnePasswordProvider) Sync(ctx context.Context, entries []Entry, values *xsync.MapOf[Ref, SyncedValue]) error

Sync will fetch all secrets from the 1Password vault and store them in the values map. Do not just sync the o.Vault, instead find all vaults found in entries and sync them.

func (OnePasswordProvider) SyncInterval added in v0.290.0 

func (o OnePasswordProvider) SyncInterval() time.Duration

type ProjectConfigResolver 

type ProjectConfigResolver[R Role] struct {
	Config string `` /* 126-byte string literal not displayed */
}

ProjectConfigResolver is parametric Resolver that loads values from either a project's configuration or secrets maps based on the type parameter.

See the projectconfig package for details on the configuration file format.

func (ProjectConfigResolver[R]) Get 

func (p ProjectConfigResolver[R]) Get(ctx context.Context, ref Ref) (*url.URL, error)

func (ProjectConfigResolver[R]) List 

func (p ProjectConfigResolver[R]) List(ctx context.Context) ([]Entry, error)

func (ProjectConfigResolver[R]) Role added in v0.145.0 

func (p ProjectConfigResolver[R]) Role() R

func (ProjectConfigResolver[R]) Set 

func (p ProjectConfigResolver[R]) Set(ctx context.Context, ref Ref, key *url.URL) error

func (ProjectConfigResolver[From]) Unset 

func (p ProjectConfigResolver[From]) Unset(ctx context.Context, ref Ref) error

type Provider 

type Provider[R Role] interface {
	Role() R
	Key() string

	// Store a configuration value and return its key.
	Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)
	// Delete a configuration value.
	Delete(ctx context.Context, ref Ref) error
}

Provider is a generic interface for storing and retrieving configuration and secrets.

type Ref 

type Ref struct {
	Module optional.Option[string]
	Name   string
}

A Ref is a reference to a configuration value.

func NewRef 

func NewRef(module, name string) Ref

NewRef creates a new Ref.

If [module] is empty, the Ref is considered to be a global configuration value.

func ParseRef 

func ParseRef(s string) (Ref, error)

func (Ref) String 

func (k Ref) String() string

func (*Ref) UnmarshalText 

func (k *Ref) UnmarshalText(text []byte) error

type Role added in v0.138.0 

type Role interface {
	Secrets | Configuration
}

Role of Manager, either Secrets or Configuration.

type Router added in v0.271.4 

type Router[R Role] interface {
	Role() R
	Get(ctx context.Context, ref Ref) (key *url.URL, err error)
	Set(ctx context.Context, ref Ref, key *url.URL) error
	Unset(ctx context.Context, ref Ref) error
	List(ctx context.Context) ([]Entry, error)
}

A Router resolves configuration names to keys that are then used to load values from a Provider.

This indirection allows for the storage of configuration values to be abstracted from the configuration itself. For example, the ftl-project.toml file contains per-module and global configuration maps, but the secrets themselves may be stored in a separate secret store such as a system keychain.

type Secrets added in v0.138.0 

type Secrets struct{}

func (Secrets) String added in v0.145.0 

func (Secrets) String() string

type SyncedValue added in v0.290.0 

type SyncedValue struct {
	Value []byte

	// VersionToken is a way of storing a version provided by the source of truth (eg: lastModified)
	// it is nil when:
	// - the owner of the cache is not using version tokens
	// - the cache is updated after writing
	VersionToken optional.Option[VersionToken]
}

type SynchronousProvider added in v0.290.0 

type SynchronousProvider[R Role] interface {
	Provider[R]

	Load(ctx context.Context, ref Ref, key *url.URL) ([]byte, error)
}

SynchronousProvider is an interface for providers that can load values on-demand. This is recommended if the provider allows inexpensive loading of values.

type VersionToken added in v0.290.0 

type VersionToken any

Source Files

View all Source files

Directories

Path Synopsis
Package dal provides a data abstraction layer for managing module configurations
 Package dal provides a data abstraction layer for managing module configurations

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.