configuration

package
v0.351.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 20, 2024 License: Apache-2.0 Imports: 41 Imported by: 0

Documentation

Overview

Package configuration is the FTL configuration and secret management API.

The full design is documented here.

A Manager is the high-level interface to storing, listing, and retrieving secrets and configuration. A [Resolver] is the next layer, mapping names to a storage location key such as environment variables, keychain, etc. The Provider is the final layer, responsible for actually storing and retrieving values in concrete storage.

A constructed Manager and its providers are parametric on either secrets or configuration and thus cannot be used interchangeably.

Index

Constants

This section is empty.

Variables

View Source 
var ErrNotFound = errors.New("not found")

ErrNotFound is returned when a configuration entry is not found or cannot be resolved.

Functions

func ConfigFromEnvironment 

func ConfigFromEnvironment() []string

func ContextWithConfig 

func ContextWithConfig(ctx context.Context, configManager *Manager[Configuration]) context.Context

ContextWithConfig adds a configuration manager to the given context.

func ContextWithSecrets 

func ContextWithSecrets(ctx context.Context, secretsManager *Manager[Secrets]) context.Context

ContextWithSecrets adds a secrets manager to the given context.

func ProviderKeyForAccessor 

func ProviderKeyForAccessor(accessor *url.URL) string

Types

type ASM 

type ASM struct {
	// contains filtered or unexported fields
}

ASM implements a Provider for AWS Secrets Manager (ASM). Only supports loading "string" secrets, not binary secrets.

One controller is elected as the leader and is responsible for syncing the cache of secrets from ASM (see asmLeader). Others get secrets from the leader via AdminService (see asmFollower).

func NewASM 

func NewASM(ctx context.Context, secretsClient *secretsmanager.Client, advertise *url.URL, leaser leases.Leaser) *ASM

func (*ASM) Delete 

func (a *ASM) Delete(ctx context.Context, ref Ref) error

func (ASM) Key 

func (ASM) Key() string

func (ASM) Role 

func (ASM) Role() Secrets

func (*ASM) Store 

func (a *ASM) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

Store and if the secret already exists, update it.

func (*ASM) Sync 

func (a *ASM) Sync(ctx context.Context, entries []Entry, values *xsync.MapOf[Ref, SyncedValue]) error

func (*ASM) SyncInterval 

func (a *ASM) SyncInterval() time.Duration

type AsynchronousProvider 

type AsynchronousProvider[R Role] interface {
	Provider[R]

	SyncInterval() time.Duration

	// Sync is called periodically to update the cache with the latest values.
	//
	// SyncInterval() provides the expected time between syncs.
	// If Sync() returns an error, sync will be retried with an exponential backoff.
	//
	// Values should be updated by Sync().
	// An array of known entries from the router is provided in case it is helpful, but the provider can store any values it wants.
	Sync(ctx context.Context, entries []Entry, values *xsync.MapOf[Ref, SyncedValue]) error
}

AsynchronousProvider is an interface for providers that support syncing values. This is recommended if the provider allows batch access, or is expensive to load.

type Configuration 

type Configuration struct{}

func (Configuration) String 

func (Configuration) String() string

type DBConfigProvider 

type DBConfigProvider struct {
	// contains filtered or unexported fields
}

DBConfigProvider is a configuration provider that stores configuration in its key.

func NewDBConfigProvider 

func NewDBConfigProvider(dal DBConfigProviderDAL) DBConfigProvider

func (DBConfigProvider) Delete 

func (d DBConfigProvider) Delete(ctx context.Context, ref Ref) error

func (DBConfigProvider) Key 

func (DBConfigProvider) Key() string

func (DBConfigProvider) Load 

func (d DBConfigProvider) Load(ctx context.Context, ref Ref, key *url.URL) ([]byte, error)

func (DBConfigProvider) Role 

func (DBConfigProvider) Role() Configuration

func (DBConfigProvider) Store 

func (d DBConfigProvider) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

type DBConfigProviderDAL 

type DBConfigProviderDAL interface {
	GetModuleConfiguration(ctx context.Context, module optional.Option[string], name string) ([]byte, error)
	SetModuleConfiguration(ctx context.Context, module optional.Option[string], name string, value []byte) error
	UnsetModuleConfiguration(ctx context.Context, module optional.Option[string], name string) error
}

type DBConfigResolver 

type DBConfigResolver struct {
	// contains filtered or unexported fields
}

DBConfigResolver loads values a project's configuration from the given database.

func NewDBConfigResolver 

func NewDBConfigResolver(db DBConfigResolverDAL) DBConfigResolver

func (DBConfigResolver) Get 

func (d DBConfigResolver) Get(ctx context.Context, ref Ref) (*url.URL, error)

func (DBConfigResolver) List 

func (d DBConfigResolver) List(ctx context.Context) ([]Entry, error)

func (DBConfigResolver) Role 

func (d DBConfigResolver) Role() Configuration

func (DBConfigResolver) Set 

func (d DBConfigResolver) Set(ctx context.Context, ref Ref, key *url.URL) error

func (DBConfigResolver) Unset 

func (d DBConfigResolver) Unset(ctx context.Context, ref Ref) error

type DBConfigResolverDAL 

type DBConfigResolverDAL interface {
	ListModuleConfiguration(ctx context.Context) ([]sql.ModuleConfiguration, error)
}

type DBSecretResolver 

type DBSecretResolver struct {
	// contains filtered or unexported fields
}

DBSecretResolver loads values a project's secrets from the given database.

func NewDBSecretResolver 

func NewDBSecretResolver(db DBSecretResolverDAL) DBSecretResolver

func (DBSecretResolver) Get 

func (d DBSecretResolver) Get(ctx context.Context, ref Ref) (*url.URL, error)

func (DBSecretResolver) List 

func (d DBSecretResolver) List(ctx context.Context) ([]Entry, error)

func (DBSecretResolver) Role 

func (d DBSecretResolver) Role() Secrets

func (DBSecretResolver) Set 

func (d DBSecretResolver) Set(ctx context.Context, ref Ref, key *url.URL) error

func (DBSecretResolver) Unset 

func (d DBSecretResolver) Unset(ctx context.Context, ref Ref) error

type DBSecretResolverDAL 

type DBSecretResolverDAL interface {
	GetModuleSecretURL(ctx context.Context, module optional.Option[string], name string) (string, error)
	ListModuleSecrets(ctx context.Context) ([]dal.ModuleSecret, error)
	SetModuleSecretURL(ctx context.Context, module optional.Option[string], name string, url string) error
	UnsetModuleSecret(ctx context.Context, module optional.Option[string], name string) error
}

type Entry 

type Entry struct {
	Ref
	Accessor *url.URL
}

Entry in the configuration store.

type EnvarProvider 

type EnvarProvider[R Role] struct{}

EnvarProvider is a configuration provider that reads secrets or configuration from environment variables.

func (EnvarProvider[R]) Delete 

func (e EnvarProvider[R]) Delete(ctx context.Context, ref Ref) error

func (EnvarProvider[R]) Key 

func (EnvarProvider[R]) Key() string

func (EnvarProvider[R]) Load 

func (e EnvarProvider[R]) Load(ctx context.Context, ref Ref, key *url.URL) ([]byte, error)

func (EnvarProvider[R]) Role 

func (EnvarProvider[R]) Role() R

func (EnvarProvider[R]) Store 

func (e EnvarProvider[R]) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

type InlineProvider 

type InlineProvider[R Role] struct{}

InlineProvider is a configuration provider that stores configuration in its key.

func (InlineProvider[R]) Delete 

func (InlineProvider[R]) Delete(ctx context.Context, ref Ref) error

func (InlineProvider[R]) Key 

func (InlineProvider[R]) Key() string

func (InlineProvider[R]) Load 

func (InlineProvider[R]) Load(ctx context.Context, ref Ref, key *url.URL) ([]byte, error)

func (InlineProvider[R]) Role 

func (InlineProvider[R]) Role() R

func (InlineProvider[R]) Store 

func (InlineProvider[R]) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

type KeychainProvider 

type KeychainProvider struct{}

func (KeychainProvider) Delete 

func (k KeychainProvider) Delete(ctx context.Context, ref Ref) error

func (KeychainProvider) Key 

func (k KeychainProvider) Key() string

func (KeychainProvider) Load 

func (k KeychainProvider) Load(ctx context.Context, ref Ref, key *url.URL) ([]byte, error)

func (KeychainProvider) Role 

func (KeychainProvider) Role() Secrets

func (KeychainProvider) Store 

func (k KeychainProvider) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

type Manager 

type Manager[R Role] struct {
	// contains filtered or unexported fields
}

Manager is a high-level configuration manager that abstracts the details of the Router and Provider interfaces.

func ConfigFromContext 

func ConfigFromContext(ctx context.Context) *Manager[Configuration]

ConfigFromContext retrieves the configuration.Manager previously added to the context with ContextWithConfig.

func New 

func New[R Role](ctx context.Context, router Router[R], providers []Provider[R]) (*Manager[R], error)

New configuration manager.

func NewConfigurationManager 

func NewConfigurationManager(ctx context.Context, router Router[Configuration]) (*Manager[Configuration], error)

NewConfigurationManager creates a new configuration manager with the default configuration providers.

func NewDefaultConfigurationManagerFromConfig 

func NewDefaultConfigurationManagerFromConfig(ctx context.Context, config string) (*Manager[Configuration], error)

NewDefaultConfigurationManagerFromConfig creates a new configuration manager from the project config found in the config paths.

func NewDefaultSecretsManagerFromConfig 

func NewDefaultSecretsManagerFromConfig(ctx context.Context, config string, opVault string) (*Manager[Secrets], error)

NewDefaultSecretsManagerFromConfig creates a new secrets manager from the project config found in the config paths.

func NewSecretsManager 

func NewSecretsManager(ctx context.Context, router Router[Secrets], opVault string, config string) (*Manager[Secrets], error)

NewSecretsManager creates a new secrets manager with the default secret providers.

func SecretsFromContext 

func SecretsFromContext(ctx context.Context) *Manager[Secrets]

SecretsFromContext retrieves the secrets configuration.Manager previously added to the context with ContextWithConfig.

func (*Manager[R]) Get 

func (m *Manager[R]) Get(ctx context.Context, ref Ref, value any) error

Get a configuration value from the active providers.

"value" must be a pointer to a Go type that can be unmarshalled from JSON.

func (*Manager[R]) List 

func (m *Manager[R]) List(ctx context.Context) ([]Entry, error)

func (*Manager[R]) MapForModule 

func (m *Manager[R]) MapForModule(ctx context.Context, module string) (map[string][]byte, error)

MapForModule combines all configuration values visible to the module. Local values take precedence.

func (*Manager[R]) Set 

func (m *Manager[R]) Set(ctx context.Context, pkey string, ref Ref, value any) error

Set a configuration value, encoding "value" as JSON before storing it.

func (*Manager[R]) SetJSON 

func (m *Manager[R]) SetJSON(ctx context.Context, pkey string, ref Ref, value json.RawMessage) error

SetJSON sets a configuration value using raw JSON data.

func (*Manager[R]) Unset 

func (m *Manager[R]) Unset(ctx context.Context, pkey string, ref Ref) error

Unset a configuration value in all providers.

type ManualSyncProvider 

type ManualSyncProvider[R Role] struct {
	// contains filtered or unexported fields
}

ManualSyncProvider prevents normal syncs by returning a very high sync interval when syncAndWait() is called, it starts returning a 0 sync interval and then then blocks until sync completes. See why we didn't use mock clocks to schedule syncs here: https://github.com/TBD54566975/ftl/issues/2092

func NewManualSyncProvider 

func NewManualSyncProvider[R Role](provider AsynchronousProvider[R]) *ManualSyncProvider[R]

func (*ManualSyncProvider[R]) Delete 

func (a *ManualSyncProvider[R]) Delete(ctx context.Context, ref Ref) error

func (*ManualSyncProvider[R]) Key 

func (a *ManualSyncProvider[R]) Key() string

func (*ManualSyncProvider[R]) Role 

func (a *ManualSyncProvider[R]) Role() R

func (*ManualSyncProvider[R]) Store 

func (a *ManualSyncProvider[R]) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

func (*ManualSyncProvider[R]) Sync 

func (a *ManualSyncProvider[R]) Sync(ctx context.Context, entries []Entry, values *xsync.MapOf[Ref, SyncedValue]) error

func (*ManualSyncProvider[R]) SyncAndWait 

func (a *ManualSyncProvider[R]) SyncAndWait() error

func (*ManualSyncProvider[R]) SyncInterval 

func (a *ManualSyncProvider[R]) SyncInterval() time.Duration

type Obfuscator 

type Obfuscator struct {
	// contains filtered or unexported fields
}

Obfuscator hides and reveals a value, but does not provide real security instead the aim of this Obfuscator is to make values not easily human readable

Obfuscation is done by XOR-ing the input with the AES key. Length of key must be 16, 24 or 32 bytes (corresponding to AES-128, AES-192 or AES-256 keys).

func (Obfuscator) Obfuscate 

func (o Obfuscator) Obfuscate(input []byte) ([]byte, error)

Obfuscate takes a value and returns an obfuscated value (encoded in base64)

func (Obfuscator) Reveal 

func (o Obfuscator) Reveal(input []byte) ([]byte, error)

Reveal takes an obfuscated value and de-obfuscates the base64 encoded value

type ObfuscatorProvider 

type ObfuscatorProvider interface {
	// contains filtered or unexported methods
}

type OnePasswordProvider 

type OnePasswordProvider struct {
	Vault       string
	ProjectName string
}

OnePasswordProvider is a configuration provider that reads passwords from 1Password vaults via the "op" command line tool.

func (OnePasswordProvider) Delete 

func (o OnePasswordProvider) Delete(ctx context.Context, ref Ref) error

func (OnePasswordProvider) Key 

func (o OnePasswordProvider) Key() string

func (OnePasswordProvider) Role 

func (OnePasswordProvider) Role() Secrets

func (OnePasswordProvider) Store 

func (o OnePasswordProvider) Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)

Store will save the given secret in 1Password via the `op` command.

op does not support "create or update" as a single command. Neither does it support specifying an ID on create. Because of this, we need check if the item exists before creating it, and update it if it does.

func (OnePasswordProvider) Sync 

func (o OnePasswordProvider) Sync(ctx context.Context, entries []Entry, values *xsync.MapOf[Ref, SyncedValue]) error

Sync will fetch all secrets from the 1Password vault and store them in the values map. Do not just sync the o.Vault, instead find all vaults found in entries and sync them.

func (OnePasswordProvider) SyncInterval 

func (o OnePasswordProvider) SyncInterval() time.Duration

type ProjectConfigResolver 

type ProjectConfigResolver[R Role] struct {
	Config string `` /* 126-byte string literal not displayed */
}

ProjectConfigResolver is parametric Resolver that loads values from either a project's configuration or secrets maps based on the type parameter.

See the projectconfig package for details on the configuration file format.

func (ProjectConfigResolver[R]) Get 

func (p ProjectConfigResolver[R]) Get(ctx context.Context, ref Ref) (*url.URL, error)

func (ProjectConfigResolver[R]) List 

func (p ProjectConfigResolver[R]) List(ctx context.Context) ([]Entry, error)

func (ProjectConfigResolver[R]) Role 

func (p ProjectConfigResolver[R]) Role() R

func (ProjectConfigResolver[R]) Set 

func (p ProjectConfigResolver[R]) Set(ctx context.Context, ref Ref, key *url.URL) error

func (ProjectConfigResolver[From]) Unset 

func (p ProjectConfigResolver[From]) Unset(ctx context.Context, ref Ref) error

type Provider 

type Provider[R Role] interface {
	Role() R
	Key() string

	// Store a configuration value and return its key.
	Store(ctx context.Context, ref Ref, value []byte) (*url.URL, error)
	// Delete a configuration value.
	Delete(ctx context.Context, ref Ref) error
}

Provider is a generic interface for storing and retrieving configuration and secrets.

type Ref 

type Ref struct {
	Module optional.Option[string]
	Name   string
}

A Ref is a reference to a configuration value.

func NewRef 

func NewRef(module, name string) Ref

NewRef creates a new Ref.

If [module] is empty, the Ref is considered to be a global configuration value.

func ParseRef 

func ParseRef(s string) (Ref, error)

func (Ref) String 

func (k Ref) String() string

func (*Ref) UnmarshalText 

func (k *Ref) UnmarshalText(text []byte) error

type Role 

type Role interface {
	Secrets | Configuration
}

Role of Manager, either Secrets or Configuration.

type Router 

type Router[R Role] interface {
	Role() R
	Get(ctx context.Context, ref Ref) (key *url.URL, err error)
	Set(ctx context.Context, ref Ref, key *url.URL) error
	Unset(ctx context.Context, ref Ref) error
	List(ctx context.Context) ([]Entry, error)
}

A Router resolves configuration names to keys that are then used to load values from a Provider.

This indirection allows for the storage of configuration values to be abstracted from the configuration itself. For example, the ftl-project.toml file contains per-module and global configuration maps, but the secrets themselves may be stored in a separate secret store such as a system keychain.

type Secrets 

type Secrets struct{}

func (Secrets) String 

func (Secrets) String() string

type SyncedValue 

type SyncedValue struct {
	Value []byte

	// VersionToken is a way of storing a version provided by the source of truth (eg: lastModified)
	// it is nil when:
	// - the owner of the cache is not using version tokens
	// - the cache is updated after writing
	VersionToken optional.Option[VersionToken]
}

type SynchronousProvider 

type SynchronousProvider[R Role] interface {
	Provider[R]

	Load(ctx context.Context, ref Ref, key *url.URL) ([]byte, error)
}

SynchronousProvider is an interface for providers that can load values on-demand. This is recommended if the provider allows inexpensive loading of values.

type VersionToken 

type VersionToken any

Source Files

View all Source files

Directories

Path Synopsis
Package dal provides a data abstraction layer for managing module configurations
 Package dal provides a data abstraction layer for managing module configurations

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.