README
¶
Pulumi ESC (Environments, Secrets, and Configuration)
Pulumi ESC is a new product from Pulumi that manages and tames secrets and configuration complexity across all of your cloud infrastructure and application environments. Pulumi ESC introduces a new category of configuration-as-code product, motivated by our experience working with hundreds of Pulumi IaC customers to address their needs in managing secrets and configuration at scale within their Pulumi infrastructure and across other cloud applications and infrastructure projects.
Pulumi ESC enables teams to aggregate secrets and configuration from many sources, manage hierarchical collections of configuration and secrets ("environments"), and consume those configuration and secrets from a variety of different infrastructure and application services. Pulumi ESC works hand-in-hand with Pulumi IaC to simplify configuration management, but also works independently from Pulumi IaC, as a solution for managing environments, secrets and configuration for any application or infrastructure project.
For example, the Pulumi ESC CLI (esc) makes it possible to give your developers immediate, just-in-time authenticated and short-lived access to cloud credentials across any cloud provider with just a single command: esc run aws-staging -- aws s3 ls.
Pulumi ESC is offered as a managed service as part of Pulumi Cloud, and this repo contains the implementation of the following key components of the ESC offering:
- The
escCLI: A CLI tool for managing and consuming environments, secrets and configuration using Pulumi ESC. - The Pulumi ESC evaluator: The core specification and implementation of the document format for defining environments, and the syntax and semantics for evaluating environments to produce a set of configuration and secrets.
Resources
How Pulumi ESC works
- Pulumi ESC enables you to define environments, which are collections of secrets and configuration. Each environment can be composed from multiple environments.
- Pulumi ESC supports a variety of configuration and secrets sources, and it has an extensible plugin model that allows third-party sources.
- Pulumi ESC has a rich API that allows for easy integration. Every value in an environment can be accessed from any execution environment.
- Every environment can be locked down with RBAC, versioned, and audited.
Building the CLI Locally
You can build the CLI locally for testing by running:
$ make install
This will produce an esc binary in your GOBIN directory.
Why Pulumi ESC?
Pulumi ESC was designed to address a set of challenges that many infrastructure and application development teams face in managing configuration and secrets across their various environments:
- Secrets and Configuration Sprawl: Data in many systems. Challenging to audit. Lots of application-specific logic to acquire and compose configuration from multiple sources. Divergent solutions for Infrastructure and Application configuration.
- Duplication and Copy/Paste: Secrets are duplicated in many places. Frequently coupled to application/system-specific configuration stores.
- Too Many Long-lived Static Secrets: Long lived static credentials are over-used, exposing companies to significant security risk. Rotation is operationally challenging. Not all systems support direct integration with OIDC and other dynamic secrets provisioning systems.
Pulumi ESC was born to address these problems and needs head on. It does so through a few core design principles:
- Hierarchical and Composable: Environments contain collections of secrets and configuration, but can also import one or more other environments. Values can be overridden, interpolated from other values, and arbitrarily nested. This allows for flexible composition and reuse, and avoids copy paste.
- Any Secrets Provider: Support for dynamic configuration providers allow Pulumi ESC to integrate with secrets stored in any other provider. Organizations often use AWS Secrets Manager, Vault, Azure OIDC and/or 1Password plus many more sources of truth for their secrets and configuration. Pulumi ESC supports them all, providing a single interface to your configuration and secrets, no matter where their source of truth is. Pulumi ESC works with these tools to provide improved management of secrets and configuration.
- Consume from Anywhere: The
escCLI and the Pulumi ESC Rest API enables environments to be accessed from any application, infrastructure provider, automation system. At launch, first-class integrations are available with Pulumi IaC, local environment and .env files, GitHub Actions, and more. - Auditable: Environments must be “opened” to compute and see the set of value they provide, and this action is recorded in audit logs, including a full record of how each value was sourced from within the hierarchy of environments that contributed to it.
- Authentication and RBAC: Pulumi ESC brokers access to secrets and configuration that live in other systems, and so authentication and granular RBAC are critical to ensure robust access controls across your organization. Pulumi ESC leverages the same Pulumi Cloud identity, RBAC, Teams, SAML/SCIM and scoped access tokens that are used for Pulumi IaC today, extending these all to managing access to Environments as well as Stacks.
- Configuration as Code: Environments are defined as YAML documents which can describe how to project and compose secrets and configuration, integrate dynamic configuration providers, and compute new configuration from other values (construing a URL from a DNS name, or concatenating multiple configuration values into a derived value). The incredible flexibility of a code-based approach over traditional point-and-click interfaces allows Pulumi ESC to offer rich expressiveness for managing complex configuration.
- Fully Managed: Pulumi ESC is offered as a fully managed cloud service in Pulumi Cloud (and Pulumi Cloud Self-hosted in the near future). The pulumi/esc project is open source, and contains the evaluation engine for environments, the esc CLI, and in the future, the extensible plugins for source and target integrations.
Documentation
¶
Index ¶
Constants ¶
const AnonymousEnvironmentName = "<yaml>"
Variables ¶
var ErrReservedContextkey = errors.New("reserved context key")
Functions ¶
This section is empty.
Types ¶
type AccessExpr ¶
type AccessExpr struct {
// The receiver to access.
Receiver Range `json:"receiver"`
// The accessors to evaluate.
Accessors []Accessor `json:"accessors"`
}
An AccessExpr represents a property access with a receiving value.
type Accessor ¶
type Accessor struct {
// The integer index of the element to access. Mutually exclusive with Key.
Index *int `json:"index,omitempty"`
// The key of the property to access. Mutually exclusive with Index.
Key *string `json:"key,omitempty"`
// The range of the accessor.
Range Range `json:"range,omitempty"`
}
An Accessor is an element index or property name.
type BuiltinExpr ¶
type BuiltinExpr struct {
Name string `json:"name"`
NameRange Range `json:"nameRange"`
ArgSchema *schema.Schema `json:"argSchema"`
Arg Expr `json:"arg"`
}
A BuiltinExpr is a call to a builtin function.
type EnvExecContext ¶ added in v0.8.3
type EnvExecContext interface {
// Returns the current execution context values
Values() map[string]Value
// Returns the root evaluated environment.
// For anonymous environments, it resolves to the "rootest" non anonymous environment.
GetRootEnvironmentName() string
// Returns the current environment being evaluated.
GetCurrentEnvironmentName() string
}
type Environment ¶
type Environment struct {
// Exprs contains the AST for each expression in the environment definition.
Exprs map[string]Expr `json:"exprs,omitempty"`
// Properties contains the detailed values produced by the environment.
Properties map[string]Value `json:"properties,omitempty"`
// Schema contains the schema for Properties.
Schema *schema.Schema `json:"schema,omitempty"`
// ExecutionContext contains the values + schema for the execution context passed to the root environment.
ExecutionContext *EvaluatedExecutionContext `json:"executionContext,omitempty"`
}
An Environment contains the result of evaluating an environment definition.
func (*Environment) GetEnvironmentVariables ¶ added in v0.5.7
func (e *Environment) GetEnvironmentVariables() map[string]Value
GetEnvironmentVariables returns any environment variables defined by the environment.
Environment variables are any scalar values in the top-level `environmentVariables` property. Boolean and number values are converted to their string representation. The results remain Values in order to retain secret- and unknown-ness.
func (*Environment) GetTemporaryFiles ¶ added in v0.5.7
func (e *Environment) GetTemporaryFiles() map[string]Value
GetTemporaryFiles returns any temporary files defined by the environment.
Temporary files are any string values in the top-level `files` property. The key for each file is the name of the environment variable that should hold the actual path to the temporary file once it is written.
type EvaluatedExecutionContext ¶ added in v0.8.1
type ExecContext ¶ added in v0.8.1
type ExecContext struct {
// contains filtered or unexported fields
}
func NewExecContext ¶ added in v0.8.1
func NewExecContext(values map[string]Value) (*ExecContext, error)
func (*ExecContext) CopyForEnv ¶ added in v0.8.1
func (ec *ExecContext) CopyForEnv(envName string) *ExecContext
func (*ExecContext) GetCurrentEnvironmentName ¶ added in v0.8.3
func (ec *ExecContext) GetCurrentEnvironmentName() string
func (*ExecContext) GetRootEnvironmentName ¶ added in v0.8.3
func (ec *ExecContext) GetRootEnvironmentName() string
func (*ExecContext) Values ¶ added in v0.8.1
func (ec *ExecContext) Values() map[string]Value
type Expr ¶
type Expr struct {
// The range of the expression.
Range Range `json:"range"`
// The schema of the expression's result.
Schema *schema.Schema `json:"schema,omitempty"`
// The expression that defined this expression's base value, if any.
Base *Expr `json:"base,omitempty"`
// Ranges for the object's keys, if this is an object expression.
KeyRanges map[string]Range `json:"keyRanges,omitempty"`
// The literal value, if this is a literal expression (nil, bool, json.Number, or string)
Literal any `json:"literal,omitempty"`
// The interpolations, if this is a string interpolation expression.
Interpolate []Interpolation `json:"interpolate,omitempty"`
// The property accessors, if this is a symbol expression.
Symbol []PropertyAccessor `json:"symbol,omitempty"`
// The access, if this is an access expression.
Access *AccessExpr `json:"access,omitempty"`
// The list elements, if this is a list expression.
List []Expr `json:"list,omitempty"`
// The object properties, if this is an object expression.
Object map[string]Expr `json:"object,omitempty"`
// The builtin, if this is a call to a builtin function.
Builtin *BuiltinExpr `json:"builtin,omitempty"`
}
An Expr holds information about an expression in an environment definition.
type Interpolation ¶
type Interpolation struct {
// The text of the expression. Precedes the stringified Value in the output.
Text string `json:"text,omitempty"`
// The value to interpolate.
Value []PropertyAccessor `json:"value,omitempty"`
}
An Interpolation holds information about a part of an interpolated string expression.
type Pos ¶
type Pos struct {
// Line is the source code line where this position points. Lines are counted starting at 1 and incremented for each
// newline character encountered.
Line int `json:"line"`
// Column is the source code column where this position points. Columns are counted in visual cells starting at 1,
// and are incremented roughly per grapheme cluster encountered.
Column int `json:"column"`
// Byte is the byte offset into the file where the indicated position begins.
Byte int `json:"byte"`
}
A Pos defines a position within an environment definition.
type PropertyAccessor ¶
type PropertyAccessor struct {
Accessor
// The range of the expression that defines the resolved value.
Value Range `json:"value"`
}
A PropertyAccessor is a single accessor that is associated with a resolved value.
type Provider ¶
type Provider interface {
// Schema returns the provider's input and output schemata.
Schema() (inputs, outputs *schema.Schema)
// Open retrieves the provider's secrets.
Open(ctx context.Context, inputs map[string]Value, executionContext EnvExecContext) (Value, error)
}
A Provider provides environments access to dynamic secrets. These secrets may be generated at runtime, fetched from other services, etc.
type Range ¶
type Range struct {
// The name of the environment.
Environment string `json:"environment,omitempty"`
// The beginning of the range.
Begin Pos `json:"begin"`
// The end of the range.
End Pos `json:"end"`
}
A Range defines a range within an environment definition.
type Trace ¶
type Trace struct {
// Def is the range of the expression that computed a value.
Def Range `json:"def"`
// Base is the base value with which a value was merged.
Base *Value `json:"base,omitempty"`
}
Trace holds information about the expression and base of a value.
type Value ¶
type Value struct {
// Value holds the concrete representation of the value. May be nil, bool, json.Number, string, []Value, or
// map[string]Value.
Value any `json:"value,omitempty"`
// Secret is true if this value is secret.
Secret bool `json:"secret,omitempty"`
// Unknown is true if this value is unknown.
Unknown bool `json:"unknown,omitempty"`
// Trace holds information about the expression that computed this value and the value (if any) with which it was
// merged.
Trace Trace `json:"trace"`
}
A Value is the result of evaluating an expression within an environment definition.
func FromJSON ¶ added in v0.5.5
FromJSON converts a plain-old-JSON value (i.e. a value of type nil, bool, json.Number, string, []any, or map[string]any) into a Value.
func (Value) ToJSON ¶
ToJSON converts a Value into a plain-old-JSON value (i.e. a value of type nil, bool, json.Number, string, []any, or map[string]any). If redact is true, secrets are replaced with [secret].
func (Value) ToString ¶
ToString returns the string representation of this value. If redact is true, secrets are replaced with [secret].
func (*Value) UnmarshalJSON ¶
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
esc/cli/client
Package client implements a client for the Pulumi Service HTTP/REST API.
|
Package client implements a client for the Pulumi Service HTTP/REST API. |
|
internal
|
|