starenv

README

Starenv

starenv (*env) allows populating environmental variables from variety of sources, such as AWS Parameter Store, GPG encrypted files and more, with extreme ease.

For the impatient, import the autoload package and you're set:

package main

import (
  "fmt"
  "os"

  _ "github.com/oxplot/starenv/autoload"
)

func main() {
  fmt.Printf("GITHUB_TOKEN=%s\n", os.Getenv("GITHUB_TOKEN"))
}

and set the value of the environmental variable to load from Parameter Store:

$ export GITHUB_TOKEN=*ssm:/github_token
$ go run main.go
GITHUB_TOKEN=abcdef-1235143-abcdef-123-abcdef-12314

or from a GPG encrypted file:

$ export GITHUB_TOKEN=*gpg*file:github_token.gpg
$ go run main.go
GITHUB_TOKEN=abcdef-1235143-abcdef-123-abcdef-12314

why not ditch the file and embed its content:

$ export GITHUB_TOKEN=*gpg*b64:eNeO7D2rBrBOOcW6TuETyHdyPEOaAfdgaTzgOTSvROI=
$ go run main.go
GITHUB_TOKEN=abcdef-1235143-abcdef-123-abcdef-12314

and thanks to the amazing godotenv which is run as part of starenv's autoload package, you can even do:

$ echo 'GITHUB_TOKEN=*keyring:awesome_app/github_token' > .env
$ go run main.go
GITHUB_TOKEN=abcdef-1235143-abcdef-123-abcdef-12314

For a full list, see the docs.

Documentation

Overview

Package starenv implements populating environmental variables from variety of sources.

Usage

Simplest way to use starenv is to import the autoload package:

import _ "github.com/oxplot/starenv/autoload"

The above will iterate through all environmental variables looking for specially formatted values which tell it where to load the final values from. After the above is imported, you can use the usual os.Getenv() to get the value of environmental variables.

Ref Pipline

The source of a value is defined as a pipeline of "derefer" tags followed by the reference for the last derefer. Here is an example of a environmental variable specifying to load its value from a base64 encoded file and decrypt it using GPG:

GITHUB_TOKEN=*gpg*b64*file:~/.github_token

Each derefer is applied in reverse, starting with "file" which loads the content of "~/.github_token". "b64" then decodes it and finally "gpg" decrypts it.

If the value of an environmental variable starts with Loader.Star (which defaults to "*"), it is treated as a pipeline. Otherwise, it's treated as a literal value and left unchagned. In the unlikely case where a literal value starting with Loader.Star is needed, the following can be used:

GLOB_PAT=*:*.terraform

Here the blank derefer treats everything after "*:" as literal and returns it, thus leading to GLOB_PAT being set to "*.terraform".

Package autoload registers a set of derefers that are included in derefer package with appropriate tags. To have more control over tags and the timing of when the loading happens, you can register each derefer manually and call Load() to populate the env vars. When using the autoload package, .env file is also read and loaded. To disable this, set DOTENV_ENABLED=0.

Any type that implements Derefer methods can be registered and used in the pipeline.

Index

• Variables
• func Base64(ref string) (string, error)
• func Bzip2(ref string) (string, error)
• func Env(ref string) (string, error)
• func Flate(ref string) (string, error)
• func GPG(ref string) (string, error)
• func Gzip(ref string) (string, error)
• func Hex(ref string) (string, error)
• func Keyring(ref string) (string, error)
• func TempFile(v string) (string, error)
• type AWSParameterStore
  • func NewAWSParameterStore() (*AWSParameterStore, error)
  • func NewAWSParameterStoreWithConfig(c aws.Config) *AWSParameterStore
  • func (d *AWSParameterStore) Deref(ref string) (string, error)
• type Derefer
• type DereferFunc
  • func (d DereferFunc) Deref(ref string) (string, error)
• type File
  • func (f *File) Deref(ref string) (string, error)
• type HTTP
  • func (h *HTTP) Deref(ref string) (string, error)
• type LazyDerefer
  • func (l *LazyDerefer) Deref(ref string) (string, error)
• type Loader
  • func NewLoader() *Loader
  • func (l *Loader) Load() []error
  • func (l *Loader) Register(tag string, d Derefer)
• type S3
  • func NewS3() (*S3, error)
  • func NewS3WithConfig(c aws.Config) *S3
  • func (d *S3) Deref(ref string) (string, error)

Variables

var DefaultDerefers = map[string]func() (Derefer, error){
	"env": func() (Derefer, error) {
		return DereferFunc(Env), nil
	},
	"b64": func() (Derefer, error) {
		return DereferFunc(Base64), nil
	},
	"hex": func() (Derefer, error) {
		return DereferFunc(Hex), nil
	},
	"ssm": func() (Derefer, error) {
		return NewAWSParameterStore()
	},
	"pssm": func() (Derefer, error) {
		d, err := NewAWSParameterStore()
		if err != nil {
			return nil, err
		}
		d.Plaintext = true
		return d, nil
	},
	"s3": func() (Derefer, error) {
		return NewS3()
	},
	"file": func() (Derefer, error) {
		return &File{ExpandHome: true}, nil
	},
	"gpg": func() (Derefer, error) {
		return DereferFunc(GPG), nil
	},
	"keyring": func() (Derefer, error) {
		return DereferFunc(Keyring), nil
	},
	"https": func() (Derefer, error) {
		return &HTTP{}, nil
	},
	"http": func() (Derefer, error) {
		return &HTTP{DefaultInsecure: true}, nil
	},
	"tmpfile": func() (Derefer, error) {
		return DereferFunc(TempFile), nil
	},
	"gz": func() (Derefer, error) {
		return DereferFunc(Gzip), nil
	},
	"bz2": func() (Derefer, error) {
		return DereferFunc(Bzip2), nil
	},
	"flate": func() (Derefer, error) {
		return DereferFunc(Flate), nil
	},
}

DefaultDerefers is a mapping of default tags to derefer creator functions that use sensible default config. When using autoload package, these derefers are automatically loaded with default tags.

var (
	// DefaultLoader is the default loader which has all the DefaultDerefers
	// registered with it.
	DefaultLoader = NewLoader()
)

Functions

func Base64

func Base64(ref string) (string, error)

Base64 decodes base64 encoded ref and returns it. Default tag for this derefer is "b64".

func Bzip2

func Bzip2(ref string) (string, error)

Bzip2 decompresses bzipped2 compressed ref and returns it. Default tag for this derefer is "bz2".

func Env

func Env(ref string) (string, error)

Env returns value of environmental variable with given name or empty string if not set.

func Flate

func Flate(ref string) (string, error)

Flate decompresses flate compressed ref and returns it. Default tag for this derefer is "flate".

func GPG

func GPG(ref string) (string, error)

GPG takes encrypted content ref, decrypts it and returns it. It calls on external gpg command to do this. Default tag for this derefer is "gpg".

func Gzip

func Gzip(ref string) (string, error)

Gzip decompresses gzipped compressed ref and returns it. Default tag for this derefer is "gz".

func Hex

func Hex(ref string) (string, error)

Hex decodes hex encoded ref and returns it. Default tag for this derefer is "hex".

func Keyring

func Keyring(ref string) (string, error)

Keyring uses system provided secret storage to retrive the secret stored for ref and returns it. ref must be formatted as "service/secret" where "service" is a grouping under which the secret "secret" is stored. The exact definition of "service" depends on the operating system. Refer to github.com/zalando/go-keyring module for more info.

On linux, you can store a secret via GNOME's SecretService using the command line:

secret-tool store --label my_app_secret service my_app username user123

You can then pass "my_app/user123" as ref to Keyring() method to retrieve it.

Default tag for this derefer is "keyring".

func TempFile

func TempFile(v string) (string, error)

TempFile creates a temporary file and stores the content of ref in it and returns its path. This is useful for storing secrets in files without having their content in the env var. Under most OSes, temp directory content is held in memory and isn't written to disk, this ensures a further layer of security for secrets. Note that the file is NOT deleted automatically.

Types

type AWSParameterStore

type AWSParameterStore struct {

	// By default, the value is decrypted. Set this to true to retrieve
	// non-encrypted values.
	Plaintext bool
	// contains filtered or unexported fields
}

AWSParameterStore derefs a Parameter Store ARN to its value. Default tag for this derefer is "ssm". "pssm" tag is set to Plaintext mode of this derefer.

func NewAWSParameterStore

func NewAWSParameterStore() (*AWSParameterStore, error)

NewAWSParameterStore creates a new AWSParameterStore derefer with default configuration.

func NewAWSParameterStoreWithConfig

func NewAWSParameterStoreWithConfig(c aws.Config) *AWSParameterStore

NewAWSParameterStoreWithConfig creates a new AWSParameterStore derefer with the given configuration.

func (*AWSParameterStore) Deref

func (d *AWSParameterStore) Deref(ref string) (string, error)

Deref returns the value of the parameter ref which is the ARN of the resource in the form (where [..] is optional):

[arn:aws:ssm:<region>:<account-number>:parameter]/...

type Derefer

type Derefer interface {
	Deref(ref string) (value string, err error)
}

Derefer is an interface that wraps Deref method.

Deref method is called with the recursively derefed value of all subsequent derefers in the pipeline. ref is therefore a literal by the time it's passed to this method.

type DereferFunc

type DereferFunc func(ref string) (string, error)

DereferFunc type is an adapter to allow use of ordinary functions as derefers.

func (DereferFunc) Deref

func (d DereferFunc) Deref(ref string) (string, error)

Deref calls f(ref).

type File

type File struct {
	// ExpandHome when set to true will replace "~/" with the current user's home
	// directory just like a standard POSIX shell would do.
	ExpandHome bool
}

File derefs a file path to the content of the file. Default tag for this derefer is "file".

func (*File) Deref

func (f *File) Deref(ref string) (string, error)

Deref treats ref as path to a file and returns the content of the file.

type HTTP

type HTTP struct {
	// By default, if no scheme is provided, https is assumed. Setting this to
	// true will instead assume http.
	DefaultInsecure bool
}

HTTP derefs a URL to its content. Default tag for this derefer is "https". Tag "http" exists for the insecure config.

func (*HTTP) Deref

func (h *HTTP) Deref(ref string) (string, error)

Deref treats ref as a URL and returns the response body of a GET request. You can leave off the scheme and take advantange of the default tag for this derefer:

*https://example.com

instead of:

*https:https://example.com

type LazyDerefer

type LazyDerefer struct {
	New func() (Derefer, error)
	// contains filtered or unexported fields
}

LazyDerefer is a derefer that encapsulates a derefer creator function and delays its call until the first deref call.

func (*LazyDerefer) Deref

func (l *LazyDerefer) Deref(ref string) (string, error)

Deref calls the underlying derefer's Deref method.

type Loader

type Loader struct {
	// Star is the prefix and separator of derefer tags which defaults to "*".
	Star string
	// contains filtered or unexported fields
}

Loader holds a registry of derefers which are looked up and applied to values of all environmental variables when Load() is called.

func NewLoader

func NewLoader() *Loader

NewLoader returns a new loader with empty "" tag mapped to to a passthrough derefer. This is needed to allow for environmental variable values which start with "*":

GLOB_PAT=*:*.terraform

The above will resolve to "*.terraform".

func (*Loader) Load

func (l *Loader) Load() []error

Load iterates through all environmental variables and recursively derefs their values appropriately. If STARENV_ENABLED env var is set to 0, it does nothing. Error in loading one variable does not stop the process. All errors are collected and returned as a slice. The errored out variables are set to empty string.

func (*Loader) Register

func (l *Loader) Register(tag string, d Derefer)

Register maps a tag with a derefer. Tags that include ":" are unusable.

type S3

type S3 struct {
	// contains filtered or unexported fields
}

S3 derefs an S3 bucket and object path to its content. Default tag for this derefer is "s3".

func NewS3

func NewS3() (*S3, error)

NewS3 creates a new S3 derefer with default configuration.

func NewS3WithConfig

func NewS3WithConfig(c aws.Config) *S3

NewS3WithConfig creates a new S3 derefer with the given configuration.

func (*S3) Deref

func (d *S3) Deref(ref string) (string, error)

Deref returns the content of the S3 object given the bucket and path, and optionally the object version. If version is omitted, latest version of the object is used.

bucket/path/to/object[@version]