pvc

README

PVC

PVC (polyvinyl chloride) is a simple, generic secret retrieval library that supports multiple backends.

PVC lets applications access secrets without caring too much about where they happen to be stored. The use case is to allow secrets to come from local/insecure backends during development and testing, and then from Vault in production without significant code changes required.

Backends

• Vault KV Version 1
• Environment variables
• JSON file
• File Tree (local filesystem, one file per secret)

Secret Values

PVC makes some assumptions about how your secrets are stored in the various backends:

• If using Vault, there must be exactly one key called "value" for any given secret path (this can be overridden with WithVaultValueKey("foo")). The data associated with the value key will be retrieved and returned literally to the client as a byte slice. Binary values must be Base64-encoded.
• If using JSON or environment variables, the value will be treated as a string and returned as a byte slice. Binary values should be Base64-encoded (same as Vault).
• If using the file tree backend, you must supply an absolute root path which will be combined with the secret ID (after mapping). This file path will be read as the secret contents.

File Tree

This is intended to be useful for local development secrets in the filesystem, or using the Vault Sidecar Injector.

Example:

Given the following:

-> Root Path: /vault/secrets
-> Secret ID: webservice/production/db/password
-> Mapping: {{ .ID }}.txt

PVC would attempt to read this file when Get() is called:

/vault/secrets/webservice/production/db/password.txt

Vault Authentication

The Vault backend supports token, Kubernetes, and AppRole authentication.

Example

package main

import (
	"fmt"

	"github.com/dollarshaveclub/pvc"
)

func main() {

	// environment variable backend
	sc, _ := pvc.NewSecretsClient(pvc.WithEnvVarBackend(), pvc.WithMapping("SECRET_MYAPP_{{ .ID }}"))
	secret, _ := sc.Get("foo") // fetches the env var "SECRET_MYAPP_FOO"

	// JSON file backend
	sc, _ = pvc.NewSecretsClient(pvc.WithJSONFileBackend("secrets.json"))
	secret, _ = sc.Get("foo") // fetches the value in secrets.json under the key "foo"

	fmt.Printf("foo: %v\n", string(secret))

	// Vault backend
	sc, _ = pvc.NewSecretsClient(
		pvc.WithVaultBackend(pvc.TokenVaultAuth, "http://vault.example.com:8200"),
		pvc.WithVaultToken("some token"),
		pvc.WithMapping("secret/development/{{ .ID }}"))
	secret, _ = sc.Get("foo") // fetches the value from Vault (using token auth) from path secret/development/foo

	fmt.Printf("foo: %v\n", string(secret))

	// Automatic struct filling
	type Secrets struct {
		Username      string `secret:"secret/username"` // secret id: secret/username
		Password      string `secret:"secret/password"`
		EncryptionKey []byte `secret:"secret/enc_key"` // fields can be strings or byte slices
	}

	secrets := Secrets{}

	// Fill automatically fills the fields in the secrets struct that have "secret" tags
	err := sc.Fill(&secrets)
	if err != nil {
		panic(err)
	}

	fmt.Printf("my username is: %v\n", secrets.Username)
	fmt.Printf("my password is: %v\n", secrets.Password)
	fmt.Printf("my key length is %d\n", len(secrets.EncryptionKey))
}

See also example/

Documentation

Index

• Constants
• Variables
• type SecretDefinition
• type SecretMapper
• type SecretsClient
  • func NewSecretsClient(ops ...SecretsClientOption) (*SecretsClient, error)
  • func (sc *SecretsClient) Fill(s interface{}) error
  • func (sc *SecretsClient) Get(id string) ([]byte, error)
• type SecretsClientOption
  • func WithEnvVarBackend() SecretsClientOption
  • func WithFileTreeBackend(rootPath string) SecretsClientOption
  • func WithJSONFileBackend(path string) SecretsClientOption
  • func WithMapping(mapping string) SecretsClientOption
  • func WithVaultAuthRetries(retries uint) SecretsClientOption
  • func WithVaultAuthRetryDelay(secs uint) SecretsClientOption
  • func WithVaultBackend(auth VaultAuthentication, host string) SecretsClientOption
  • func WithVaultK8sAuth(jwt, role string) SecretsClientOption
  • func WithVaultK8sAuthPath(path string) SecretsClientOption
  • func WithVaultRoleID(roleid string) SecretsClientOption
  • func WithVaultToken(token string) SecretsClientOption
  • func WithVaultValueKey(key string) SecretsClientOption
• type VaultAuthentication

Constants

const (
	DefaultFileTreeMapping  = "{{ .ID }}"
	DefaultFileTreeRootPath = "/vault/secrets"
)

Default mapping for this backend

const (
	DefaultEnvVarMapping = "SECRET_{{ .ID }}" // DefaultEnvVarMapping is uppercased after interpolation for convenience
)

Default mapping for this backend

const (
	DefaultJSONFileMapping = "{{ .ID }}"
)

Default mapping for this backend

const (
	DefaultVaultMapping = "secret/{{ .ID }}"
)

Default mapping for this backend

Variables

var DefaultVaultValueKey = "value"

var MaxFileTreeFileSizeBytes int64 = 2_000_000 // 2 MB

MaxFileTreeSizeBytes indicates the maximum file size we will read

var SecretStructTag = "secret"

SecretStructTag is the default struct tag name

Types

type SecretDefinition

type SecretDefinition struct {
	ID         string // arbitrary identifier for this secret
	VaultPath  string // path in Vault (no leading slash, eg "secret/foo/bar")
	EnvVarName string // environment variable name
	JSONKey    string // key in JSON object
}

SecretDefinition defines a secret and how it can be accessed via the various backends

type SecretMapper

type SecretMapper interface {
	MapSecret(id string) (string, error)
}

SecretMapper maps secrets

type SecretsClient

type SecretsClient struct {
	// contains filtered or unexported fields
}

SecretsClient is the client that retrieves secret values

func NewSecretsClient

func NewSecretsClient(ops ...SecretsClientOption) (*SecretsClient, error)

NewSecretsClient returns a SecretsClient configured according to the SecretsClientOptions supplied. Exactly one backend must be enabled. Weird things will happen if you mix options with incompatible backends.

func (*SecretsClient) Fill

func (sc *SecretsClient) Fill(s interface{}) error

Fill takes a pointer to any struct type and fills any fields annotated with SecretStructTag secret ids. Annotated fields must *only* be string or []byte, any other type will cause this method to return an error. Note that Fill doesn't check the secret type; if the field value is a string, the byte slice returned by the backend for that secret will be converted to a string.

func (*SecretsClient) Get

func (sc *SecretsClient) Get(id string) ([]byte, error)

Get returns the value of a secret from the configured backend

type SecretsClientOption

type SecretsClientOption func(*secretsClientConfig)

SecretsClientOption defines options when creating a SecretsClient

func WithEnvVarBackend

func WithEnvVarBackend() SecretsClientOption

WithEnvVarBackend enables the environment variable backend. Any characters in the secret ID that are not alphanumeric ASCII or underscores (legal env var characters) will be replaced by underscores after mapping.

func WithFileTreeBackend

func WithFileTreeBackend(rootPath string) SecretsClientOption

WithFileTree enables the FileTreeBackend. With this backend, PVC reads one individual file per secret ID. Sub-paths under the root should be implemented with directory separators in the secret ID. The path that results from the root path + secret ID mapping will be read as the secret. This must be an absolute filesystem path.

func WithJSONFileBackend

func WithJSONFileBackend(path string) SecretsClientOption

WithJSONFileBackend enables the JSON file backend. The file should contain a single JSON object associating a name with a value: { "mysecret": "pa55w0rd"}. Path is required and must be a valid path to the JSON file.

func WithMapping

func WithMapping(mapping string) SecretsClientOption

WithMapping sets the template string mapping to determine the location for each secret in the backend. The secret ID will be interpolated as ".ID". Example (Vault Backend): "secret/foo/bar/{{ .ID }}". Example (Env Var Backend): "MYAPP_SECRET_{{ .ID }}" Example (JSON Backend): "{{ .ID }}"

func WithVaultAuthRetries

func WithVaultAuthRetries(retries uint) SecretsClientOption

WithVaultAuthRetries sets the number of retries if authentication fails (default: 0)

func WithVaultAuthRetryDelay

func WithVaultAuthRetryDelay(secs uint) SecretsClientOption

WithVaultAuthRetryDelay sets the delay in seconds between authentication attempts (default: 0)

func WithVaultBackend

func WithVaultBackend(auth VaultAuthentication, host string) SecretsClientOption

WithVaultBackend enables the Vault backend with the requested authentication type and host (ex: https//my.vault.com:8300)

func WithVaultK8sAuth

func WithVaultK8sAuth(jwt, role string) SecretsClientOption

WithVaultK8sAuth sets the Kubernetes JWT and role to use for authentication

func WithVaultK8sAuthPath

func WithVaultK8sAuthPath(path string) SecretsClientOption

WithVaultK8sAuthPath sets the path for the k8s Vault auth backend (defaults to "kubernetes" otherwise)

func WithVaultRoleID

func WithVaultRoleID(roleid string) SecretsClientOption

WithVaultRoleID sets the RoleID when using AppRole authentication

func WithVaultToken

func WithVaultToken(token string) SecretsClientOption

WithVaultToken sets the token to use when using token auth

func WithVaultValueKey

func WithVaultValueKey(key string) SecretsClientOption

type VaultAuthentication

type VaultAuthentication int

VaultAuthentication enumerates the supported Vault authentication methods

const (
	UnknownVaultAuth VaultAuthentication = iota // Unknown/unset
	TokenVaultAuth                              // Token authentication
	AppRoleVaultAuth                            // AppRole
	K8sVaultAuth                                // Kubernetes
)

Supported Vault authentication methods