storageconsul

README

Caddy 2 cluster / Certmagic TLS cluster support for Consul K/V

Consul K/V Storage for Caddy TLS data.

This cluster plugin enables Caddy 2 to store TLS data like keys and certificates in Consul's K/V store so you don't have to rely on a shared filesystem. This allows you to use Caddy 2 in distributed environment and use a centralized storage for auto-generated certificates that is shared between all Caddy instances.

With this plugin it is possible to use multiple Caddy instances with the same HTTPS domain for instance with DNS round-robin. All data that is saved in the KV store is encrypted using AES.

The version of this plugin in the master branch supports Caddy 2.0.0+ using CertMagic's Storage Interface

Older versions

• For Caddy 0.10.x to 0.11.1 : use the old_storage_interface branch.
• For Caddy 1.x : use the caddy1 branch.

Configuration

Caddy configuration

ATTENTION: The name of the storage module in configurations has been changed to consul to align with other storage modules.

You need to specify consul as the storage module in Caddy's configuration. This can be done in the config file of using the admin API.

JSON (reference)

{
  "admin": {
    "listen": "0.0.0.0:2019"
  },
  "storage": {
    "module": "consul",
    "address": "localhost:8500",
    "prefix": "caddytls",
    "token": "consul-access-token",
    "aes_key": "consultls-1234567890-caddytls-32"
  }
}

Caddyfile (reference)

{
    storage consul {
           address      "127.0.0.1:8500"
           token        "consul-access-token"
           timeout      10
           prefix       "caddytls"
           value_prefix "myprefix"
           aes_key      "consultls-1234567890-caddytls-32"
           tls_enabled  "false"
           tls_insecure "true"
    }
}

:443 {
}

Consul configuration

Because this plugin uses the official Consul API client you can use all ENV variables like CONSUL_HTTP_ADDR or CONSUL_HTTP_TOKEN to define your Consul address and token. For more information see https://github.com/hashicorp/consul/blob/master/api/api.go

Without any further configuration a running Consul on 127.0.0.1:8500 is assumed.

There are additional ENV variables for this plugin:

• CADDY_CLUSTERING_CONSUL_AESKEY defines your personal AES key to use when encrypting data. It needs to be 32 characters long.
• CADDY_CLUSTERING_CONSUL_PREFIX defines the prefix for the keys in KV store. Default is caddytls

Consul ACL Policy

To access Consul you need a token with a valid ACL policy. Assuming you configured cadytls as your K/V path prefix you can use the following settings:

key_prefix "caddytls" {
	policy = "write"
}
session_prefix "" {
	policy = "write"
}
node_prefix "" {
	policy = "read"
}
agent_prefix "" {
	policy = "read"
}

Documentation

Index

• Constants
• func ConsulQueryDefaults(ctx context.Context) *consul.QueryOptions
• type ConsulStorage
  • func New() *ConsulStorage
  • func (ConsulStorage) CaddyModule() caddy.ModuleInfo
  • func (cs *ConsulStorage) CertMagicStorage() (certmagic.Storage, error)
  • func (cs *ConsulStorage) DecryptStorageData(bytes []byte) (*StorageData, error)
  • func (cs ConsulStorage) Delete(ctx context.Context, key string) error
  • func (cs *ConsulStorage) EncryptStorageData(data *StorageData) ([]byte, error)
  • func (cs ConsulStorage) Exists(ctx context.Context, key string) bool
  • func (cs *ConsulStorage) GetLock(key string) (*consul.Lock, bool)
  • func (cs ConsulStorage) List(ctx context.Context, prefix string, recursive bool) ([]string, error)
  • func (cs ConsulStorage) Load(ctx context.Context, key string) ([]byte, error)
  • func (cs *ConsulStorage) Lock(ctx context.Context, key string) error
  • func (cs *ConsulStorage) Provision(ctx caddy.Context) error
  • func (cs ConsulStorage) Stat(ctx context.Context, key string) (certmagic.KeyInfo, error)
  • func (cs ConsulStorage) Store(ctx context.Context, key string, value []byte) error
  • func (cs *ConsulStorage) Unlock(_ context.Context, key string) error
  • func (cs *ConsulStorage) UnmarshalCaddyfile(d *caddyfile.Dispenser) error
• type StorageData

Constants

const (
	// DefaultPrefix defines the default prefix in KV store
	DefaultPrefix = "caddytls"

	// DefaultAESKey needs to be 32 bytes long
	DefaultAESKey = "consultls-1234567890-caddytls-32"

	// DefaultValuePrefix sets a prefix to KV values to check validation
	DefaultValuePrefix = "caddy-storage-consul"

	// DefaultTimeout is the default timeout for Consul connections
	DefaultTimeout = 10

	// EnvNameAESKey defines the env variable name to override AES key
	EnvNameAESKey = "CADDY_CLUSTERING_CONSUL_AESKEY"

	// EnvNamePrefix defines the env variable name to override KV key prefix
	EnvNamePrefix = "CADDY_CLUSTERING_CONSUL_PREFIX"

	// EnvValuePrefix defines the env variable name to override KV value prefix
	EnvValuePrefix = "CADDY_CLUSTERING_CONSUL_VALUEPREFIX"
)

Functions

func ConsulQueryDefaults

func ConsulQueryDefaults(ctx context.Context) *consul.QueryOptions

Types

type ConsulStorage

type ConsulStorage struct {
	certmagic.Storage
	ConsulClient *consul.Client

	Address     string `json:"address"`
	Token       string `json:"token"`
	Timeout     int    `json:"timeout"`
	Prefix      string `json:"prefix"`
	ValuePrefix string `json:"value_prefix"`
	AESKey      []byte `json:"aes_key"`
	TlsEnabled  bool   `json:"tls_enabled"`
	TlsInsecure bool   `json:"tls_insecure"`
	// contains filtered or unexported fields
}

ConsulStorage allows to store certificates and other TLS resources in a shared cluster environment using Consul's key/value-store. It uses distributed locks to ensure consistency.

func New

func New() *ConsulStorage

New connects to Consul and returns a ConsulStorage

func (ConsulStorage) CaddyModule

func (ConsulStorage) CaddyModule() caddy.ModuleInfo

func (*ConsulStorage) CertMagicStorage

func (cs *ConsulStorage) CertMagicStorage() (certmagic.Storage, error)

func (*ConsulStorage) DecryptStorageData

func (cs *ConsulStorage) DecryptStorageData(bytes []byte) (*StorageData, error)

func (ConsulStorage) Delete

func (cs ConsulStorage) Delete(ctx context.Context, key string) error

Delete a key from Consul KV

func (*ConsulStorage) EncryptStorageData

func (cs *ConsulStorage) EncryptStorageData(data *StorageData) ([]byte, error)

func (ConsulStorage) Exists

func (cs ConsulStorage) Exists(ctx context.Context, key string) bool

Exists checks if a key exists

func (*ConsulStorage) GetLock

func (cs *ConsulStorage) GetLock(key string) (*consul.Lock, bool)

func (ConsulStorage) List

func (cs ConsulStorage) List(ctx context.Context, prefix string, recursive bool) ([]string, error)

List returns a list with all keys under a given prefix

func (ConsulStorage) Load

func (cs ConsulStorage) Load(ctx context.Context, key string) ([]byte, error)

Load retrieves the value for a key from Consul KV

func (*ConsulStorage) Lock

func (cs *ConsulStorage) Lock(ctx context.Context, key string) error

Lock acquires a distributed lock for the given key or blocks until it gets one

func (*ConsulStorage) Provision

func (cs *ConsulStorage) Provision(ctx caddy.Context) error

Provision is called by Caddy to prepare the module

func (ConsulStorage) Stat

func (cs ConsulStorage) Stat(ctx context.Context, key string) (certmagic.KeyInfo, error)

Stat returns statistic data of a key

func (ConsulStorage) Store

func (cs ConsulStorage) Store(ctx context.Context, key string, value []byte) error

Store saves encrypted data value for a key in Consul KV

func (*ConsulStorage) Unlock

func (cs *ConsulStorage) Unlock(_ context.Context, key string) error

Unlock releases a specific lock

func (*ConsulStorage) UnmarshalCaddyfile

func (cs *ConsulStorage) UnmarshalCaddyfile(d *caddyfile.Dispenser) error

UnmarshalCaddyfile parses plugin settings from Caddyfile

storage consul {
    address      "127.0.0.1:8500"
    token        "consul-access-token"
    timeout      10
    prefix       "caddytls"
    value_prefix "myprefix"
    aes_key      "consultls-1234567890-caddytls-32"
    tls_enabled  "false"
    tls_insecure "true"
}

type StorageData

type StorageData struct {
	Value    []byte    `json:"value"`
	Modified time.Time `json:"modified"`
}

StorageData describes the data that is saved to KV