storageredis

README

Caddy Cluster / Certmagic TLS cluster support for Redis

This plugin is based on similar plugin using Consul. Most of the aspect is also similar, I pretty much copy the crypto implementation. The reason I use Redis is because it easier to setup.

For now, this will support redis as single instance, or with replica, but NOT the cluster. This plugin utilize go-redis/redis for its client access and redislock for it's locking mechanism. See distlock for the lock algorithm.

This plugin currently work with versions of Caddy v2, for the previous version of Caddy use caddy-v1 branch.

Configuration

You enable Redis storage with Caddy by setting the storage module used, for example

{
	//all value is optional, here is the default
    storage redis {
        host          "127.0.0.1"
        port          6379
        address       "127.0.0.1:6379" // no default, but is build from host+":"+port, if set, then host and port is ignored
        username      ""
        password      ""
        db            1
        key_prefix    "caddytls"
        value_prefix  "caddy-storage-redis"
        timeout       5
        tls_enabled   "false"
        tls_insecure  "true"
        aes_key       "redistls-01234567890-caddytls-32" // optional, but must have 32 length
    }
    // because the option are set using env, there are no need for additional option value
}

:443 {

}

JSON example

{
    "admin": {
        "listen": "0.0.0.0:2019"
    },
    "storage": {
        "address": "redis:6379",
        "aes_key": "redistls-01234567890-caddytls-32",
        "db": 1,
        "host": "redis",
        "key_prefix": "caddytls",
        "module": "redis",
        "password": "",
        "port": "6379",
        "timeout": 5,
        "tls_enabled": false,
        "tls_insecure": true,
        "value_prefix": "caddy-storage-redis"
    }
}

There are additional environment variable for this plugin:

• CADDY_CLUSTERING_REDIS_HOST defines Redis Host, default is 127.0.0.1
• CADDY_CLUSTERING_REDIS_PORT defines Redis Port, default is 6379
• CADDY_CLUSTERING_REDIS_USERNAME defines Redis username, default is empty
• CADDY_CLUSTERING_REDIS_PASSWORD defines Redis password, default is empty
• CADDY_CLUSTERING_REDIS_DB defines Redis DB, default is 0
• CADDY_CLUSTERING_REDIS_TIMEOUT defines Redis Dial,Read,Write timeout, default is set to 5 for 5 seconds
• CADDY_CLUSTERING_REDIS_AESKEY defines your personal AES key to use when encrypting data. It needs to be 32 characters long.
• CADDY_CLUSTERING_REDIS_KEYPREFIX defines the prefix for the keys. Default is caddytls
• CADDY_CLUSTERING_REDIS_VALUEPREFIX defines the prefix for the values. Default is caddy-storage-redis
• CADDY_CLUSTERING_REDIS_TLS defines whether use Redis TLS Connection or not
• CADDY_CLUSTERING_REDIS_TLS_INSECURE defines whether verify Redis TLS Connection or not

TODO

• Add Redis Cluster or Sentinel support (probably need to update the distlock implementation first)

Documentation

Index

• Constants
• type RedisStorage
  • func (rd *RedisStorage) BuildRedisClient(ctx context.Context) error
  • func (RedisStorage) CaddyModule() caddy.ModuleInfo
  • func (rd *RedisStorage) CertMagicStorage() (certmagic.Storage, error)
  • func (rd *RedisStorage) DecryptStorageData(bytes []byte) (*StorageData, error)
  • func (rd RedisStorage) Delete(_ context.Context, key string) error
  • func (rd *RedisStorage) EncryptStorageData(data *StorageData) ([]byte, error)
  • func (rd RedisStorage) Exists(_ context.Context, key string) bool
  • func (rd *RedisStorage) GetAESKeyByte() []byte
  • func (rd *RedisStorage) GetConfigValue()
  • func (rd RedisStorage) List(_ context.Context, prefix string, recursive bool) ([]string, error)
  • func (rd RedisStorage) Load(_ context.Context, key string) ([]byte, error)
  • func (rd *RedisStorage) Lock(ctx context.Context, key string) error
  • func (rd *RedisStorage) Provision(ctx caddy.Context) error
  • func (rd *RedisStorage) ReplaceEnvConfigCaddy()
  • func (rd RedisStorage) Stat(_ context.Context, key string) (certmagic.KeyInfo, error)
  • func (rd RedisStorage) Store(_ context.Context, key string, value []byte) error
  • func (rd RedisStorage) String() string
  • func (rd *RedisStorage) Unlock(_ context.Context, key string) error
  • func (rd *RedisStorage) UnmarshalCaddyfile(d *caddyfile.Dispenser) error
• type StorageData

Constants

const (
	// LockDuration is lock time duration
	LockDuration = 10 * time.Second

	// LockFreshnessInterval is how often to update a lock's TTL. Locks with a TTL
	// more than this duration in the past (plus a grace period for latency) can be
	// considered stale.
	LockFreshnessInterval = 3 * time.Second

	// LockPollInterval is how frequently to check the existence of a lock
	LockPollInterval = 1 * time.Second

	// ScanCount is how many scan command might return
	ScanCount int64 = 100

	// DefaultAESKey needs to be 32 bytes long
	DefaultAESKey = ""

	// DefaultKeyPrefix defines the default prefix in KV store
	DefaultKeyPrefix = "caddytls"

	// DefaultValuePrefix sets a prefix to KV values to check validation
	DefaultValuePrefix = "caddy-storage-redis"

	// DefaultRedisHost define the Redis instance host
	DefaultRedisHost = "127.0.0.1"

	// DefaultRedisPort define the Redis instance port
	DefaultRedisPort = "6379"

	// DefaultRedisDB define the Redis DB number
	DefaultRedisDB = 0

	// DefaultRedisPassword define the Redis instance Username, if any
	DefaultRedisUsername = ""

	// DefaultRedisPassword define the Redis instance password, if any
	DefaultRedisPassword = ""

	// DefaultRedisTimeout define the Redis wait time in (s)
	DefaultRedisTimeout = 5

	// DefaultRedisTLS define the Redis TLS connection
	DefaultRedisTLS = false

	// DefaultRedisTLSInsecure define the Redis TLS connection
	DefaultRedisTLSInsecure = true

	// EnvNameRedisHost defines the env variable name to override Redis host
	EnvNameRedisHost = "CADDY_CLUSTERING_REDIS_HOST"

	// EnvNameRedisPort defines the env variable name to override Redis port
	EnvNameRedisPort = "CADDY_CLUSTERING_REDIS_PORT"

	// EnvNameRedisDB defines the env variable name to override Redis db number
	EnvNameRedisDB = "CADDY_CLUSTERING_REDIS_DB"

	// EnvNameRedisUsername defines the env variable name to override Redis username
	EnvNameRedisUsername = "CADDY_CLUSTERING_REDIS_USERNAME"

	// EnvNameRedisPassword defines the env variable name to override Redis password
	EnvNameRedisPassword = "CADDY_CLUSTERING_REDIS_PASSWORD"

	// EnvNameRedisTimeout defines the env variable name to override Redis wait timeout for dial, read, write
	EnvNameRedisTimeout = "CADDY_CLUSTERING_REDIS_TIMEOUT"

	// EnvNameAESKey defines the env variable name to override AES key
	EnvNameAESKey = "CADDY_CLUSTERING_REDIS_AESKEY"

	// EnvNameKeyPrefix defines the env variable name to override KV key prefix
	EnvNameKeyPrefix = "CADDY_CLUSTERING_REDIS_KEYPREFIX"

	// EnvNameValuePrefix defines the env variable name to override KV value prefix
	EnvNameValuePrefix = "CADDY_CLUSTERING_REDIS_VALUEPREFIX"

	// EnvNameTLSEnabled defines the env variable name to whether enable Redis TLS Connection or not
	EnvNameTLSEnabled = "CADDY_CLUSTERING_REDIS_TLS"

	// EnvNameTLSInsecure defines the env variable name to whether verify Redis TLS Connection or not
	EnvNameTLSInsecure = "CADDY_CLUSTERING_REDIS_TLS_INSECURE"
)

Types

type RedisStorage

type RedisStorage struct {
	Client       *redis.Client
	ClientLocker *redislock.Client
	Logger       *zap.SugaredLogger

	Address     string `json:"address"`
	Host        string `json:"host"`
	Port        string `json:"port"`
	DB          int    `json:"db"`
	Username    string `json:"username"`
	Password    string `json:"password"`
	Timeout     int    `json:"timeout"`
	KeyPrefix   string `json:"key_prefix"`
	ValuePrefix string `json:"value_prefix"`
	AesKey      string `json:"aes_key"`
	TlsEnabled  bool   `json:"tls_enabled"`
	TlsInsecure bool   `json:"tls_insecure"`
	// contains filtered or unexported fields
}

RedisStorage contain Redis client, and plugin option

func (*RedisStorage) BuildRedisClient

func (rd *RedisStorage) BuildRedisClient(ctx context.Context) error

GetRedisStorage build RedisStorage with it's client

func (RedisStorage) CaddyModule

func (RedisStorage) CaddyModule() caddy.ModuleInfo

register caddy module with ID caddy.storage.redis

func (*RedisStorage) CertMagicStorage

func (rd *RedisStorage) CertMagicStorage() (certmagic.Storage, error)

CertMagicStorage converts s to a certmagic.Storage instance.

func (*RedisStorage) DecryptStorageData

func (rd *RedisStorage) DecryptStorageData(bytes []byte) (*StorageData, error)

DecryptStorageData decrypt storage data, so we can read it

func (RedisStorage) Delete

func (rd RedisStorage) Delete(_ context.Context, key string) error

Delete deletes key.

func (*RedisStorage) EncryptStorageData

func (rd *RedisStorage) EncryptStorageData(data *StorageData) ([]byte, error)

EncryptStorageData encrypt storage data, so it won't be plain data

func (RedisStorage) Exists

func (rd RedisStorage) Exists(_ context.Context, key string) bool

Exists returns true if the key exists

func (*RedisStorage) GetAESKeyByte

func (rd *RedisStorage) GetAESKeyByte() []byte

func (*RedisStorage) GetConfigValue

func (rd *RedisStorage) GetConfigValue()

GetConfigValue get Config value from env, if already been set by Caddyfile, don't overwrite

func (RedisStorage) List

func (rd RedisStorage) List(_ context.Context, prefix string, recursive bool) ([]string, error)

List returns all keys that match prefix.

func (RedisStorage) Load

func (rd RedisStorage) Load(_ context.Context, key string) ([]byte, error)

Load retrieves the value at key.

func (*RedisStorage) Lock

func (rd *RedisStorage) Lock(ctx context.Context, key string) error

Lock is to lock value

func (*RedisStorage) Provision

func (rd *RedisStorage) Provision(ctx caddy.Context) error

func (*RedisStorage) ReplaceEnvConfigCaddy

func (rd *RedisStorage) ReplaceEnvConfigCaddy()

func (RedisStorage) Stat

func (rd RedisStorage) Stat(_ context.Context, key string) (certmagic.KeyInfo, error)

Stat returns information about key.

func (RedisStorage) Store

func (rd RedisStorage) Store(_ context.Context, key string, value []byte) error

Store values at key

func (RedisStorage) String

func (rd RedisStorage) String() string

func (*RedisStorage) Unlock

func (rd *RedisStorage) Unlock(_ context.Context, key string) error

Unlock is to unlock value

func (*RedisStorage) UnmarshalCaddyfile

func (rd *RedisStorage) UnmarshalCaddyfile(d *caddyfile.Dispenser) error

type StorageData

type StorageData struct {
	Value    []byte    `json:"value"`
	Modified time.Time `json:"modified"`
}

StorageData describe the data that is stored in KV storage