dynamodbstorage

README

DynamoDB Storage adapter for CertMagic

CertMagic Is an awesome package for doing all the heavy lifting with Let's Encrypt for certificate provisioning and use. In order to be flexible about how you use CertMagic, it has support for various methods of storing certificates. CertMagic has a Storage Interface allowing for multiple storage implementations.

This package is an implementation of the Storage interface that uses DynamoDB for certificate storage. We created this implementation for use in a clustered environment where our application runs in containers behind a load-balancer with no shared filesystem.

Authentication with AWS

This package uses the AWS Go library for interactions with DynamoDB. As such that library can detect your AWS credentials in multiple ways, in the following order:

1. Environment Variables
2. Shared Credentials file
3. Shared Configuration file (if SharedConfig is enabled)
4. EC2 Instance Metadata (credentials only)

For more information about authentication see https://docs.aws.amazon.com/sdk-for-go/api/aws/session/.

Usage

package whatever

import (
    "github.com/caddyserver/certmagic"

    dynamodbstore "github.com/silinternational/certmagic-storage-dynamodb/v3"
)

// ...

certmagic.Default.Storage = &dynamodbstore.DynamoDBStorage{
    Table:               "CertMagic",
    LockTimeout:         2 * time.Minute // optional: default is 5 minutes
    LockPollingInterval: 2 * time.Second // optional: default is 5 seconds
}

// ...

Only the table name is required, but you can also override the default values for LockTimeout and LockPollingTimeout if you want. Technically you can also override AwsEndpoint, AwsRegion, and AwsDisableSSL if you are running your own DynamoDB service. These settings are used in the unit tests so you can look there for examples.

Testing locally

You can build and run the tests for this package locally so long as you have Docker and Docker Compose available. Just run docker-compose run test. You could also run the DynamoDB local service separately and just run the tests from your local system, just be sure to adjust the AWS_ENDPOINT environment variable to point to where you have dynamodb-local running.

Creating the DynamoDB Table

Command line:

aws dynamodb create-table \
    --table-name CertMagic \
    --billing-mode PAY_PER_REQUEST \
    --attribute-definitions AttributeName=PrimaryKey,AttributeType=S \
    --key-schema AttributeName=PrimaryKey,KeyType=HASH

Terraform

resource "aws_dynamodb_table" "CertMagic" {
  name           = "CertMagic"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "PrimaryKey"

  attribute {
    name = "PrimaryKey"
    type = "S"
  }
}

Contributing

Please do, we like reported issues and pull requests.

License

MIT License

Copyright (c) 2022 SIL International

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Documentation

Index

• type Item
• type Storage
  • func (Storage) CaddyModule() caddy.ModuleInfo
  • func (s *Storage) CertMagicStorage() (certmagic.Storage, error)
  • func (s *Storage) Delete(_ context.Context, key string) error
  • func (s *Storage) Exists(ctx context.Context, key string) bool
  • func (s *Storage) List(_ context.Context, prefix string, recursive bool) ([]string, error)
  • func (s *Storage) Load(_ context.Context, key string) ([]byte, error)
  • func (s *Storage) Lock(ctx context.Context, key string) error
  • func (s *Storage) Stat(_ context.Context, key string) (certmagic.KeyInfo, error)
  • func (s *Storage) Store(_ context.Context, key string, value []byte) error
  • func (s *Storage) Unlock(ctx context.Context, key string) error
  • func (s *Storage) UnmarshalCaddyfile(d *caddyfile.Dispenser) error

Types

type Item

type Item struct {
	PrimaryKey  string    `json:"PrimaryKey"`
	Contents    string    `json:"Contents"`
	LastUpdated time.Time `json:"LastUpdated"`
}

Item holds structure of domain, certificate data, and last updated for marshaling with DynamoDb

type Storage

type Storage struct {
	// Table - [required] DynamoDB table name
	Table      string           `json:"table,omitempty"`
	AwsSession *session.Session `json:"-"`

	// AwsEndpoint - [optional] provide an override for DynamoDB service.
	// By default it'll use the standard production DynamoDB endpoints.
	// Useful for testing with a local DynamoDB instance.
	AwsEndpoint string `json:"aws_endpoint,omitempty"`

	// AwsRegion - [optional] region using DynamoDB in.
	// Useful for testing with a local DynamoDB instance.
	AwsRegion string `json:"aws_region,omitempty"`

	// AwsDisableSSL - [optional] disable SSL for DynamoDB connections. Default: false
	// Only useful for local testing, do not use outside of local testing.
	AwsDisableSSL bool `json:"aws_disable_ssl,omitempty"`

	// LockTimeout - [optional] how long to wait for a lock to be created. Default: 5 minutes
	LockTimeout caddy.Duration `json:"lock_timeout,omitempty"`

	// LockPollingInterval - [optional] how often to check for lock released. Default: 5 seconds
	LockPollingInterval caddy.Duration `json:"lock_polling_interval,omitempty"`
}

Storage implements certmagic.Storage to facilitate storage of certificates in DynamoDB for a clustered environment. Also implements certmagic.Locker to facilitate locking and unlocking of cert data during storage

func (Storage) CaddyModule

func (Storage) CaddyModule() caddy.ModuleInfo

CaddyModule returns the Caddy module information.

func (*Storage) CertMagicStorage

func (s *Storage) CertMagicStorage() (certmagic.Storage, error)

CertMagicStorage converts s to a certmagic.Storage instance.

func (*Storage) Delete

func (s *Storage) Delete(_ context.Context, key string) error

Delete deletes key.

func (*Storage) Exists

func (s *Storage) Exists(ctx context.Context, key string) bool

Exists returns true if the key exists and there was no error checking.

func (*Storage) List

func (s *Storage) List(_ context.Context, prefix string, recursive bool) ([]string, error)

List returns all keys that match prefix. If recursive is true, non-terminal keys will be enumerated (i.e. "directories" should be walked); otherwise, only keys prefixed exactly by prefix will be listed.

func (*Storage) Load

func (s *Storage) Load(_ context.Context, key string) ([]byte, error)

Load retrieves the value at key.

func (*Storage) Lock

func (s *Storage) Lock(ctx context.Context, key string) error

Lock acquires the lock for key, blocking until the lock can be obtained or an error is returned. Note that, even after acquiring a lock, an idempotent operation may have already been performed by another process that acquired the lock before - so always check to make sure idempotent operations still need to be performed after acquiring the lock.

The actual implementation of obtaining of a lock must be an atomic operation so that multiple Lock calls at the same time always results in only one caller receiving the lock at any given time.

To prevent deadlocks, all implementations (where this concern is relevant) should put a reasonable expiration on the lock in case Unlock is unable to be called due to some sort of network failure or system crash.

func (*Storage) Stat

func (s *Storage) Stat(_ context.Context, key string) (certmagic.KeyInfo, error)

Stat returns information about key.

func (*Storage) Store

func (s *Storage) Store(_ context.Context, key string, value []byte) error

Store puts value at key.

func (*Storage) Unlock

func (s *Storage) Unlock(ctx context.Context, key string) error

Unlock releases the lock for key. This method must ONLY be called after a successful call to Lock, and only after the critical section is finished, even if it errored or timed out. Unlock cleans up any resources allocated during Lock.

func (*Storage) UnmarshalCaddyfile

func (s *Storage) UnmarshalCaddyfile(d *caddyfile.Dispenser) error

UnmarshalCaddyfile sets up the storage module from Caddyfile tokens. Syntax:

dynamodb <table_name> {
    aws_endpoint <endpoint>
    aws_region   <region>
}

Only the table name is required.