s3

Documentation

Overview

The `s3` plugin for SHIELD is intended to be a back-end storage plugin, wrapping Amazon's Simple Storage Service (S3). It should be compatible with other services which emulate the S3 API, offering similar storage solutions for private cloud offerings (such as OpenStack Swift). However, this plugin has only been tested with Amazon S3.

PLUGIN FEATURES

This plugin implements functionality suitable for use with the following SHIELD Job components:

Target: no
Store:  yes

PLUGIN CONFIGURATION

The endpoint configuration passed to this plugin is used to determine how to connect to S3, and where to place/retrieve the data once connected. your endpoint JSON should look something like this:

{
    "s3_host":             "s3.amazonaws.com", # default
    "access_key_id":       "your-access-key-id",
    "secret_access_key":   "your-secret-access-key",
    "skip_ssl_validation":  false,
    "bucket":              "bucket-name",
    "prefix":              "/path/inside/bucket/to/place/backup/data",
    "signature_version":   "4",  # should be 2 or 4. Defaults to 4
    "socks5_proxy":        ""    # optionally defined SOCKS5 proxy to use for the s3 communications
    "s3_port":             ""    # optionally defined port to use for the s3 communications
}

Default Configuration

{
    "s3_host"             : "s3.amazonawd.com",
    "signature_version"   : "4",
    "skip_ssl_validation" : false
}

`prefix` will default to the empty string, and backups will be placed in the root of the bucket.

The `s3_port` field is optional. If specified, `s3_host` cannot be empty.

STORE DETAILS

When storing data, this plugin connects to the S3 service, and uploads the data into the specified bucket, using a path/filename with the following format:

<prefix>/<YYYY>/<MM>/<DD>/<HH-mm-SS>-<UUID>

Upon successful storage, the plugin then returns this filename to SHIELD to use as the `store_key` when the data needs to be retrieved, or purged.

RETRIEVE DETAILS

When retrieving data, this plugin connects to the S3 service, and retrieves the data located in the specified bucket, identified by the `store_key` provided by SHIELD.

PURGE DETAILS

When purging data, this plugin connects to the S3 service, and deletes the data located in the specified bucket, identified by the `store_key` provided by SHIELD.

DEPENDENCIES

None.