s3backupstorage

README

Recently added options to enable usage of an S3 appliance: Cloudian
HyperStore:
        --s3_backup_aws_endpoint <host:port> (port is required)
        --s3_backup_force_path_style=true/false
        --s3_backup_log_level <level> can be one of: LogOff, LogDebug, LogDebugWithSigning, LogDebugWithHTTPBody, LogDebugWithRequestRetries, LogDebugWithRequestErrors.  Default: LogOff

By default the s3 client will try to connect to
<path>.<region>.amazonaws.com.  Adjusting the endpoint will allow this
to be changed.

Given the way the FQDN is configured the TLS certificate may not match the
server's "base" (<region>.<end_point_address>) due to the leading <path>
so setting --s3_backup_force_path_style=true will force the s3 client to
connect to <region>.<endpoint> and then make a request using the full
path within the http calls.

--s3backup_log_level enables more verbose logging of the S3 calls.

Documentation

Overview

Package s3backupstorage implements the BackupStorage interface for AWS S3.

AWS access credentials are configured via standard AWS means, such as: - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables - credentials file at ~/.aws/credentials - if running on an EC2 instance, an IAM role See details at http://blogs.aws.amazon.com/security/post/Tx3D6U6WSFGOK2H/A-New-and-Standardized-Way-to-Manage-Credentials-in-the-AWS-SDKs

Index

• type ClosedConnectionRetryer
  • func (retryer *ClosedConnectionRetryer) MaxRetries() int
  • func (retryer *ClosedConnectionRetryer) RetryRules(r *request.Request) time.Duration
  • func (retryer *ClosedConnectionRetryer) ShouldRetry(r *request.Request) bool
• type S3BackupHandle
  • func (bh *S3BackupHandle) AbortBackup(ctx context.Context) error
  • func (bh *S3BackupHandle) AddFile(ctx context.Context, filename string, filesize int64) (io.WriteCloser, error)
  • func (bh *S3BackupHandle) Directory() string
  • func (bh *S3BackupHandle) EndBackup(ctx context.Context) error
  • func (bh *S3BackupHandle) Error() error
  • func (bh *S3BackupHandle) HasErrors() bool
  • func (bh *S3BackupHandle) Name() string
  • func (bh *S3BackupHandle) ReadFile(ctx context.Context, filename string) (io.ReadCloser, error)
  • func (bh *S3BackupHandle) RecordError(err error)
• type S3BackupStorage
  • func (bs *S3BackupStorage) Close() error
  • func (bs *S3BackupStorage) ListBackups(ctx context.Context, dir string) ([]backupstorage.BackupHandle, error)
  • func (bs *S3BackupStorage) RemoveBackup(ctx context.Context, dir, name string) error
  • func (bs *S3BackupStorage) StartBackup(ctx context.Context, dir, name string) (backupstorage.BackupHandle, error)
• type S3ServerSideEncryption

Types

type ClosedConnectionRetryer

type ClosedConnectionRetryer struct {
	// contains filtered or unexported fields
}

ClosedConnectionRetryer implements the aws request.Retryer interface and is used to retry closed connection errors during MultipartUpload operations.

func (*ClosedConnectionRetryer) MaxRetries

func (retryer *ClosedConnectionRetryer) MaxRetries() int

MaxRetries is part of the Retryer interface. It defers to the underlying aws Retryer for the max number of retries.

func (*ClosedConnectionRetryer) RetryRules

func (retryer *ClosedConnectionRetryer) RetryRules(r *request.Request) time.Duration

RetryRules is part of the Retryer interface. It defers to the underlying aws Retryer to compute backoff rules.

func (*ClosedConnectionRetryer) ShouldRetry

func (retryer *ClosedConnectionRetryer) ShouldRetry(r *request.Request) bool

ShouldRetry is part of the Retryer interface. It retries on errors that occur due to a closed network connection, and then falls back to the underlying aws Retryer for checking additional retry conditions.

type S3BackupHandle

type S3BackupHandle struct {
	// contains filtered or unexported fields
}

S3BackupHandle implements the backupstorage.BackupHandle interface.

func (*S3BackupHandle) AbortBackup

func (bh *S3BackupHandle) AbortBackup(ctx context.Context) error

AbortBackup is part of the backupstorage.BackupHandle interface.

func (*S3BackupHandle) AddFile

func (bh *S3BackupHandle) AddFile(ctx context.Context, filename string, filesize int64) (io.WriteCloser, error)

AddFile is part of the backupstorage.BackupHandle interface.

func (*S3BackupHandle) Directory

func (bh *S3BackupHandle) Directory() string

Directory is part of the backupstorage.BackupHandle interface.

func (*S3BackupHandle) EndBackup

func (bh *S3BackupHandle) EndBackup(ctx context.Context) error

EndBackup is part of the backupstorage.BackupHandle interface.

func (*S3BackupHandle) Error

func (bh *S3BackupHandle) Error() error

Error is part of the concurrency.ErrorRecorder interface.

func (*S3BackupHandle) HasErrors

func (bh *S3BackupHandle) HasErrors() bool

HasErrors is part of the concurrency.ErrorRecorder interface.

func (*S3BackupHandle) Name

func (bh *S3BackupHandle) Name() string

Name is part of the backupstorage.BackupHandle interface.

func (*S3BackupHandle) ReadFile

func (bh *S3BackupHandle) ReadFile(ctx context.Context, filename string) (io.ReadCloser, error)

ReadFile is part of the backupstorage.BackupHandle interface.

func (*S3BackupHandle) RecordError

func (bh *S3BackupHandle) RecordError(err error)

RecordError is part of the concurrency.ErrorRecorder interface.

type S3BackupStorage

type S3BackupStorage struct {
	// contains filtered or unexported fields
}

S3BackupStorage implements the backupstorage.BackupStorage interface.

func (*S3BackupStorage) Close

func (bs *S3BackupStorage) Close() error

Close is part of the backupstorage.BackupStorage interface.

func (*S3BackupStorage) ListBackups

func (bs *S3BackupStorage) ListBackups(ctx context.Context, dir string) ([]backupstorage.BackupHandle, error)

ListBackups is part of the backupstorage.BackupStorage interface.

func (*S3BackupStorage) RemoveBackup

func (bs *S3BackupStorage) RemoveBackup(ctx context.Context, dir, name string) error

RemoveBackup is part of the backupstorage.BackupStorage interface.

func (*S3BackupStorage) StartBackup

func (bs *S3BackupStorage) StartBackup(ctx context.Context, dir, name string) (backupstorage.BackupHandle, error)

StartBackup is part of the backupstorage.BackupStorage interface.

type S3ServerSideEncryption

type S3ServerSideEncryption struct {
	// contains filtered or unexported fields
}