s3

README

S3 Storage Backend

The S3 backend provides support for Amazon S3 and compatible services.

Configuration

To use the S3 backend, you need to specify the following options in your connection string:

• endpoint: The endpoint URL of your S3 service.
• region: The AWS region.
• accessKey: Your AWS access key.
• secretKey: Your AWS secret key.
• sse: Server Side Encryption setting.
• pathStyle: Whether to use path-style URLs.
• accelerate: Whether to use S3 Transfer Acceleration.

A connection string for the S3 backend looks like this:

s3://myBucketName/my/prefix?region=region&accessKey=accessKey&secretKey=secretKey&sse=sse&pathStyle=true

Replace myBucketName/my/prefix, region, accessKey, secretKey, and sse with your actual parameters.

Documentation

Index

• Constants
• type Backend
  • func New(opts Options) (*Backend, error)
  • func (b *Backend) AbortMultipartUpload(ctx context.Context, opts *types.AbortMultipartUploadOpts) error
  • func (b *Backend) CompleteMultipartUpload(ctx context.Context, opts *types.CompleteMultipartUploadOpts) error
  • func (b *Backend) CreateMultipartUpload(ctx context.Context, opts *types.CreateMultipartUploadOpts) (*types.CreateMultipartUploadResult, error)
  • func (b *Backend) Delete(ctx context.Context, opts *types.DeleteOpts) error
  • func (b *Backend) Download(ctx context.Context, opts *types.DownloadOpts) (*types.DownloadResult, error)
  • func (b *Backend) GeneratePresignedUploadURL(ctx context.Context, opts *types.GeneratePresignedUploadURLOpts) (*types.GeneratePresignedUploadURLResult, error)
  • func (b *Backend) List(ctx context.Context, opts *types.ListOpts) (*types.ListResult, error)
  • func (b *Backend) Move(ctx context.Context, opts *types.MoveOpts) error
  • func (b *Backend) Stat(ctx context.Context, opts *types.StatOpts) (*types.StatResult, error)
  • func (b *Backend) Upload(ctx context.Context, opts *types.UploadOpts) error
  • func (b *Backend) UploadPart(ctx context.Context, opts *types.UploadPartOpts) (*types.UploadPartResult, error)
• type Options

Constants

const (
	// https://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html#RESTErrorResponses
	ErrCodeNoSuchKey = "NoSuchKey"
)

Types

type Backend

type Backend struct {
	Bucket        string
	Client        *s3.Client
	Presign       *s3.PresignClient
	Downloader    *manager.Downloader
	Prefix        string
	Uploader      *manager.Uploader
	SSE           string
	UseAccelerate bool
}

Backend is a storage backend for S3

func New

func New(opts Options) (*Backend, error)

New creates a new S3 backend

func (*Backend) AbortMultipartUpload

func (b *Backend) AbortMultipartUpload(ctx context.Context, opts *types.AbortMultipartUploadOpts) error

AbortMultipartUpload aborts the multipart upload

func (*Backend) CompleteMultipartUpload

func (b *Backend) CompleteMultipartUpload(ctx context.Context, opts *types.CompleteMultipartUploadOpts) error

CompleteMultipartUpload completes a multipart upload

func (*Backend) CreateMultipartUpload

func (b *Backend) CreateMultipartUpload(ctx context.Context, opts *types.CreateMultipartUploadOpts) (*types.CreateMultipartUploadResult, error)

CreateMultipartUpload initiates a multipart upload.

func (*Backend) Delete

func (b *Backend) Delete(ctx context.Context, opts *types.DeleteOpts) error

DeleteObject removes an object from a S3 bucket, at prefix

func (*Backend) Download

func (b *Backend) Download(ctx context.Context, opts *types.DownloadOpts) (*types.DownloadResult, error)

Download downloads an object from S3 bucket, at given path

func (*Backend) GeneratePresignedUploadURL

func (b *Backend) GeneratePresignedUploadURL(ctx context.Context, opts *types.GeneratePresignedUploadURLOpts) (*types.GeneratePresignedUploadURLResult, error)

GeneratePresignedUploadURL generates a presigned upload URL for an object in a S3 bucket

func (*Backend) List

func (b *Backend) List(ctx context.Context, opts *types.ListOpts) (*types.ListResult, error)

List lists all objects in a S3 bucket, at path Note: This function does not handle pagination and will return a maximum of 1000 objects. If there are more than 1000 objects with the specified path, consider using pagination to retrieve all objects.

func (*Backend) Move

func (b *Backend) Move(ctx context.Context, opts *types.MoveOpts) error

MoveObject moves an object from one path to another within a S3 bucket

func (*Backend) Stat

func (b *Backend) Stat(ctx context.Context, opts *types.StatOpts) (*types.StatResult, error)

Stat returns information about an object in a S3 bucket, at given path

func (*Backend) Upload

func (b *Backend) Upload(ctx context.Context, opts *types.UploadOpts) error

Upload uploads an object to a S3 bucket, at prefix. Will intelligently buffering large files into smaller chunks and sending them in parallel across multiple goroutines

func (*Backend) UploadPart

func (b *Backend) UploadPart(ctx context.Context, opts *types.UploadPartOpts) (*types.UploadPartResult, error)

UploadPart writes a part of a multipart upload

type Options

type Options struct {
	Endpoint      string
	Prefix        string
	Region        string
	AccessKey     string
	SecretKey     string
	BucketName    string
	SSE           string
	PathStyle     bool
	UseAccelerate bool
}

Options is a struct for S3 options