s3blob

Documentation

Overview

Package s3blob provides a blob implementation that uses S3. Use OpenBucket to construct a *blob.Bucket.

Open URLs

For blob.Open URLs, s3blob registers for the scheme "s3"; URLs start with "s3://".

The URL's Host is used as the bucket name. The AWS session is created as described in https://docs.aws.amazon.com/sdk-for-go/api/aws/session/. The following query options are supported:

• region: The AWS region for requests; sets aws.Config.Region.
• endpoint: The endpoint URL (hostname only or fully qualified URI); sets aws.Config.Endpoint.
• disableSSL: A value of "true" disables SSL when sending requests; sets aws.Config.DisableSSL.
• s3ForcePathStyle: A value of "true" forces the request to use path-style addressing; sets aws.Config.S3ForcePathStyle.

Example URL:

s3://mybucket?region=us-east-1

As

s3blob exposes the following types for As:

• Bucket: *s3.S3
• Error: awserr.Error
• ListObject: s3.Object for objects, s3.CommonPrefix for "directories"
• ListOptions.BeforeList: *s3.ListObjectsV2Input
• Reader: s3.GetObjectOutput
• Attributes: s3.HeadObjectOutput
• WriterOptions.BeforeWrite: *s3manager.UploadInput

Example

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/aws/aws-sdk-go/aws/session"
	"gocloud.dev/blob/s3blob"
)

func main() {
	// Set the environment variables AWS uses to load the session.
	// https://docs.aws.amazon.com/sdk-for-go/api/aws/session/
	prevAccessKey := os.Getenv("AWS_ACCESS_KEY")
	prevSecretKey := os.Getenv("AWS_SECRET_KEY")
	prevRegion := os.Getenv("AWS_REGION")
	os.Setenv("AWS_ACCESS_KEY", "myaccesskey")
	os.Setenv("AWS_SECRET_KEY", "mysecretkey")
	os.Setenv("AWS_REGION", "us-east-1")
	defer func() {
		os.Setenv("AWS_ACCESS_KEY", prevAccessKey)
		os.Setenv("AWS_SECRET_KEY", prevSecretKey)
		os.Setenv("AWS_REGION", prevRegion)
	}()

	// Establish an AWS session.
	session, err := session.NewSession(nil)
	if err != nil {
		log.Fatal(err)
	}

	// Create a *blob.Bucket.
	ctx := context.Background()
	b, err := s3blob.OpenBucket(ctx, session, "my-bucket", nil)
	if err != nil {
		log.Fatal(err)
	}
	_, err = b.ReadAll(ctx, "my-key")
	if err != nil {
		// This is expected due to the fake credentials we used above.
		fmt.Println("ReadAll failed due to invalid credentials")
	}

}

Output:

ReadAll failed due to invalid credentials

Example (Open)

package main

import (
	"context"

	"gocloud.dev/blob"
)

func main() {
	_, _ = blob.Open(context.Background(), "s3://my-bucket")

}

Index

• func OpenBucket(ctx context.Context, sess client.ConfigProvider, bucketName string, ...) (*blob.Bucket, error)
• type Options

Examples

• Package
• Package (Open)

Functions

func OpenBucket

func OpenBucket(ctx context.Context, sess client.ConfigProvider, bucketName string, opts *Options) (*blob.Bucket, error)

OpenBucket returns a *blob.Bucket backed by S3. See the package documentation for an example.

Types

type Options

type Options struct{}

Options sets options for constructing a *blob.Bucket backed by fileblob.