gcsblob

Documentation

Overview

Package gcsblob provides a blob implementation that uses GCS. Use OpenBucket to construct a *blob.Bucket.

URLs

For blob.OpenBucket, gcsblob registers for the scheme "gs". The default URL opener will creating a connection using use default credentials from the environment, as described in https://cloud.google.com/docs/authentication/production. To customize the URL opener, or for more details on the URL format, see URLOpener. See https://gocloud.dev/concepts/urls/ for background information.

Escaping

Go CDK supports all UTF-8 strings; to make this work with providers lacking full UTF-8 support, strings must be escaped (during writes) and unescaped (during reads). The following escapes are performed for gcsblob:

• Blob keys: ASCII characters 10 and 13 are escaped to "__0x<hex>__". Additionally, the "/" in "../" is escaped in the same way.

As

gcsblob exposes the following types for As:

• Bucket: *storage.Client
• Error: *googleapi.Error
• ListObject: storage.ObjectAttrs
• ListOptions.BeforeList: *storage.Query
• Reader: *storage.Reader
• ReaderOptions.BeforeRead: **storage.ObjectHandle, *storage.Reader
• Attributes: storage.ObjectAttrs
• CopyOptions.BeforeCopy: *CopyObjectHandles, *storage.Copier
• WriterOptions.BeforeWrite: **storage.ObjectHandle, *storage.Writer

Example (OpenBucketFromURL)

package main

import (
	"context"
	"log"

	"gocloud.dev/blob"
)

func main() {
	// This example is used in https://gocloud.dev/howto/blob/open-bucket/#gcs

	// import _ "gocloud.dev/blob/gcsblob"

	// Variables set up elsewhere:
	ctx := context.Background()

	// blob.OpenBucket creates a *blob.Bucket from a URL.
	// This URL will open the bucket "my-bucket" using default credentials.
	bucket, err := blob.OpenBucket(ctx, "gs://my-bucket")
	if err != nil {
		log.Fatal(err)
	}
	defer bucket.Close()
}

Index

• Constants
• Variables
• func OpenBucket(ctx context.Context, client *gcp.HTTPClient, bucketName string, opts *Options) (*blob.Bucket, error)
• type CopyObjectHandles
• type Options
• type URLOpener
  • func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

Examples

• Package (OpenBucketFromURL)
• OpenBucket

Constants

const Scheme = "gs"

Scheme is the URL scheme gcsblob registers its URLOpener under on blob.DefaultMux.

Variables

var Set = wire.NewSet(
	Options{},
	URLOpener{},
)

Set holds Wire providers for this package.

Functions

func OpenBucket

func OpenBucket(ctx context.Context, client *gcp.HTTPClient, bucketName string, opts *Options) (*blob.Bucket, error)

OpenBucket returns a *blob.Bucket backed by an existing GCS bucket. See the package documentation for an example.

Example

package main

import (
	"context"
	"log"

	"gocloud.dev/blob/gcsblob"
	"gocloud.dev/gcp"
)

func main() {
	// This example is used in https://gocloud.dev/howto/blob/open-bucket/#gcs-ctor

	// Variables set up elsewhere:
	ctx := context.Background()

	// Your GCP credentials.
	// See https://cloud.google.com/docs/authentication/production
	// for more info on alternatives.
	creds, err := gcp.DefaultCredentials(ctx)
	if err != nil {
		log.Fatal(err)
	}

	// Create an HTTP client.
	// This example uses the default HTTP transport and the credentials
	// created above.
	client, err := gcp.NewHTTPClient(
		gcp.DefaultTransport(),
		gcp.CredentialsTokenSource(creds))
	if err != nil {
		log.Fatal(err)
	}

	// Create a *blob.Bucket.
	bucket, err := gcsblob.OpenBucket(ctx, client, "my-bucket", nil)
	if err != nil {
		log.Fatal(err)
	}
	defer bucket.Close()
}

Types

type CopyObjectHandles

type CopyObjectHandles struct {
	Dst, Src *storage.ObjectHandle
}

CopyObjectHandles holds the ObjectHandles for the destination and source of a Copy. It is used by the BeforeCopy As hook.

type Options

type Options struct {
	// GoogleAccessID represents the authorizer for SignedURL.
	// Required to use SignedURL.
	// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
	GoogleAccessID string

	// PrivateKey is the Google service account private key.
	// Exactly one of PrivateKey or SignBytes must be non-nil to use SignedURL.
	// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
	PrivateKey []byte

	// SignBytes is a function for implementing custom signing.
	// Exactly one of PrivateKey or SignBytes must be non-nil to use SignedURL.
	// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
	SignBytes func([]byte) ([]byte, error)
}

Options sets options for constructing a *blob.Bucket backed by GCS.

type URLOpener

type URLOpener struct {
	// Client must be set to a non-nil HTTP client authenticated with
	// Cloud Storage scope or equivalent.
	Client *gcp.HTTPClient

	// Options specifies the default options to pass to OpenBucket.
	Options Options
}

URLOpener opens GCS URLs like "gs://mybucket".

The URL host is used as the bucket name.

The following query parameters are supported:

• access_id: sets Options.GoogleAccessID
• private_key_path: path to read for Options.PrivateKey

func (*URLOpener) OpenBucketURL

func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

OpenBucketURL opens the GCS bucket with the same name as the URL's host.