gcsblob

Documentation

Overview

Package gcsblob provides a blob implementation that uses GCS. Use OpenBucket to construct a *blob.Bucket.

Open URLs

For blob.OpenBucket URLs, gcsblob registers for the scheme "gs"; URLs start with "gs://", like "gs://mybucket". blob.OpenBucket will use Application Default Credentials, as described in https://cloud.google.com/docs/authentication/production. If you want to use different credentials or find details on the format of the URL, see URLOpener.

Escaping

Go CDK supports all UTF-8 strings; to make this work with providers lacking full UTF-8 support, strings must be escaped (during writes) and unescaped (during reads). The following escapes are performed for gcsblob:

• Blob keys: ASCII characters 10 and 13 are escaped to "__0x<hex>__". Additionally, the "/" in "../" is escaped in the same way.

As

gcsblob exposes the following types for As:

• Bucket: *storage.Client
• Error: *googleapi.Error
• ListObject: storage.ObjectAttrs
• ListOptions.BeforeList: *storage.Query
• Reader: storage.Reader
• Attributes: storage.ObjectAttrs
• WriterOptions.BeforeWrite: *storage.Writer

Example (Open)

package main

import (
	"context"

	"gocloud.dev/blob"
)

func main() {
	ctx := context.Background()

	// Open creates a *blob.Bucket from a URL.
	// This URL will open the bucket "my-bucket" using default credentials.
	b, err := blob.OpenBucket(ctx, "gs://my-bucket")
	_ = b
	_ = err
}

Example (Read)

package main

import (
	"context"
	"log"

	"gocloud.dev/blob/gcsblob"
	"gocloud.dev/gcp"
)

func main() {
	// Your GCP credentials.
	// See https://cloud.google.com/docs/authentication/production
	// for more info on alternatives.
	ctx := context.Background()
	creds, err := gcp.DefaultCredentials(ctx)
	if err != nil {
		log.Fatal(err)
	}

	// Create an HTTP client.
	// This example uses the default HTTP transport and the credentials created
	// above.
	client, err := gcp.NewHTTPClient(gcp.DefaultTransport(), gcp.CredentialsTokenSource(creds))
	if err != nil {
		return
	}

	// Create a *blob.Bucket.
	b, err := gcsblob.OpenBucket(ctx, client, "my-bucket", nil)
	if err != nil {
		log.Fatal(err)
	}

	// Now we can use b to read or write files to the container.
	data, err := b.ReadAll(ctx, "my-key")
	if err != nil {
		log.Fatal(err)
	}
	_ = data
}

Index

• Constants
• func OpenBucket(ctx context.Context, client *gcp.HTTPClient, bucketName string, opts *Options) (*blob.Bucket, error)
• type Options
• type URLOpener
  • func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

Examples

• Package (Open)
• Package (Read)

Constants

const Scheme = "gs"

Scheme is the URL scheme gcsblob registers its URLOpener under on blob.DefaultMux.

Functions

func OpenBucket

func OpenBucket(ctx context.Context, client *gcp.HTTPClient, bucketName string, opts *Options) (*blob.Bucket, error)

OpenBucket returns a *blob.Bucket backed by an existing GCS bucket. See the package documentation for an example.

Types

type Options

type Options struct {
	// GoogleAccessID represents the authorizer for SignedURL.
	// Required to use SignedURL.
	// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
	GoogleAccessID string

	// PrivateKey is the Google service account private key.
	// Exactly one of PrivateKey or SignBytes must be non-nil to use SignedURL.
	// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
	PrivateKey []byte

	// SignBytes is a function for implementing custom signing.
	// Exactly one of PrivateKey or SignBytes must be non-nil to use SignedURL.
	// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
	SignBytes func([]byte) ([]byte, error)
}

Options sets options for constructing a *blob.Bucket backed by GCS.

type URLOpener

type URLOpener struct {
	// Client must be set to a non-nil HTTP client authenticated with
	// Cloud Storage scope or equivalent.
	Client *gcp.HTTPClient

	// Options specifies the default options to pass to OpenBucket.
	Options Options
}

URLOpener opens GCS URLs like "gs://mybucket".

This opener supports the following query parameters:

access_id: sets Options.GoogleAccessID
private_key_path: path to read for Options.PrivateKey

func (*URLOpener) OpenBucketURL

func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

OpenBucketURL opens the GCS bucket with the same name as the URL's host.