Documentation
¶
Overview ¶
Package gcsblob provides a blob implementation that uses GCS. Use OpenBucket to construct a *blob.Bucket.
URLs ¶
For blob.OpenBucket, gcsblob registers for the scheme "gs". The default URL opener will set up a connection using default credentials from the environment, as described in https://cloud.google.com/docs/authentication/production. Some environments, such as GCE, come without a private key. In such cases the IAM Credentials API will be configured for use in Options.MakeSignBytes, which will introduce latency to any and all calls to bucket.SignedURL that you can avoid by installing a service account credentials file or obtaining and configuring a private key: https://cloud.google.com/iam/docs/creating-managing-service-account-keys
To customize the URL opener, or for more details on the URL format, see URLOpener. See https://gocloud.dev/concepts/urls/ for background information.
Escaping ¶
Go CDK supports all UTF-8 strings; to make this work with services lacking full UTF-8 support, strings must be escaped (during writes) and unescaped (during reads). The following escapes are performed for gcsblob:
- Blob keys: ASCII characters 10 and 13 are escaped to "__0x<hex>__". Additionally, the "/" in "../" is escaped in the same way.
As ¶
gcsblob exposes the following types for As:
- Bucket: *storage.Client
- Error: *googleapi.Error
- ListObject: storage.ObjectAttrs
- ListOptions.BeforeList: *storage.Query
- Reader: *storage.Reader
- ReaderOptions.BeforeRead: **storage.ObjectHandle, *storage.Reader (if accessing both, must be in that order)
- Attributes: storage.ObjectAttrs
- CopyOptions.BeforeCopy: *CopyObjectHandles, *storage.Copier (if accessing both, must be in that order)
- WriterOptions.BeforeWrite: **storage.ObjectHandle, *storage.Writer (if accessing both, must be in that order)
- SignedURLOptions.BeforeSign: *storage.SignedURLOptions
Example (OpenBucketFromURL) ¶
package main
import (
"context"
"log"
"github.com/hy9be/gocloud/blob"
)
func main() {
// PRAGMA: This example is used on gocloud.dev; PRAGMA comments adjust how it is shown and can be ignored.
// PRAGMA: On gocloud.dev, add a blank import: _ "github.com/hy9be/gocloud/blob/gcsblob"
// PRAGMA: On gocloud.dev, hide lines until the next blank line.
ctx := context.Background()
// blob.OpenBucket creates a *blob.Bucket from a URL.
// This URL will open the bucket "my-bucket" using default credentials.
bucket, err := blob.OpenBucket(ctx, "gs://my-bucket")
if err != nil {
log.Fatal(err)
}
defer bucket.Close()
}
Output:
Index ¶
Examples ¶
Constants ¶
const Scheme = "gs"
Scheme is the URL scheme gcsblob registers its URLOpener under on blob.DefaultMux.
Variables ¶
Set holds Wire providers for this package.
Functions ¶
func OpenBucket ¶
func OpenBucket(ctx context.Context, client *gcp.HTTPClient, bucketName string, opts *Options) (*blob.Bucket, error)
OpenBucket returns a *blob.Bucket backed by an existing GCS bucket. See the package documentation for an example.
Example ¶
package main
import (
"context"
"log"
"github.com/hy9be/gocloud/blob/gcsblob"
"github.com/hy9be/gocloud/gcp"
)
func main() {
// PRAGMA: This example is used on gocloud.dev; PRAGMA comments adjust how it is shown and can be ignored.
// PRAGMA: On gocloud.dev, hide lines until the next blank line.
ctx := context.Background()
// Your GCP credentials.
// See https://cloud.google.com/docs/authentication/production
// for more info on alternatives.
creds, err := gcp.DefaultCredentials(ctx)
if err != nil {
log.Fatal(err)
}
// Create an HTTP client.
// This example uses the default HTTP transport and the credentials
// created above.
client, err := gcp.NewHTTPClient(
gcp.DefaultTransport(),
gcp.CredentialsTokenSource(creds))
if err != nil {
log.Fatal(err)
}
// Create a *blob.Bucket.
bucket, err := gcsblob.OpenBucket(ctx, client, "my-bucket", nil)
if err != nil {
log.Fatal(err)
}
defer bucket.Close()
}
Output:
Types ¶
type CopyObjectHandles ¶
type CopyObjectHandles struct {
Dst, Src *storage.ObjectHandle
}
CopyObjectHandles holds the ObjectHandles for the destination and source of a Copy. It is used by the BeforeCopy As hook.
type Options ¶
type Options struct {
// GoogleAccessID represents the authorizer for SignedURL.
// Required to use SignedURL.
// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
GoogleAccessID string
// PrivateKey is the Google service account private key.
// Exactly one of PrivateKey or SignBytes must be non-nil to use SignedURL.
// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
// Deprecated: Use MakeSignBytes instead.
PrivateKey []byte
// SignBytes is a function for implementing custom signing.
// Exactly one of PrivateKey, SignBytes, or MakeSignBytes must be non-nil to use SignedURL.
// See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
// Deprecated: Use MakeSignBytes instead.
SignBytes func([]byte) ([]byte, error)
// MakeSignBytes is a factory for functions that are being used in place of an empty SignBytes.
// If your implementation of 'SignBytes' needs a request context, set this instead.
MakeSignBytes func(requestCtx context.Context) SignBytesFunc
}
Options sets options for constructing a *blob.Bucket backed by GCS.
type SignBytesFunc ¶
SignBytesFunc is shorthand for the signature of Options.SignBytes.
type URLOpener ¶
type URLOpener struct {
// Client must be set to a non-nil HTTP client authenticated with
// Cloud Storage scope or equivalent.
Client *gcp.HTTPClient
// Options specifies the default options to pass to OpenBucket.
Options Options
}
URLOpener opens GCS URLs like "gs://mybucket".
The URL host is used as the bucket name.
The following query parameters are supported:
- access_id: sets Options.GoogleAccessID
- private_key_path: path to read for Options.PrivateKey
Currently their use is limited to SignedURL.
Source Files