gocloudsampleprovider

README

This is a simple custom provider for the Go CDK blob API.

The provider is named gocloudsampleprovider and it's a clone of fileblob; the URL is registered at gcspfile://.

Documentation

Overview

Package fileblob provides a blob implementation that uses the filesystem. Use OpenBucket to construct a *blob.Bucket.

To avoid partial writes, fileblob writes to a temporary file and then renames the temporary file to the final path on Close. By default, it creates these temporary files in `os.TempDir`. If `os.TempDir` is on a different mount than your base bucket path, the `os.Rename` will fail with `invalid cross-device link`. To avoid this, either configure the temp dir to use by setting the environment variable `TMPDIR`, or set `Options.NoTempDir` to `true` (fileblob will create the temporary files next to the actual files instead of in a temporary directory).

By default fileblob stores blob metadata in "sidecar" files under the original filename with an additional ".attrs" suffix. This behaviour can be changed via `Options.Metadata`; writing of those metadata files can be suppressed by setting it to `MetadataDontWrite` or its equivalent "metadata=skip" in the URL for the opener. In either case, absent any stored metadata many `blob.Attributes` fields will be set to default values.

URLs

For blob.OpenBucket, fileblob registers for the scheme "file". To customize the URL opener, or for more details on the URL format, see URLOpener. See https://gocloud.dev/concepts/urls/ for background information.

Escaping

Go CDK supports all UTF-8 strings; to make this work with services lacking full UTF-8 support, strings must be escaped (during writes) and unescaped (during reads). The following escapes are performed for fileblob:

• Blob keys: ASCII characters 0-31 are escaped to "__0x<hex>__". If os.PathSeparator != "/", it is also escaped. Additionally, the "/" in "../", the trailing "/" in "//", and a trailing "/" is key names are escaped in the same way. On Windows, the characters "<>:"|?*" are also escaped.

As

fileblob exposes the following types for As:

• Bucket: os.FileInfo
• Error: *os.PathError
• ListObject: os.FileInfo
• Reader: io.Reader
• ReaderOptions.BeforeRead: *os.File
• Attributes: os.FileInfo
• CopyOptions.BeforeCopy: *os.File
• WriterOptions.BeforeWrite: *os.File

Index

• Constants
• func OpenBucket(dir string, opts *Options) (*blob.Bucket, error)
• type Options
• type URLOpener
  • func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)
• type URLSigner
• type URLSignerHMAC
  • func NewURLSignerHMAC(baseURL *url.URL, secretKey []byte) *URLSignerHMAC
  • func (h *URLSignerHMAC) KeyFromURL(ctx context.Context, sURL *url.URL) (string, error)
  • func (h *URLSignerHMAC) URLFromKey(ctx context.Context, key string, opts *driver.SignedURLOptions) (*url.URL, error)

Constants

const (
	// Metadata gets written to a separate file.
	MetadataInSidecar metadataOption = ""
	// Writes won't carry metadata, as per the package docstring.
	MetadataDontWrite metadataOption = "skip"
)

Settings for Options.Metadata.

const Scheme = "gcspfile"

Scheme is the URL scheme fileblob registers its URLOpener under on blob.DefaultMux.

Functions

func OpenBucket

func OpenBucket(dir string, opts *Options) (*blob.Bucket, error)

OpenBucket creates a *blob.Bucket backed by the filesystem and rooted at dir, which must exist. See the package documentation for an example.

Types

type Options

type Options struct {
	// URLSigner implements signing URLs (to allow access to a resource without
	// further authorization) and verifying that a given URL is unexpired and
	// contains a signature produced by the URLSigner.
	// URLSigner is only required for utilizing the SignedURL API.
	URLSigner URLSigner

	// If true, create the directory backing the Bucket if it does not exist
	// (using os.MkdirAll).
	CreateDir bool

	// The FileMode to use when creating directories for the top-level directory
	// backing the bucket (when CreateDir is true), and for subdirectories for keys.
	// Defaults to 0777.
	DirFileMode os.FileMode

	// If true, don't use os.TempDir for temporary files, but instead place them
	// next to the actual files. This may result in "stranded" temporary files
	// (e.g., if the application is killed before the file cleanup runs).
	//
	// If your bucket directory is on a different mount than os.TempDir, you will
	// need to set this to true, as os.Rename will fail across mount points.
	NoTempDir bool

	// Refers to the strategy for how to deal with metadata (such as blob.Attributes).
	// For supported values please see the Metadata* constants.
	// If left unchanged, 'MetadataInSidecar' will be used.
	Metadata metadataOption
}

Options sets options for constructing a *blob.Bucket backed by fileblob.

type URLOpener

type URLOpener struct {
	// Options specifies the default options to pass to OpenBucket.
	Options Options
}

func (*URLOpener) OpenBucketURL

func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

OpenBucketURL opens a blob.Bucket based on u.

type URLSigner

type URLSigner interface {
	// URLFromKey defines how the bucket's object key will be turned
	// into a signed URL. URLFromKey must be safe to call from multiple goroutines.
	URLFromKey(ctx context.Context, key string, opts *driver.SignedURLOptions) (*url.URL, error)

	// KeyFromURL must be able to validate a URL returned from URLFromKey.
	// KeyFromURL must only return the object if if the URL is
	// both unexpired and authentic. KeyFromURL must be safe to call from
	// multiple goroutines. Implementations of KeyFromURL should not modify
	// the URL argument.
	KeyFromURL(ctx context.Context, surl *url.URL) (string, error)
}

URLSigner defines an interface for creating and verifying a signed URL for objects in a fileblob bucket. Signed URLs are typically used for granting access to an otherwise-protected resource without requiring further authentication, and callers should take care to restrict the creation of signed URLs as is appropriate for their application.

type URLSignerHMAC

type URLSignerHMAC struct {
	// contains filtered or unexported fields
}

URLSignerHMAC signs URLs by adding the object key, expiration time, and a hash-based message authentication code (HMAC) into the query parameters. Values of URLSignerHMAC with the same secret key will accept URLs produced by others as valid.

func NewURLSignerHMAC

func NewURLSignerHMAC(baseURL *url.URL, secretKey []byte) *URLSignerHMAC

NewURLSignerHMAC creates a URLSignerHMAC. If the secret key is empty, then NewURLSignerHMAC panics.

func (*URLSignerHMAC) KeyFromURL

func (h *URLSignerHMAC) KeyFromURL(ctx context.Context, sURL *url.URL) (string, error)

KeyFromURL checks expiry and signature, and returns the object key only if the signed URL is both authentic and unexpired.

func (*URLSignerHMAC) URLFromKey

func (h *URLSignerHMAC) URLFromKey(ctx context.Context, key string, opts *driver.SignedURLOptions) (*url.URL, error)

URLFromKey creates a signed URL by copying the baseURL and appending the object key, expiry, and signature as a query params.