gcs

Documentation

Overview

Package gcs contains helpers that facilitate data transfer of Resources into Google Cloud Storage.

Index

• Constants
• Variables
• func JoinPath(elems ...string) string
• func PathComponents(uri string) (bucket, relativePath string, err error)
• type Client
  • func NewClient(ctx context.Context, bucketName, endpointURL string) (Client, error)
  • func (gcsClient Client) GetFileReader(ctx context.Context, fileName string) (io.ReadCloser, error)
  • func (gcsClient Client) GetFileWriter(ctx context.Context, fileName string) io.WriteCloser
  • func (gcsClient Client) IsBucketInProject(ctx context.Context, project string) (bool, error)

Constants

const DefaultCloudStorageEndpoint = "https://storage.googleapis.com/"

DefaultCloudStorageEndpoint represents the default cloud storage API endpoint. This should be passed to NewClient unless in a test environment.

Variables

var ErrInvalidGCSPath = errors.New("the GCS path is not valid. a bucket and folder must be included, along with a gs:// prefix. For example gs://bucket/folder")

ErrInvalidGCSPath is an error indicating the GCS path is not valid.

Functions

func JoinPath

func JoinPath(elems ...string) string

JoinPath is roughly equivalent to path/filepath.Join, except that it always uses forward slashes regardless of platform (because GCS does not recognize backslashes used by windows).

Each path element backslashes converted to forward slashes, and has leading and trailing slashes removed. Elements are then joined with forward slashes.

Warning: this may not be fully compatible with how directory paths are supposed to work, and should not be used except for writing to GCS. For writing files to a local filesystem, use path/filepath.Join.

func PathComponents

func PathComponents(uri string) (bucket, relativePath string, err error)

PathComponents takes a GCS path (e.g. gs://some_bucket/relative/path) and returns the bucket name and the relative path. For example, gs://some_bucket/relative/path would return some_bucket and relative/path. At least a bucket and a folder must be included.

Types

type Client

type Client struct {
	*storage.Client
	// contains filtered or unexported fields
}

Client represents a GCS API client belonging to some project.

func NewClient

func NewClient(ctx context.Context, bucketName, endpointURL string) (Client, error)

NewClient creates and returns a new gcs client for use in writing resources to an existing GCS bucket. Note `bucketName` must belong to an existing bucket. See here for how to create a GCS bucket: https://cloud.google.com/storage/docs/creating-buckets. TODO(b/243677730): Add support for creating buckets.

func (Client) GetFileReader

func (gcsClient Client) GetFileReader(ctx context.Context, fileName string) (io.ReadCloser, error)

GetFileReader returns a reader for a file in GCS named `fileName`. ErrObjectNotExist will be returned if the object is not found.

The caller must call Close on the returned Reader when done reading.

func (Client) GetFileWriter

func (gcsClient Client) GetFileWriter(ctx context.Context, fileName string) io.WriteCloser

GetFileWriter returns a write closer that allows the user to write to a file named `fileName` in the pre defined GCS bucket. Closing the write closer will send the written data to GCS.

func (Client) IsBucketInProject

func (gcsClient Client) IsBucketInProject(ctx context.Context, project string) (bool, error)

IsBucketInProject returns true if the bucket is in the GCP project.