storage

Documentation

Index

• func Copy(ctx context.Context, src, dst FileRef) (err error)
• type Bucket
• type CloudBucket
  • func NewCloudBucket(ctx context.Context, url URL) (bucket *CloudBucket, err error)
  • func NewCloudBucketFromURL(ctx context.Context, url string) (bucket *CloudBucket, err error)
  • func (c *CloudBucket) CreateReader(ctx context.Context, ref FileRef) (reader io.ReadCloser, err error)
  • func (c *CloudBucket) CreateWriter(ctx context.Context, ref FileRef) (writer io.WriteCloser, err error)
  • func (c *CloudBucket) FileRef(path string) FileRef
  • func (c *CloudBucket) Url() URL
• type CloudFileRef
  • func (c *CloudFileRef) Bucket() Bucket
  • func (c *CloudFileRef) Path() string
  • func (c *CloudFileRef) Url() URL
• type FileRef
• type URL
  • func NewURL(driver, bucket string) URL
  • func ParseURL(bucketURL string) (parsed URL, err error)
  • func (u URL) String() string

Functions

func Copy

func Copy(ctx context.Context, src, dst FileRef) (err error)

Copy mimics the Go io.Copy function It is possible to copy FileRefs between buckets

TODO: implement a CopyWithin function that is optimized for copying within the same bucket Most cloud providers have APIs for this This will require extending the Bucket interface to support this https://gocloud.dev/blob supports this

Types

type Bucket

type Bucket interface {
	Url() URL
	FileRef(path string) FileRef
	CreateWriter(ctx context.Context, ref FileRef) (writer io.WriteCloser, err error)
	CreateReader(ctx context.Context, ref FileRef) (reader io.ReadCloser, err error)
}

Bucket is an abstraction for a storage bucket Implementers of this interface could reside locally or over networked storage This could be any cloud service or your local machines file system

type CloudBucket

type CloudBucket struct {
	// contains filtered or unexported fields
}

CloudBucket is a concrete implementation of the Bucket interface It adapts the gocloud.dev/blob.Bucket interface to our internal Bucket interface

func NewCloudBucket

func NewCloudBucket(ctx context.Context, url URL) (bucket *CloudBucket, err error)

NewCloudBucket opens a bucket (this could be in the cloud or local) This operation will return an error if the driver in the provided URL isn't supported This can be one of gs, file, s3, etc... See https://gocloud.dev/blob for more information

func NewCloudBucketFromURL

func NewCloudBucketFromURL(ctx context.Context, url string) (bucket *CloudBucket, err error)

NewCloudBucketFromURL opens a bucket by parsing the provided URL

func (*CloudBucket) CreateReader

func (c *CloudBucket) CreateReader(ctx context.Context, ref FileRef) (reader io.ReadCloser, err error)

CreateReader takes a FileRef and returns a reader where content can be read It is the callers responsibility to close the reader when they're done

func (*CloudBucket) CreateWriter

func (c *CloudBucket) CreateWriter(ctx context.Context, ref FileRef) (writer io.WriteCloser, err error)

CreateWriter takes a FileRef and returns a writer where content can be written It is the callers responsibility to close the writer when they're done Failing to do so can result in incomplete writes In the case of google cloud storage the file upload is transactional it will fail silently

func (*CloudBucket) FileRef

func (c *CloudBucket) FileRef(path string) FileRef

FileRef takes a path and returns a FileRef This should be relative to the buckets root

func (*CloudBucket) Url

func (c *CloudBucket) Url() URL

Url only returns the base URL for the bucket itself not a FileRef

type CloudFileRef

type CloudFileRef struct {
	// contains filtered or unexported fields
}

CloudFileRef is a concrete implementation of the FileRef interface It adapts the gocloud.dev/blob.Bucket interface to our internal FileRef interface

func (*CloudFileRef) Bucket

func (c *CloudFileRef) Bucket() Bucket

Bucket returns a pointer to the bucket that this file reference belongs to

func (*CloudFileRef) Path

func (c *CloudFileRef) Path() string

Path returns the path of the file reference relative to the bucket Given a URL of gs://bucket/path/to/file the path would be path/to/file

func (*CloudFileRef) Url

func (c *CloudFileRef) Url() URL

Url returns the URL for the file reference including the bucket as the base gs://bucket/path/to/file

type FileRef

type FileRef interface {
	Url() URL
	Path() string
	Bucket() Bucket
}

FileRef is an abstraction for a file reference This interface doesn't hold a handle to any specific file It acts as a pointer to a file in storage It is the buckets responsibility to open handles and exposing them using the std io interfaces

type URL

type URL struct {
	Driver string
	Bucket string
	Path   string
}

URL urls usually have a protocol/schema This concrete structure contains information about the storage system itself When interacting with cloud systems this can be in the form of gs://bucket/fil When interacting with local systems this can be in the form of file://bucket/file

func NewURL

func NewURL(driver, bucket string) URL

NewURL is a convenience function for creating an instance of a URL

func ParseURL

func ParseURL(bucketURL string) (parsed URL, err error)

ParseURL is a convenience function for parsing a URL string

func (URL) String

func (u URL) String() string

String serializes the URL to a string Usually this is in the form of driver://bucket/path