blobstore

Documentation

Index

• Variables
• func GetBytes(ctx context.Context, bs Blobstore, key string, br BlobRange) ([]byte, string, error)
• func IsCheckAndPutError(err error) bool
• func IsNotFoundError(err error) bool
• func PutBytes(ctx context.Context, bs Blobstore, key string, data []byte) (string, error)
• type BlobRange
  • func NewBlobRange(offset, length int64) BlobRange
• type Blobstore
• type CheckAndPutError
  • func (err CheckAndPutError) Error() string
• type GCSBlobstore
  • func NewGCSBlobstore(gcs *storage.Client, bucketName, prefix string) *GCSBlobstore
  • func (bs *GCSBlobstore) CheckAndPut(ctx context.Context, expectedVersion, key string, reader io.Reader) (string, error)
  • func (bs *GCSBlobstore) Exists(ctx context.Context, key string) (bool, error)
  • func (bs *GCSBlobstore) Get(ctx context.Context, key string, br BlobRange) (io.ReadCloser, string, error)
  • func (bs *GCSBlobstore) Put(ctx context.Context, key string, reader io.Reader) (string, error)
• type InMemoryBlobstore
  • func NewInMemoryBlobstore() *InMemoryBlobstore
  • func (bs *InMemoryBlobstore) CheckAndPut(ctx context.Context, expectedVersion, key string, reader io.Reader) (string, error)
  • func (bs *InMemoryBlobstore) Exists(ctx context.Context, key string) (bool, error)
  • func (bs *InMemoryBlobstore) Get(ctx context.Context, key string, br BlobRange) (io.ReadCloser, string, error)
  • func (bs *InMemoryBlobstore) Put(ctx context.Context, key string, reader io.Reader) (string, error)
• type LocalBlobstore
  • func NewLocalBlobstore(dir string) *LocalBlobstore
  • func (bs *LocalBlobstore) CheckAndPut(ctx context.Context, expectedVersion, key string, reader io.Reader) (string, error)
  • func (bs *LocalBlobstore) Exists(ctx context.Context, key string) (bool, error)
  • func (bs *LocalBlobstore) Get(ctx context.Context, key string, br BlobRange) (io.ReadCloser, string, error)
  • func (bs *LocalBlobstore) Put(ctx context.Context, key string, reader io.Reader) (string, error)
• type NotFound
  • func (nf NotFound) Error() string

Variables

var AllRange = NewBlobRange(0, 0)

AllRange is a BlobRange instance covering all values

Functions

func GetBytes

func GetBytes(ctx context.Context, bs Blobstore, key string, br BlobRange) ([]byte, string, error)

GetBytes is a utility method calls bs.Get and handles reading the data from the returned io.ReadCloser and closing it.

func IsCheckAndPutError

func IsCheckAndPutError(err error) bool

IsCheckAndPutError is a helper method used to determine if CheckAndPut errors resulted because of version mismatches (Which happens when you have multiple) writers of a blob with a given key.

func IsNotFoundError

func IsNotFoundError(err error) bool

IsNotFoundError is a helper method used to determine if returned errors resulted because the key didn't exist as opposed to something going wrong.

func PutBytes

func PutBytes(ctx context.Context, bs Blobstore, key string, data []byte) (string, error)

PutBytes is a utility method calls bs.Put by wrapping the supplied []byte in an io.Reader

Types

type BlobRange

type BlobRange struct {
	// contains filtered or unexported fields
}

BlobRange represents a segment of a blob of data. Offset is the beginning of the range and Length is the size. If Length is 0 that means all data beyond offset will be read. Lengths cannot be negative. Negative offsets indicate distance from the end of the end of the blob.

func NewBlobRange

func NewBlobRange(offset, length int64) BlobRange

NewBlobRange creates a BlobRange with a given offset and length

type Blobstore

type Blobstore interface {
	Exists(ctx context.Context, key string) (bool, error)
	Get(ctx context.Context, key string, br BlobRange) (io.ReadCloser, string, error)
	Put(ctx context.Context, key string, reader io.Reader) (string, error)
	CheckAndPut(ctx context.Context, expectedVersion, key string, reader io.Reader) (string, error)
}

Blobstore is an interface for storing and retrieving blobs of data by key

type CheckAndPutError

type CheckAndPutError struct {
	Key             string
	ExpectedVersion string
	ActualVersion   string
}

CheckAndPutError is an error type used when CheckAndPut fails because of a version mismatch.

func (CheckAndPutError) Error

func (err CheckAndPutError) Error() string

Error (Required method of error) returns an error message for debugging

type GCSBlobstore

type GCSBlobstore struct {
	// contains filtered or unexported fields
}

GCSBlobstore provides a GCS implementation of the Blobstore interface

func NewGCSBlobstore

func NewGCSBlobstore(gcs *storage.Client, bucketName, prefix string) *GCSBlobstore

NewGCSBlobstore creates a new instance of a GCSBlobstare

func (*GCSBlobstore) CheckAndPut

func (bs *GCSBlobstore) CheckAndPut(ctx context.Context, expectedVersion, key string, reader io.Reader) (string, error)

CheckAndPut will check the current version of a blob against an expectedVersion, and if the versions match it will update the data and version associated with the key

func (*GCSBlobstore) Exists

func (bs *GCSBlobstore) Exists(ctx context.Context, key string) (bool, error)

Exists returns true if a blob exists for the given key, and false if it does not. For InMemoryBlobstore instances error should never be returned (though other implementations of this interface can)

func (*GCSBlobstore) Get

func (bs *GCSBlobstore) Get(ctx context.Context, key string, br BlobRange) (io.ReadCloser, string, error)

Get retrieves an io.reader for the portion of a blob specified by br along with its version

func (*GCSBlobstore) Put

func (bs *GCSBlobstore) Put(ctx context.Context, key string, reader io.Reader) (string, error)

Put sets the blob and the version for a key

type InMemoryBlobstore

type InMemoryBlobstore struct {
	// contains filtered or unexported fields
}

InMemoryBlobstore provides an in memory implementation of the Blobstore interface

func NewInMemoryBlobstore

func NewInMemoryBlobstore() *InMemoryBlobstore

NewInMemoryBlobstore creates an instance of an InMemoryBlobstore

func (*InMemoryBlobstore) CheckAndPut

func (bs *InMemoryBlobstore) CheckAndPut(ctx context.Context, expectedVersion, key string, reader io.Reader) (string, error)

CheckAndPut will check the current version of a blob against an expectedVersion, and if the versions match it will update the data and version associated with the key

func (*InMemoryBlobstore) Exists

func (bs *InMemoryBlobstore) Exists(ctx context.Context, key string) (bool, error)

Exists returns true if a blob exists for the given key, and false if it does not. For InMemoryBlobstore instances error should never be returned (though other implementations of this interface can)

func (*InMemoryBlobstore) Get

func (bs *InMemoryBlobstore) Get(ctx context.Context, key string, br BlobRange) (io.ReadCloser, string, error)

Get retrieves an io.reader for the portion of a blob specified by br along with its version

func (*InMemoryBlobstore) Put

func (bs *InMemoryBlobstore) Put(ctx context.Context, key string, reader io.Reader) (string, error)

Put sets the blob and the version for a key

type LocalBlobstore

type LocalBlobstore struct {
	RootDir string
}

LocalBlobstore is a Blobstore implementation that uses the local filesystem

func NewLocalBlobstore

func NewLocalBlobstore(dir string) *LocalBlobstore

NewLocalBlobstore returns a new LocalBlobstore instance

func (*LocalBlobstore) CheckAndPut

func (bs *LocalBlobstore) CheckAndPut(ctx context.Context, expectedVersion, key string, reader io.Reader) (string, error)

CheckAndPut will check the current version of a blob against an expectedVersion, and if the versions match it will update the data and version associated with the key

func (*LocalBlobstore) Exists

func (bs *LocalBlobstore) Exists(ctx context.Context, key string) (bool, error)

Exists returns true if a blob exists for the given key, and false if it does not. error may be returned if there are errors accessing the filesystem data.

func (*LocalBlobstore) Get

func (bs *LocalBlobstore) Get(ctx context.Context, key string, br BlobRange) (io.ReadCloser, string, error)

Get retrieves an io.reader for the portion of a blob specified by br along with its version

func (*LocalBlobstore) Put

func (bs *LocalBlobstore) Put(ctx context.Context, key string, reader io.Reader) (string, error)

Put sets the blob and the version for a key

type NotFound

type NotFound struct {
	Key string
}

NotFound is an error type used only when a key is not found in a Blobstore.

func (NotFound) Error

func (nf NotFound) Error() string

Error returns the key which was not found