blobrepo

Documentation

Overview

Package blobrepo implements a generic reflow.Repository on top of a blob.Bucket.

Index

• func Dial(u *url.URL) (reflow.Repository, error)
• func Register(scheme string, store blob.Store)
• type Repository
  • func (r *Repository) Collect(ctx context.Context, live liveset.Liveset) error
  • func (r *Repository) CollectWithThreshold(ctx context.Context, live liveset.Liveset, dead liveset.Liveset, ...) error
  • func (r *Repository) Get(ctx context.Context, id digest.Digest) (io.ReadCloser, error)
  • func (r *Repository) GetFile(ctx context.Context, id digest.Digest, w io.WriterAt) (int64, error)
  • func (r *Repository) Put(ctx context.Context, body io.Reader) (digest.Digest, error)
  • func (r *Repository) PutFile(ctx context.Context, file reflow.File, body io.Reader) error
  • func (r *Repository) ReadFrom(_ context.Context, id digest.Digest, u *url.URL) error
  • func (r *Repository) Stat(ctx context.Context, id digest.Digest) (reflow.File, error)
  • func (r *Repository) String() string
  • func (r *Repository) URL() *url.URL
  • func (r *Repository) WriteTo(_ context.Context, id digest.Digest, u *url.URL) error

Functions

func Dial

func Dial(u *url.URL) (reflow.Repository, error)

Dial dials a blob repository. The URL must have the form:

type://bucket/prefix

TODO(marius): we should support shipping authentication information in the URL also.

func Register

func Register(scheme string, store blob.Store)

Register registers a blob store implementation used to dial repositories for the provided scheme.

Types

type Repository

type Repository struct {
	Bucket blob.Bucket
	Prefix string
}

Repository implements an blob-backed Repository. Objects are stored in the given bucket under the given prefix, followed by "objects":

type://bucket/<prefix>/objects/sha256:<hex>...

In-progress uploads are stored under "uploads":

type://bucket/<prefix>/uploads/<hex>

Prefix may be empty.

func (*Repository) Collect

func (r *Repository) Collect(ctx context.Context, live liveset.Liveset) error

Collect is not supported on S3.

func (*Repository) CollectWithThreshold

func (r *Repository) CollectWithThreshold(ctx context.Context, live liveset.Liveset, dead liveset.Liveset, threshold time.Time, dryRun bool) error

CollectWithThreshold removes from this repository any objects which are not in the liveset and which have not been accessed more recently than the liveset's threshold time

func (*Repository) Get

func (r *Repository) Get(ctx context.Context, id digest.Digest) (io.ReadCloser, error)

Get retrieves an object from the repository.

func (*Repository) GetFile

func (r *Repository) GetFile(ctx context.Context, id digest.Digest, w io.WriterAt) (int64, error)

GetFile retrieves an object from the repository directly to the a io.WriterAt. This uses the S3 download manager to download chunks concurrently.

func (*Repository) Put

func (r *Repository) Put(ctx context.Context, body io.Reader) (digest.Digest, error)

Put installs an object into the repository; its digest ID is returned.

func (*Repository) PutFile

func (r *Repository) PutFile(ctx context.Context, file reflow.File, body io.Reader) error

PutFile installs a file into the repository. PutFile uses the S3 upload manager directly.

func (*Repository) ReadFrom

func (r *Repository) ReadFrom(_ context.Context, id digest.Digest, u *url.URL) error

ReadFrom is unsupported by the blob repository.

TODO(marius): we can support other s3r here by performing CopyObjects.

func (*Repository) Stat

func (r *Repository) Stat(ctx context.Context, id digest.Digest) (reflow.File, error)

Stat queries the repository for object metadata.

func (*Repository) String

func (r *Repository) String() string

String returns the repository URL.

func (*Repository) URL

func (r *Repository) URL() *url.URL

URL returns the URL for this repository. It is of the form:

<type>://bucket/prefix

func (*Repository) WriteTo

func (r *Repository) WriteTo(_ context.Context, id digest.Digest, u *url.URL) error

WriteTo is unsupported by the blob repository.

TODO(marius): we can support other s3r here by performing CopyObjects.