gcsproxy

Documentation

Index

• type MutableObject
  • func NewMutableObject(chunkSize uint64, o *gcs.Object, bucket gcs.Bucket, leaser lease.FileLeaser, ...) (mo *MutableObject)
  • func (mo *MutableObject) CheckInvariants()
  • func (mo *MutableObject) Destroy() (err error)
  • func (mo *MutableObject) ReadAt(ctx context.Context, buf []byte, offset int64) (n int, err error)
  • func (mo *MutableObject) SourceGeneration() int64
  • func (mo *MutableObject) Stat(ctx context.Context, needClobbered bool) (sr StatResult, err error)
  • func (mo *MutableObject) Sync(ctx context.Context) (err error)
  • func (mo *MutableObject) Truncate(ctx context.Context, n int64) (err error)
  • func (mo *MutableObject) WriteAt(ctx context.Context, buf []byte, offset int64) (n int, err error)
• type ReadProxy
  • func NewReadProxy(chunkSize uint64, leaser lease.FileLeaser, bucket gcs.Bucket, o *gcs.Object, ...) (rp *ReadProxy)
  • func (rp *ReadProxy) CheckInvariants()
  • func (rp *ReadProxy) Destroy() (err error)
  • func (rp *ReadProxy) ReadAt(ctx context.Context, buf []byte, offset int64) (n int, err error)
  • func (rp *ReadProxy) Size() (size int64)
  • func (rp *ReadProxy) Upgrade(ctx context.Context) (rwl lease.ReadWriteLease, err error)
• type StatResult

Types

type MutableObject

type MutableObject struct {
	// contains filtered or unexported fields
}

A view on a particular generation of an object in GCS that allows random access reads and writes.

Reads may involve reading from a local cache. Writes are buffered locally until the Sync method is called, at which time a new generation of the object is created.

This type is not safe for concurrent access. The user must provide external synchronization around the methods where it is not otherwise noted.

func NewMutableObject

func NewMutableObject(
	chunkSize uint64,
	o *gcs.Object,
	bucket gcs.Bucket,
	leaser lease.FileLeaser,
	clock timeutil.Clock) (mo *MutableObject)

Create a view on the given GCS object generation, using the supplied leaser to mediate temporary space usage.

chunkSize is used when reading contents from GCS. See notes on NewReadProxy.

REQUIRES: o != nil

func (*MutableObject) CheckInvariants

func (mo *MutableObject) CheckInvariants()

Panic if any internal invariants are violated. Careful users can call this at appropriate times to help debug weirdness. Consider using syncutil.InvariantMutex to automate the process.

func (*MutableObject) Destroy

func (mo *MutableObject) Destroy() (err error)

Destroy any local file caches, putting the proxy into an indeterminate state. The MutableObject must not be used after calling this method, regardless of outcome.

func (*MutableObject) ReadAt

func (mo *MutableObject) ReadAt(
	ctx context.Context,
	buf []byte,
	offset int64) (n int, err error)

Make a random access read into our view of the content. May block for network access.

Guarantees that err != nil if n < len(buf)

func (*MutableObject) SourceGeneration

func (mo *MutableObject) SourceGeneration() int64

Return the generation of the object from which the current contents of this proxy were branched. If Sync has been successfully called, this is the generation most recently returned by Sync. Otherwise it is the generation from which the proxy was created.

func (*MutableObject) Stat

func (mo *MutableObject) Stat(
	ctx context.Context,
	needClobbered bool) (sr StatResult, err error)

Return the current size in bytes of the content and an indication of whether the proxied object has changed out from under us (in which case Sync will fail).

sr.Clobbered will be set only if needClobbered is true. Otherwise a round trip to GCS can be saved.

func (*MutableObject) Sync

func (mo *MutableObject) Sync(ctx context.Context) (err error)

If the proxy is dirty due to having been modified, save its current contents to GCS, creating a generation with exactly those contents. Do so with a precondition such that the creation will fail if the source generation is not current. In that case, return an error of type *gcs.PreconditionError. If the proxy is not dirty, simply return nil.

After this method successfully returns, SourceGeneration returns the generation at which the contents are current.

func (*MutableObject) Truncate

func (mo *MutableObject) Truncate(ctx context.Context, n int64) (err error)

Truncate our view of the content to the given number of bytes, extending if n is greater than the current size. May block for network access. Not guaranteed to be reflected remotely until after Sync is called successfully.

func (*MutableObject) WriteAt

func (mo *MutableObject) WriteAt(
	ctx context.Context,
	buf []byte,
	offset int64) (n int, err error)

Make a random access write into our view of the content. May block for network access. Not guaranteed to be reflected remotely until after Sync is called successfully.

Guarantees that err != nil if n < len(buf)

type ReadProxy

type ReadProxy struct {
	// contains filtered or unexported fields
}

A read-only view on a particular generation of an object in GCS. Reads may involve reading from a local cache.

This type is not safe for concurrent access. The user must provide external synchronization around the methods where it is not otherwise noted.

func NewReadProxy

func NewReadProxy(
	chunkSize uint64,
	leaser lease.FileLeaser,
	bucket gcs.Bucket,
	o *gcs.Object,
	rl lease.ReadLease) (rp *ReadProxy)

Create a view on the given GCS object generation. If rl is non-nil, it must contain a lease for the contents of the object and will be used when possible instead of re-reading the object.

If the object is larger than the given chunk size, we will only read and cache portions of it at a time.

func (*ReadProxy) CheckInvariants

func (rp *ReadProxy) CheckInvariants()

Panic if any internal invariants are violated.

func (*ReadProxy) Destroy

func (rp *ReadProxy) Destroy() (err error)

Destroy any local file caches, putting the proxy into an indeterminate state. Should be used before dropping the final reference to the proxy.

func (*ReadProxy) ReadAt

func (rp *ReadProxy) ReadAt(
	ctx context.Context,
	buf []byte,
	offset int64) (n int, err error)

Make a random access read into our view of the content. May block for network access.

Guarantees that err != nil if n < len(buf)

func (*ReadProxy) Size

func (rp *ReadProxy) Size() (size int64)

Return the size of the object generation in bytes.

func (*ReadProxy) Upgrade

func (rp *ReadProxy) Upgrade(
	ctx context.Context) (rwl lease.ReadWriteLease, err error)

Return a read/write lease for the contents of the object. This implicitly destroys the proxy, which must not be used further.

type StatResult

type StatResult struct {
	// The current size in bytes of the content, including any local
	// modifications that have not been Sync'd.
	Size int64

	// The time at which the contents were last updated, or the creation time of
	// the source object if they never have been.
	Mtime time.Time

	// Has the object changed out from under us in GCS? If so, Sync will fail.
	Clobbered bool
}