gcsx

Documentation

Index

• type RandomReader
  • func NewRandomReader(o *gcs.Object, bucket gcs.Bucket) (rr RandomReader, err error)
• type StatResult
• type Syncer
  • func NewSyncer(appendThreshold int64, tmpObjectPrefix string, bucket gcs.Bucket) (os Syncer)
• type TempFile
  • func NewTempFile(content io.Reader, dir string, clock timeutil.Clock) (tf TempFile, err error)

Types

type RandomReader

type RandomReader interface {
	// Panic if any internal invariants are violated.
	CheckInvariants()

	// Matches the semantics of io.ReaderAt, with the addition of context
	// support.
	ReadAt(ctx context.Context, p []byte, offset int64) (n int, err error)

	// Return the record for the object to which the reader is bound.
	Object() (o *gcs.Object)

	// Clean up any resources associated with the reader, which must not be used
	// again.
	Destroy()
}

An object that knows how to read ranges within a particular generation of a particular GCS object. May make optimizations when it e.g. detects large sequential reads.

Not safe for concurrent access.

func NewRandomReader

func NewRandomReader(
	o *gcs.Object,
	bucket gcs.Bucket) (rr RandomReader, err error)

Create a random reader for the supplied object record that reads using the given bucket.

type StatResult

type StatResult struct {
	// The current size in bytes of the content.
	Size int64

	// It is guaranteed that all bytes in the range [0, DirtyThreshold) are
	// unmodified from the original content with which the mutable content object
	// was created.
	DirtyThreshold int64

	// The time at which the content was last updated, or nil if we've never
	// changed it.
	Mtime *time.Time
}

type Syncer

type Syncer interface {
	// Given an object record and content that was originally derived from that
	// object's contents (and potentially modified):
	//
	// *   If the temp file has not been modified, return a nil new object.
	//
	// *   Otherwise, write out a new generation in the bucket (failing with
	//     *gcs.PreconditionError if the source generation is no longer current).
	//
	// In the second case, the TempFile is destroyed. Otherwise, including when
	// this function fails, it is guaranteed to still be valid.
	SyncObject(
		ctx context.Context,
		srcObject *gcs.Object,
		content TempFile) (o *gcs.Object, err error)
}

Safe for concurrent access.

func NewSyncer

func NewSyncer(
	appendThreshold int64,
	tmpObjectPrefix string,
	bucket gcs.Bucket) (os Syncer)

Create a syncer that syncs into the supplied bucket.

When the source object has been changed only by appending, and the source object's size is at least appendThreshold, we will "append" to it by writing out a temporary blob and composing it with the source object.

Temporary blobs have names beginning with tmpObjectPrefix. We make an effort to delete them, but if we are interrupted for some reason we may not be able to do so. Therefore the user should arrange for garbage collection.

type TempFile

type TempFile interface {
	// Panic if any internal invariants are violated.
	CheckInvariants()

	// Semantics matching os.File.
	io.ReadSeeker
	io.ReaderAt
	io.WriterAt
	Truncate(n int64) (err error)

	// Return information about the current state of the content. May invalidate
	// the seek position.
	Stat() (sr StatResult, err error)

	// Throw away the resources used by the temporary file. The object must not
	// be used again.
	Destroy()
}

A temporary file that keeps track of the lowest offset at which it has been modified.

Not safe for concurrent access.

func NewTempFile

func NewTempFile(
	content io.Reader,
	dir string,
	clock timeutil.Clock) (tf TempFile, err error)

Create a temp file whose initial contents are given by the supplied reader. dir is a directory on whose file system the inode will live, or the system default temporary location if empty.