blobref

Documentation

Index

• Variables
• type BlobRef
  • func NewBlobRef(size int64, storeKey, recordKey string) *BlobRef
  • func NewChunkedBlobRef(storeKey, recordKey string, chunkCount int64) *BlobRef
  • func (b *BlobRef) Load(ps []datastore.Property) error
  • func (b *BlobRef) LoadKey(k *datastore.Key) error
  • func (b *BlobRef) ObjectPath() string
  • func (b *BlobRef) Save() ([]datastore.Property, error)
  • func (b *BlobRef) ToProto() *pb.BlobMetadata
• type BlobRefCursor
  • func NewCursor(i *ds.Iterator) *BlobRefCursor
  • func (i *BlobRefCursor) Next() (*BlobRef, error)
• type Status
  • func (s *Status) Fail()
  • func (s *Status) MarkForDeletion() error
  • func (s *Status) Ready() error

Variables

var (
	ErrReadyNotInitializing    = errors.New("Ready was called when Status is not Initializing")
	ErrMarkForDeletionNotReady = errors.New("MarkForDeletion was called when Status is not either Initializing or Ready")
)

var (
	ErrIteratorNil = errors.New("iterator is nil")
)

Types

type BlobRef

type BlobRef struct {
	// Key is the primary key for the blob entry
	Key uuid.UUID `datastore:"-"`
	// Size is the byte size of the blob
	Size int64
	// Status is the current status of the blob
	Status
	// StoreKey is the key of the store that the blob belongs to
	StoreKey string
	// RecordKey is the key of the record that the blob belongs to
	// It can be non-existent (e.g. deleted already) but then the Status
	// should not be Blob StatusReady.
	RecordKey string
	// Chunked is whether the BlobRef is chunked or not.
	Chunked bool
	// ChunkCount is the number of chunks that should be associated to the BlobRef.
	// It is set by either the client when starting a chunk upload or
	// the server when committing a chunked upload.
	ChunkCount int64

	// Checksums have checksums for each blob object associated with the BlobRef entity.
	// Record.{MD5,CRC32C} must be used for inline blobs, and
	// ChunkRef.{MD5,CRC32C} must be used for chunked blobs.
	checksums.Checksums `datastore:",flatten"`

	// Timestamps keeps track of creation and modification times and stores a randomly
	// generated UUID to maintain consistency.
	Timestamps timestamps.Timestamps
}

BlobRef is a metadata document to keep track of blobs stored in an external blob store.

func NewBlobRef

func NewBlobRef(size int64, storeKey, recordKey string) *BlobRef

NewBlobRef creates a new BlobRef as follows:

• Set a new UUID to Key
• Initialize Size and ObjectName as specified
• Set Status to BlobRefStatusInitializing
• Set current time to Timestamps (both created and updated at)

func NewChunkedBlobRef

func NewChunkedBlobRef(storeKey, recordKey string, chunkCount int64) *BlobRef

NewChunkedBlobRef creates a new BlobRef object with Size, Chunked, and ChunkCount set to 0, true, and chunkCount respectively. Other behaviors are the same as NewBlobRef

func (*BlobRef) Load

func (b *BlobRef) Load(ps []datastore.Property) error

Load implements the Datastore PropertyLoadSaver interface and converts Datstore properties to the Properties field.

func (*BlobRef) LoadKey

func (b *BlobRef) LoadKey(k *datastore.Key) error

LoadKey implements the KeyLoader interface and sets the value to the Key field.

func (*BlobRef) ObjectPath

func (b *BlobRef) ObjectPath() string

ObjectPath returns an object path for the backend blob storage.

func (*BlobRef) Save

func (b *BlobRef) Save() ([]datastore.Property, error)

Save implements the Datastore PropertyLoadSaver interface and converts the properties field in the struct to separate Datastore properties.

func (*BlobRef) ToProto

func (b *BlobRef) ToProto() *pb.BlobMetadata

ToProto returns a BlobMetadata representation of the object.

type BlobRefCursor

type BlobRefCursor struct {
	// contains filtered or unexported fields
}

BlobRefCursor is a database cursor for BlobRef.

func NewCursor

func NewCursor(i *ds.Iterator) *BlobRefCursor

func (*BlobRefCursor) Next

func (i *BlobRefCursor) Next() (*BlobRef, error)

Next advances the iterator and returns the next value. Returns nil and and iterator.Done at the end of the iterator. Returns ErrIteratorNil if the iterator is nil.

type Status

type Status int16

Status represents a current blob status.

Life of a Blob

[New Record created] --> [new Status entity with StatusInitializing]

                       /               \
                      / fail            \  success
                     v                   v
              [StatusError]            [StatusReady]
                     |       x            |
    Upload new blob  |        \ fail      | Record deleted or new blob uploaded
          or         |         \          v
   delete the record |          -------[StatusPendingDeletion]
                     v                  /
[Delete the blob entity] <-------------/   Garbage collection

const (
	// StatusUnknown represents internal error.
	StatusUnknown Status = iota
	// StatusInitializing means the blob is currently being prepared, i.e.
	// being uploaded to the blob store.
	StatusInitializing
	// StatusReady means the blob is committed and ready for use.
	StatusReady
	// StatusPendingDeletion means the blob is no longer referenced by
	// any Record entities and needs to be deleted.
	StatusPendingDeletion
	// StatusError means the blob was not uploaded due to client or server
	// errors and the corresponding record needs to be updated (either by
	// retrying blob upload or deleting the entry).
	StatusError
)

func (*Status) Fail

func (s *Status) Fail()

Fail marks the Status as StatusError. Any state can transition to StatusError.

func (*Status) MarkForDeletion

func (s *Status) MarkForDeletion() error

MarkForDeletion marks the Status as StatusPendingDeletion. Returns an error if the current Status is not StatusInitializing or StatusReady.

func (*Status) Ready

func (s *Status) Ready() error

Ready changes Status to StatusReady. It returns an error if the current Status is not StatusInitializing.