Documentation ¶
Index ¶
- Variables
- func NewHash() hash.Hash
- type Backend
- type Part
- type PartInfo
- type ReadSeekCloser
- type Refs
- type Store
- func (s *Store) FinishUpload(uploadId string, parts []Part) (idx *mongodoc.MultipartIndex, hash string, err error)
- func (s *Store) GC(refs *Refs, before time.Time) (monitoring.BlobStats, error)
- func (s *Store) NewUpload(expires time.Time) (uploadId string, err error)
- func (s *Store) Open(hash string, index *mongodoc.MultipartIndex) (ReadSeekCloser, int64, error)
- func (s *Store) Put(r io.Reader, hash string, size int64) error
- func (s *Store) PutAtTime(r io.Reader, hash string, size int64, now time.Time) error
- func (s *Store) PutPart(uploadId string, part int, r io.Reader, size, offset int64, hash string) error
- func (s *Store) RemoveExpiredUploads() error
- func (s *Store) RemoveUpload(uploadId string) error
- func (s *Store) SetOwner(uploadId, owner string, expires time.Time) error
- func (s *Store) UploadInfo(uploadId string) (UploadInfo, error)
- type UploadInfo
Constants ¶
This section is empty.
Variables ¶
var ErrBadParams = errgo.New("bad parameters")
var ErrNotFound = errgo.New("blob not found")
Functions ¶
Types ¶
type Backend ¶
type Backend interface { // Get gets a reader for the object with the given name // and its size. The returned reader should be closed after use. // If the object doesn't exist, an error with an ErrNotFound // cause should be returned. // If the object is removed while reading, the read // error's cause should be ErrNotFound. Get(name string) (r ReadSeekCloser, size int64, err error) // Put puts an object by reading its data from the given reader. // The data read from the reader should have the given size and // hex-encoded SHA384 hash. If it is not possible to read size // bytes from r then an error with a cause of io.ErrUnexpectedEOF // will be returned. Put(name string, r io.Reader, size int64, hash string) error // Remove removes the object with the given name. Remove(name string) error }
Backend represents the underlying data store used by blobstore.Store to store blob data.
func NewMongoBackend ¶
NewMongoBackend returns a backend implementation which stores data in the given MongoDB database, using prefix as a prefix for the collections created.
func NewSwiftBackend ¶
func NewSwiftBackend(cred *identity.Credentials, authmode identity.AuthMode, container, tmpdir string) Backend
NewSwiftBackend returns a backend which uses OpenStack's Swift for its operations with the given credentials and auth mode. It stores all the data objects in the container with the given name.
type PartInfo ¶
type PartInfo struct { // Hash holds the SHA384 hash of the part. Hash string // Size holds the size of the part. Size int64 // Complete holds whether the part has been // successfully uploaded. Complete bool // Offset holds the offset relative to the start of the upload. Offset int64 }
PartInfo holds information about one part of a multipart upload.
type Refs ¶
type Refs struct {
// contains filtered or unexported fields
}
Refs holds information about the existence of a set of blob hashes.
type Store ¶
type Store struct { // MinPartSize holds the minimum size of a multipart upload part. MinPartSize int64 // MaxPartSize holds the maximum size of a multipart upload part. MaxPartSize int64 // MaxParts holds the maximum number of parts that there // can be in a multipart upload. MaxParts int // contains filtered or unexported fields }
Store stores data blobs in mongodb, de-duplicating by blob hash.
func New ¶
New returns a new blob store that writes to the given database, prefixing its collections with the given prefix.
func (*Store) FinishUpload ¶
func (s *Store) FinishUpload(uploadId string, parts []Part) (idx *mongodoc.MultipartIndex, hash string, err error)
FinishUpload completes a multipart upload by joining all the given parts into one blob. The resulting blob can be opened by passing uploadId and the returned multipart index to Open.
The part numbers used will be from 0 to len(parts)-1.
This does not delete the multipart metadata, which should still be deleted explicitly by calling RemoveUpload after the index data is stored.
func (*Store) GC ¶
GC runs the garbage collector, deleting all blobs not present in refs that have not been Put since the given time. Note that it also adds any internal blobs held by in-progress uploads to refs.
func (*Store) NewUpload ¶
NewUpload created a new multipart entry to track a multipart upload. It returns an uploadId that can be used to refer to it. After creating the upload, each part must be uploaded individually, and then the whole completed by calling FinishUpload and RemoveUpload.
func (*Store) Open ¶
func (s *Store) Open(hash string, index *mongodoc.MultipartIndex) (ReadSeekCloser, int64, error)
Open opens the entry with the given hash. It returns an error with an ErrNotFound cause if the entry does not exist.
func (*Store) Put ¶
Put streams the content with the given hex-encoded SHA384 hash, and size from the given reader into blob storage.
func (*Store) PutAtTime ¶
PutAtTime is like Put but puts the content as if the current time is now. This should be used for testing purposes only.
func (*Store) PutPart ¶
func (s *Store) PutPart(uploadId string, part int, r io.Reader, size, offset int64, hash string) error
PutPart uploads a part to the given upload id. The part number is specified with the part parameter; its content will be read from r and is expected to have the given size and hex-encoded SHA384 hash. A given part may not be uploaded more than once with a different hash.
If the upload id was not found (for example, because it's expired), PutPart returns an error with an ErrNotFound cause. If one of the parameters is badly formed, it returns an error with an ErrBadParams cause.
func (*Store) RemoveExpiredUploads ¶
RemoveExpiredUploads deletes any multipart entries that have passed their expiry date.
func (*Store) RemoveUpload ¶
RemoveUpload deletes all the parts associated with the given upload id. It is a no-op if called twice on the same upload id. If the upload has an owner and isOwnedBy is not nil, isOwnedBy will be called with uploadId and the upload's current owner to determine be sure that it is not actually used. If isOwnedBy is nil, it is assumed to return true - i.e. the document is owned.
func (*Store) SetOwner ¶
SetOwner sets the "owner" of a given upload. This should be set just before the owner document is updated to refer to the upload. SetOwner will fail if the upload is already owned. The upload expiry deadline is also set to expires, so the caller has that much leeway to associate the upload id with some owner document.
func (*Store) UploadInfo ¶
func (s *Store) UploadInfo(uploadId string) (UploadInfo, error)
UploadInfo returns information on a given upload. It returns ErrNotFound if the upload has been deleted.
type UploadInfo ¶
type UploadInfo struct { // Parts holds all the known parts of the upload. // Parts that haven't been uploaded yet will have nil // elements. Parts that are in progress or have been // aborted will have false Complete fields. Parts []*PartInfo // Expires holds when the upload will expire. Expires time.Time // Hash holds the hash of the entire upload. // This will be empty until the upload has // been completed with FinishUpload. Hash string `bson:"hash,omitempty"` }
UploadInfo holds information on a given upload.
func (*UploadInfo) Index ¶
func (info *UploadInfo) Index() (*mongodoc.MultipartIndex, bool)
Index returns a multipart index suitable for opening the multipart blob with the given info. It returns false if the info does not represent a completed upload.