upload

Documentation

Index

• func DetectEmptyRanges(diskStream *diskstream.DiskStream, uploadableRanges []*common.IndexRange) ([]*common.IndexRange, error)
• func GetDataWithRanges(stream *diskstream.DiskStream, ranges []*common.IndexRange) (<-chan *DataWithRange, <-chan error)
• func LocateNonEmptyRangeIndices(stream *diskstream.DiskStream, ranges []*common.IndexRange) (<-chan int32, <-chan error)
• func LocateUploadableRanges(stream *diskstream.DiskStream, rangesToSkip []*common.IndexRange, ...) ([]*common.IndexRange, error)
• func Upload(cxt *DiskUploadContext) error
• type DataWithRange
• type DiskUploadContext

Functions

func DetectEmptyRanges

func DetectEmptyRanges(diskStream *diskstream.DiskStream, uploadableRanges []*common.IndexRange) ([]*common.IndexRange, error)

DetectEmptyRanges read the ranges identified by the parameter uploadableRanges from the disk stream, detect the empty ranges and update the uploadableRanges slice by removing the empty ranges. This method returns the updated ranges. The empty range detection required only for Fixed disk, if the stream is a expandable disk stream this method simply returns the parameter uploadableRanges as it is.

func GetDataWithRanges

func GetDataWithRanges(stream *diskstream.DiskStream, ranges []*common.IndexRange) (<-chan *DataWithRange, <-chan error)

GetDataWithRanges with start reading and streaming the ranges from the disk identified by the parameter ranges. It returns two channels, a data channel to stream the disk ranges and a channel to send any error while reading the disk. On successful completion the data channel will be closed. the caller must not expect any more value in the data channel if the error channel is signaled.

func LocateNonEmptyRangeIndices

func LocateNonEmptyRangeIndices(stream *diskstream.DiskStream, ranges []*common.IndexRange) (<-chan int32, <-chan error)

LocateNonEmptyRangeIndices scan the given range and detects a subset of ranges which contains data. It reports the indices of non-empty ranges via a channel. This method returns two channels, an int channel - used to report the non-empty range indices and error channel - used to report any error while performing empty detection. int channel will be closed on a successful completion, the caller must not expect any more value in the int channel if the error channel is signaled.

func LocateUploadableRanges

func LocateUploadableRanges(stream *diskstream.DiskStream, rangesToSkip []*common.IndexRange, pageSizeInBytes int64) ([]*common.IndexRange, error)

LocateUploadableRanges detects the uploadable ranges in a VHD stream, size of each range is at most pageSizeInBytes.

This method reads the existing ranges A from the disk stream, creates a new set of ranges B from A by removing the ranges identified by the parameter rangesToSkip, returns new set of ranges C (with each range of size at most pageSizeInBytes) by merging adjacent ranges in B or splitting ranges in B.

Note that this method will not check whether ranges of a fixed disk contains zeros, hence inorder to filter out such ranges from the uploadable ranges, caller must use LocateNonEmptyRangeIndices method.

func Upload

func Upload(cxt *DiskUploadContext) error

Upload uploads the disk ranges described by the parameter cxt, this parameter describes the disk stream to read from, the ranges of the stream to read, the destination blob and it's container, the client to communicate with Azure storage and the number of parallel go-routines to use for upload.

Types

type DataWithRange

type DataWithRange struct {
	Range *common.IndexRange
	Data  []byte
}

DataWithRange type describes a range and data associated with the range.

type DiskUploadContext

type DiskUploadContext struct {
	VhdStream             *diskstream.DiskStream    // The stream whose ranges needs to be uploaded
	AlreadyProcessedBytes int64                     // The size in bytes already uploaded
	UploadableRanges      []*common.IndexRange      // The subset of stream ranges to be uploaded
	BlobServiceClient     storage.BlobStorageClient // The client to make Azure blob service API calls
	ContainerName         string                    // The container in which page blob resides
	BlobName              string                    // The destination page blob name
	Parallelism           int                       // The number of concurrent goroutines to be used for upload
	Resume                bool                      // Indicate whether this is a new or resuming upload
	MD5Hash               []byte                    // MD5Hash to be set in the page blob properties once upload finishes
}

DiskUploadContext type describes VHD upload context, this includes the disk stream to read from, the ranges of the stream to read, the destination blob and it's container, the client to communicate with Azure storage and the number of parallel go-routines to use for upload.