targets

package
v0.6.20 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 26, 2019 License: Apache-2.0, BSD-3-Clause, MIT Imports: 8 Imported by: 1

Documentation

Index

Constants

View Source 
const PageSize uint64 = 512

PageSize page size for page blobs

Variables

This section is empty.

Functions

func NewAzureBlockTargetPipeline added in v0.6.12 

func NewAzureBlockTargetPipeline(params AzureTargetParams) pipeline.TargetPipeline

NewAzureBlockTargetPipeline TODO

func NewAzurePageTargetPipeline added in v0.6.12 

func NewAzurePageTargetPipeline(params AzureTargetParams) pipeline.TargetPipeline

NewAzurePageTargetPipeline creates a new Azure Block target

func NewFileSystemTargetPipeline added in v0.6.12 

func NewFileSystemTargetPipeline(overwrite bool, numberOfHandles int) pipeline.TargetPipeline

NewFileSystemTargetPipeline creates a new multi file target and 'n' number of handles for concurrent writes to a file.

func NewPerfTargetPipeline added in v0.6.12 

func NewPerfTargetPipeline() pipeline.TargetPipeline

NewPerfTargetPipeline creates a new passthrough target used to measure the performance of the source

Types

type AzureBlockTarget added in v0.6.12 

type AzureBlockTarget struct {
	// contains filtered or unexported fields
}

AzureBlockTarget represents an Azure Block target

func (*AzureBlockTarget) CommitList added in v0.6.12 

func (t *AzureBlockTarget) CommitList(listInfo *pipeline.TargetCommittedListInfo, numberOfBlocks int, targetName string) (msg string, err error)

CommitList implements CommitList from the pipeline.TargetPipeline interface. Commits the list of blocks to Azure Storage to finalize the transfer.

func (*AzureBlockTarget) PreProcessSourceInfo added in v0.6.12 

func (t *AzureBlockTarget) PreProcessSourceInfo(source *pipeline.SourceInfo, blockSize uint64) (err error)

PreProcessSourceInfo implementation of PreProcessSourceInfo from the pipeline.TargetPipeline interface. Checks if uncommitted blocks are present and cleans them by creating an empty blob.

func (*AzureBlockTarget) ProcessWrittenPart added in v0.6.12 

func (t *AzureBlockTarget) ProcessWrittenPart(result *pipeline.WorkerResult, listInfo *pipeline.TargetCommittedListInfo) (requeue bool, err error)

ProcessWrittenPart implements ProcessWrittenPart from the pipeline.TargetPipeline interface. Appends the written part to a list. If the part is duplicated the list is updated with a reference, to the first occurrence of the block. If the first occurrence has not yet being processed, the part is requested to be placed back in the results channel (requeue == true).

func (*AzureBlockTarget) WritePart added in v0.6.12 

func (t *AzureBlockTarget) WritePart(part *pipeline.Part) (duration time.Duration, startTime time.Time, numOfRetries int, err error)

WritePart implements WritePart from the pipeline.TargetPipeline interface. Performs a PUT block operation with the data contained in the part.

type AzurePageTarget added in v0.6.12 

type AzurePageTarget struct {
	// contains filtered or unexported fields
}

AzurePageTarget represents an Azure Block target

func (*AzurePageTarget) CommitList added in v0.6.12 

func (t *AzurePageTarget) CommitList(listInfo *pipeline.TargetCommittedListInfo, NumberOfBlocks int, targetName string) (msg string, err error)

CommitList implements CommitList from the pipeline.TargetPipeline interface. Passthrough no need to a commit for page blob.

func (*AzurePageTarget) PreProcessSourceInfo added in v0.6.12 

func (t *AzurePageTarget) PreProcessSourceInfo(source *pipeline.SourceInfo, blockSize uint64) (err error)

PreProcessSourceInfo implementation of PreProcessSourceInfo from the pipeline.TargetPipeline interface. initializes the page blob.

func (*AzurePageTarget) ProcessWrittenPart added in v0.6.12 

func (t *AzurePageTarget) ProcessWrittenPart(result *pipeline.WorkerResult, listInfo *pipeline.TargetCommittedListInfo) (requeue bool, err error)

ProcessWrittenPart implements ProcessWrittenPart from the pipeline.TargetPipeline interface. Passthrough no need to process a written part when transferring to a page blob.

func (*AzurePageTarget) WritePart added in v0.6.12 

func (t *AzurePageTarget) WritePart(part *pipeline.Part) (duration time.Duration, startTime time.Time, numOfRetries int, err error)

WritePart implements WritePart from the pipeline.TargetPipeline interface. Performs a PUT page operation with the data contained in the part. This assumes the part.BytesToRead is a multiple of the PageSize

type AzureTargetParams added in v0.6.12 

type AzureTargetParams struct {
	AccountName   string
	AccountKey    string
	Container     string
	BaseBlobURL   string
	UseServerSide bool
}

AzureTargetParams TODO

type FileSystemTarget added in v0.6.12 

type FileSystemTarget struct {
	// contains filtered or unexported fields
}

FileSystemTarget represents an OS file(s) target

func (*FileSystemTarget) CommitList added in v0.6.12 

func (t *FileSystemTarget) CommitList(listInfo *pipeline.TargetCommittedListInfo, numberOfBlocks int, targetName string) (msg string, err error)

CommitList implements CommitList from the pipeline.TargetPipeline interface. For a file download a final commit is not required and this implementation closes all the filehandles.

func (*FileSystemTarget) PreProcessSourceInfo added in v0.6.12 

func (t *FileSystemTarget) PreProcessSourceInfo(source *pipeline.SourceInfo, blockSize uint64) (err error)

PreProcessSourceInfo implementation of PreProcessSourceInfo from the pipeline.TargetPipeline interface. Passthrough no need to pre-process for a file target.

func (*FileSystemTarget) ProcessWrittenPart added in v0.6.12 

func (t *FileSystemTarget) ProcessWrittenPart(result *pipeline.WorkerResult, listInfo *pipeline.TargetCommittedListInfo) (requeue bool, err error)

ProcessWrittenPart implements ProcessWrittenPart from the pipeline.TargetPipeline interface. Passthrough implementation as no post-written-processing is required (e.g. maintain a list) when files are downloaded.

func (*FileSystemTarget) WritePart added in v0.6.12 

func (t *FileSystemTarget) WritePart(part *pipeline.Part) (duration time.Duration, startTime time.Time, numOfRetries int, err error)

WritePart implements WritePart from the pipeline.TargetPipeline interface. Writes to a local file using a filehandle received from a channel.

type FileTargetParams added in v0.6.12 

type FileTargetParams struct {
	Overwrite              bool
	NumberOfHandlesPerFile int
}

FileTargetParams TODO

type PerfTarget added in v0.6.12 

type PerfTarget struct {
}

PerfTarget TODO

func (*PerfTarget) CommitList added in v0.6.12 

func (t *PerfTarget) CommitList(listInfo *pipeline.TargetCommittedListInfo, NumberOfBlocks int, targetName string) (msg string, err error)

CommitList passthrough, no processing.

func (*PerfTarget) PreProcessSourceInfo added in v0.6.12 

func (t *PerfTarget) PreProcessSourceInfo(source *pipeline.SourceInfo, blockSize uint64) (err error)

PreProcessSourceInfo passthrough, no processing.

func (*PerfTarget) ProcessWrittenPart added in v0.6.12 

func (t *PerfTarget) ProcessWrittenPart(result *pipeline.WorkerResult, listInfo *pipeline.TargetCommittedListInfo) (requeue bool, err error)

ProcessWrittenPart passthrough, no processing.

func (*PerfTarget) WritePart added in v0.6.12 

func (t *PerfTarget) WritePart(part *pipeline.Part) (duration time.Duration, startTime time.Time, numOfRetries int, err error)

WritePart passthrough, no processing.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.