xfer

Documentation

Index

• func DecorateWithDeltaPatcher(layerData io.ReadCloser, deltaBase io.ReadSeeker) io.ReadCloser
• func IsDoNotRetryError(err error) bool
• func WithMaxDownloadAttempts(max int) func(*LayerDownloadManager)
• func WithMaxUploadAttempts(max int) func(*LayerUploadManager)
• type DoFunc
• type DoNotRetry
  • func (e DoNotRetry) Error() string
• type DownloadDescriptor
• type DownloadDescriptorWithRegistered
• type LayerDownloadManager
  • func NewLayerDownloadManager(layerStores map[string]layer.Store, concurrencyLimit int, ...) *LayerDownloadManager
  • func (ldm *LayerDownloadManager) Download(ctx context.Context, initialRootFS image.RootFS, os string, ...) (image.RootFS, func(), error)
  • func (ldm *LayerDownloadManager) SetConcurrency(concurrency int)
• type LayerUploadManager
  • func NewLayerUploadManager(concurrencyLimit int, options ...func(*LayerUploadManager)) *LayerUploadManager
  • func (lum *LayerUploadManager) SetConcurrency(concurrency int)
  • func (lum *LayerUploadManager) Upload(ctx context.Context, layers []UploadDescriptor, progressOutput progress.Output) error
• type Transfer
  • func NewTransfer() Transfer
• type TransferManager
  • func NewTransferManager(concurrencyLimit int) TransferManager
• type UploadDescriptor
• type Watcher

Functions

func DecorateWithDeltaPatcher

func DecorateWithDeltaPatcher(layerData io.ReadCloser, deltaBase io.ReadSeeker) io.ReadCloser

DecorateWithDeltaPatcher returns an io.ReadCloser that applies a delta. The delta itself is read from layerData. If deltaBase is nil, this returns layerData itself -- in other words, this transparently handles the case of being passed non-delta data.

Errors while applying the delta are reported when reading the returned io.ReadCloser.

func IsDoNotRetryError

func IsDoNotRetryError(err error) bool

IsDoNotRetryError returns true if the error is caused by DoNotRetry error, and the transfer should not be retried.

func WithMaxDownloadAttempts

func WithMaxDownloadAttempts(max int) func(*LayerDownloadManager)

WithMaxDownloadAttempts configures the maximum number of download attempts for a download manager.

func WithMaxUploadAttempts

func WithMaxUploadAttempts(max int) func(*LayerUploadManager)

WithMaxUploadAttempts configures the maximum number of upload attempts for a upload manager.

Types

type DoFunc

type DoFunc func(progressChan chan<- progress.Progress, start <-chan struct{}, inactive chan<- struct{}) Transfer

DoFunc is a function called by the transfer manager to actually perform a transfer. It should be non-blocking. It should wait until the start channel is closed before transferring any data. If the function closes inactive, that signals to the transfer manager that the job is no longer actively moving data - for example, it may be waiting for a dependent transfer to finish. This prevents it from taking up a slot.

type DoNotRetry

type DoNotRetry struct {
	Err error
}

DoNotRetry is an error wrapper indicating that the error cannot be resolved with a retry.

func (DoNotRetry) Error

func (e DoNotRetry) Error() string

Error returns the stringified representation of the encapsulated error.

type DownloadDescriptor

type DownloadDescriptor interface {
	// Key returns the key used to deduplicate downloads.
	Key() string
	// ID returns the ID for display purposes.
	ID() string
	// DiffID should return the DiffID for this layer, or an error
	// if it is unknown (for example, if it has not been downloaded
	// before).
	DiffID() (layer.DiffID, error)
	Size() int64
	// Download is called to perform the download.
	Download(ctx context.Context, progressOutput progress.Output) (io.ReadCloser, int64, error)
	// Return the DeltaBase if any
	DeltaBase() io.ReadSeeker
	// Close is called when the download manager is finished with this
	// descriptor and will not call Download again or read from the reader
	// that Download returned.
	Close()
}

A DownloadDescriptor references a layer that may need to be downloaded.

type DownloadDescriptorWithRegistered

type DownloadDescriptorWithRegistered interface {
	DownloadDescriptor
	Registered(diffID layer.DiffID)
}

DownloadDescriptorWithRegistered is a DownloadDescriptor that has an additional Registered method which gets called after a downloaded layer is registered. This allows the user of the download manager to know the DiffID of each registered layer. This method is called if a cast to DownloadDescriptorWithRegistered is successful.

type LayerDownloadManager

type LayerDownloadManager struct {
	// contains filtered or unexported fields
}

LayerDownloadManager figures out which layers need to be downloaded, then registers and downloads those, taking into account dependencies between layers.

func NewLayerDownloadManager

func NewLayerDownloadManager(layerStores map[string]layer.Store, concurrencyLimit int, options ...func(*LayerDownloadManager)) *LayerDownloadManager

NewLayerDownloadManager returns a new LayerDownloadManager.

func (*LayerDownloadManager) Download

func (ldm *LayerDownloadManager) Download(ctx context.Context, initialRootFS image.RootFS, os string, layers []DownloadDescriptor, progressOutput progress.Output) (image.RootFS, func(), error)

Download is a blocking function which ensures the requested layers are present in the layer store. It uses the string returned by the Key method to deduplicate downloads. If a given layer is not already known to present in the layer store, and the key is not used by an in-progress download, the Download method is called to get the layer tar data. Layers are then registered in the appropriate order. The caller must call the returned release function once it is done with the returned RootFS object.

func (*LayerDownloadManager) SetConcurrency

func (ldm *LayerDownloadManager) SetConcurrency(concurrency int)

SetConcurrency sets the max concurrent downloads for each pull

type LayerUploadManager

type LayerUploadManager struct {
	// contains filtered or unexported fields
}

LayerUploadManager provides task management and progress reporting for uploads.

func NewLayerUploadManager

func NewLayerUploadManager(concurrencyLimit int, options ...func(*LayerUploadManager)) *LayerUploadManager

NewLayerUploadManager returns a new LayerUploadManager.

func (*LayerUploadManager) SetConcurrency

func (lum *LayerUploadManager) SetConcurrency(concurrency int)

SetConcurrency sets the max concurrent uploads for each push

func (*LayerUploadManager) Upload

func (lum *LayerUploadManager) Upload(ctx context.Context, layers []UploadDescriptor, progressOutput progress.Output) error

Upload is a blocking function which ensures the listed layers are present on the remote registry. It uses the string returned by the Key method to deduplicate uploads.

type Transfer

type Transfer interface {
	Watch(progressOutput progress.Output) *Watcher
	Release(*Watcher)
	Context() context.Context
	Close()
	Done() <-chan struct{}
	Released() <-chan struct{}
	Broadcast(mainProgressChan <-chan progress.Progress)
}

Transfer represents an in-progress transfer.

func NewTransfer

func NewTransfer() Transfer

NewTransfer creates a new transfer.

type TransferManager

type TransferManager interface {
	// Transfer checks if a transfer with the given key is in progress. If
	// so, it returns progress and error output from that transfer.
	// Otherwise, it will call xferFunc to initiate the transfer.
	Transfer(key string, xferFunc DoFunc, progressOutput progress.Output) (Transfer, *Watcher)
	// SetConcurrency set the concurrencyLimit so that it is adjustable daemon reload
	SetConcurrency(concurrency int)
}

TransferManager is used by LayerDownloadManager and LayerUploadManager to schedule and deduplicate transfers. It is up to the TransferManager implementation to make the scheduling and concurrency decisions.

func NewTransferManager

func NewTransferManager(concurrencyLimit int) TransferManager

NewTransferManager returns a new TransferManager.

type UploadDescriptor

type UploadDescriptor interface {
	// Key returns the key used to deduplicate uploads.
	Key() string
	// ID returns the ID for display purposes.
	ID() string
	// DiffID should return the DiffID for this layer.
	DiffID() layer.DiffID
	// Upload is called to perform the Upload.
	Upload(ctx context.Context, progressOutput progress.Output) (distribution.Descriptor, error)
	// SetRemoteDescriptor provides the distribution.Descriptor that was
	// returned by Upload. This descriptor is not to be confused with
	// the UploadDescriptor interface, which is used for internally
	// identifying layers that are being uploaded.
	SetRemoteDescriptor(descriptor distribution.Descriptor)
}

An UploadDescriptor references a layer that may need to be uploaded.

type Watcher

type Watcher struct {
	// contains filtered or unexported fields
}

Watcher is returned by Watch and can be passed to Release to stop watching.