downloader

Documentation

Index

• type CacheDownloadInstruction
• type DownloadedFile
• type Downloader
  • func NewDownloader(downloadInstructs []CacheDownloadInstruction, limit int) *Downloader
  • func (d *Downloader) Download(ctx context.Context) ([]DownloadedFile, error)
• type Offset

Types

type CacheDownloadInstruction

type CacheDownloadInstruction struct {
	// Method HTTP method
	Method string  `json:"method"`
	Offset *Offset `json:"offset,omitempty"`

	// Url URL
	Url string `json:"url"`
}

type DownloadedFile

type DownloadedFile struct {
	URL      string
	FilePath string
	ETag     string
	Part     int
	Size     int64
}

type Downloader

type Downloader struct {
	// contains filtered or unexported fields
}

Downloader uses go routines to download parts of a file in parallel with a limit of 20 concurrent uploads. It is provided a list of URLs to upload and a channel to receive the results.

func NewDownloader

func NewDownloader(downloadInstructs []CacheDownloadInstruction, limit int) *Downloader

func (*Downloader) Download

func (d *Downloader) Download(ctx context.Context) ([]DownloadedFile, error)

type Offset

type Offset struct {
	// End End position of the part
	End int64 `json:"end"`

	// Part Part number
	Part int32 `json:"part"`

	// Start Start position of the part
	Start int64 `json:"start"`
}

Offset defines model for Offset.