downloader

Documentation

Index

• func DownloadJob(ctx context.Context, outputVolumes []model.StorageSpec, ...) error
• type Downloader
• type DownloaderProvider

Functions

func DownloadJob

func DownloadJob(
	ctx context.Context,

	outputVolumes []model.StorageSpec,

	publishedShardResults []model.PublishedResult,
	downloadProvider DownloaderProvider,
	settings *model.DownloaderSettings,
) error

* make a temp dir * download all cids into temp dir * ensure top level output dir exists * iterate over each shard * make new folder for shard logs * copy stdout, stderr, exitCode * append stdout, stderr to global log * iterate over each output volume * make new folder for output volume * iterate over each shard and merge files in output folder to results dir

Types

type Downloader

type Downloader interface {
	model.Providable

	// FetchResult fetches published result and saves it to downloadPath
	FetchResult(ctx context.Context, result model.PublishedResult, downloadPath string) error
}

type DownloaderProvider

type DownloaderProvider interface {
	model.Provider[model.StorageSourceType, Downloader]
}