downloader

Documentation

Index

• Constants
• func DownloadResults(ctx context.Context, publishedResults []*models.SpecConfig, ...) error
• func IsAlreadyDownloaded(target string) (bool, error)
• type DownloadItem
• type Downloader
• type DownloaderProvider
• type DownloaderSettings

Constants

const (
	DownloadFilenameStdout   = "stdout"
	DownloadFilenameStderr   = "stderr"
	DownloadFilenameExitCode = "exitCode"
	DownloadRawFolderName    = "raw"
	DownloadFolderPerm       = 0755
	DownloadFilePerm         = 0644
	DefaultDownloadTimeout   = 5 * time.Minute
)

Functions

func DownloadResults

func DownloadResults(
	ctx context.Context,
	publishedResults []*models.SpecConfig,
	downloadProvider DownloaderProvider,
	settings *DownloaderSettings,
) error

DownloadResults downloads published results from a storage source and saves them to the specific download path. It supports downloading multiple results from different jobs and will append the logs to the global log file. This behavior is left from when we supported sharded jobs, and multiple results per job. It is not currently being user, and we can evaluate removing it in the future if we don't expose merging results from multiple jobs.

* make a temp dir * download all results into temp dir * ensure top level output dir exists * iterate over each published result * copy stdout, stderr, exitCode * append stdout, stderr to global log * iterate over each output volume * make new folder for output volume * iterate over each result and merge files in output folder to results dir

func IsAlreadyDownloaded

func IsAlreadyDownloaded(target string) (bool, error)

IsAlreadyDownloaded checks if the target path exists.

Types

type DownloadItem

type DownloadItem struct {
	Result     *models.SpecConfig
	SingleFile string
	ParentPath string
}

type Downloader

type Downloader interface {
	provider.Providable

	// FetchResult fetches item and saves to disk (as per item's Target)
	FetchResult(ctx context.Context, item DownloadItem) (string, error)
}

type DownloaderProvider

type DownloaderProvider interface {
	provider.Provider[Downloader]
}

type DownloaderSettings

type DownloaderSettings struct {
	Timeout    time.Duration
	OutputDir  string
	SingleFile string
	Raw        bool
}