downloaders

Documentation

Index

• func FTPDownload(logger lager.Logger, config gonfig.Gonfig, dbInstance db.Storage, jobID string) error
• func HTTPDownload(logger lager.Logger, config gonfig.Gonfig, dbInstance db.Storage, jobID string) error
• func S3Download(logger lager.Logger, config gonfig.Gonfig, dbInstance db.Storage, jobID string) error
• type DownloadFunc
  • func GetDownloadFunc(jobSource string) DownloadFunc

Functions

func FTPDownload

func FTPDownload(logger lager.Logger, config gonfig.Gonfig, dbInstance db.Storage, jobID string) error

FTPDownload downloads the file from FTP. Job Source should be in format: ftp://login:password@host/path

func HTTPDownload

func HTTPDownload(logger lager.Logger, config gonfig.Gonfig, dbInstance db.Storage, jobID string) error

HTTPDownload function downloads sources using http protocol.

func S3Download

func S3Download(logger lager.Logger, config gonfig.Gonfig, dbInstance db.Storage, jobID string) error

S3Download downloads the file from S3 bucket. Job Source should be in format: http://AWSKEY:AWSSECRET@BUCKET.s3.amazonaws.com/OBJECT

Types

type DownloadFunc

type DownloadFunc func(logger lager.Logger, config gonfig.Gonfig, dbInstance db.Storage, jobID string) error

DownloadFunc is a function type for the multiple possible ways to download the source file

func GetDownloadFunc

func GetDownloadFunc(jobSource string) DownloadFunc

GetDownloadFunc returns the download function based on the job source.