downloader

Documentation

Index

• type Downloader
  • func New(mods ...Modifier) (*Downloader, error)
  • func (d *Downloader) Get(url string, handler protocol.ResponseHandler, eh workpool.ErrorHandler, ...) error
  • func (o *Downloader) ProtocolModifiers() []protocol.Modifier
  • func (o *Downloader) Retrier() *retry.Options
  • func (d *Downloader) Wait()
• type Flags
  • func DefaultFlags() *Flags
  • func (fl *Flags) Register(set kflags.FlagSet, prefix string) *Flags
• type Modifier
  • func FromFlags(fl *Flags) Modifier
  • func WithClientOptions(mods ...kclient.Modifier) Modifier
  • func WithContext(ctx context.Context) Modifier
  • func WithProtocolOptions(mods ...protocol.Modifier) Modifier
  • func WithRequestOptions(mods ...krequest.Modifier) Modifier
  • func WithRetryOptions(mods ...retry.Modifier) Modifier
  • func WithSchedulerOptions(mods ...scheduler.Modifier) Modifier
  • func WithTimeout(timeout time.Duration) Modifier
  • func WithWaitGroup(wg *sync.WaitGroup) Modifier
  • func WithWorkpoolOptions(mods ...workpool.Modifier) Modifier
• type Modifiers
  • func (mods Modifiers) Apply(o *options) error

Types

type Downloader

type Downloader struct {
	// contains filtered or unexported fields
}

func New

func New(mods ...Modifier) (*Downloader, error)

func (*Downloader) Get

func (d *Downloader) Get(url string, handler protocol.ResponseHandler, eh workpool.ErrorHandler, mods ...Modifier) error

Get will fetch the specified url, invoke handler to process the response, and eh to process the returned error.

Get will schedule the operation through a workpool.

The operation will fail if something goes wrong with the HTTP request, or if the handler returns error. Regardless, the error handler is invoked with the result of the Get (which could be nil, to indicate success).

When combining with WithRetry options, if the handler or get return error, the operation will be retried.

Get returns an error. But given that downloads are scheduled asynchronously, the only case when Get will return an error is if an invalid combination of flags or options was specified.

func (*Downloader) ProtocolModifiers

func (o *Downloader) ProtocolModifiers() []protocol.Modifier

func (*Downloader) Retrier

func (o *Downloader) Retrier() *retry.Options

Retrier returns a retrier with the same options that would be used by the downloader.

func (*Downloader) Wait

func (d *Downloader) Wait()

type Flags

type Flags struct {
	Timeout  time.Duration
	Retry    *retry.Flags
	Workpool *workpool.Flags
	Client   *kclient.Flags
}

func DefaultFlags

func DefaultFlags() *Flags

func (*Flags) Register

func (fl *Flags) Register(set kflags.FlagSet, prefix string) *Flags

type Modifier

type Modifier func(*options) error

func FromFlags

func FromFlags(fl *Flags) Modifier

func WithClientOptions

func WithClientOptions(mods ...kclient.Modifier) Modifier

func WithContext

func WithContext(ctx context.Context) Modifier

func WithProtocolOptions

func WithProtocolOptions(mods ...protocol.Modifier) Modifier

func WithRequestOptions

func WithRequestOptions(mods ...krequest.Modifier) Modifier

func WithRetryOptions

func WithRetryOptions(mods ...retry.Modifier) Modifier

func WithSchedulerOptions

func WithSchedulerOptions(mods ...scheduler.Modifier) Modifier

func WithTimeout

func WithTimeout(timeout time.Duration) Modifier

func WithWaitGroup

func WithWaitGroup(wg *sync.WaitGroup) Modifier

func WithWorkpoolOptions

func WithWorkpoolOptions(mods ...workpool.Modifier) Modifier

type Modifiers

type Modifiers []Modifier

func (Modifiers) Apply

func (mods Modifiers) Apply(o *options) error