paralleldl

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 24, 2016 License: MIT Imports: 13 Imported by: 1

README

parallel-dl

parallel-dl is a library for downloading files in parallel.

godoc build codecov go report

  • Go 1.7 or newer

Features

parallel-dl allows you to control these features:

  • HTTP Timeouts
  • Maximum number of concurrent requests
  • Maximum number of errors before giving up the whole requests
  • Maximum number of retries before giving up a request

API

  • Download()

That's all !

Examples

opt := &paralleldl.Options{
	Output:           "/path/to/download",
	MaxConcurrents:   2,
	MaxErrorRequests: 1,
	MaxAttempts:      4,
}
client, err := paralleldl.New(opt)
if err != nil {
	log.Fatal(err)
}

lists := []string{
	"http://example.com/file1",
	"http://example.com/file2",
	"http://example.com/file3",
	"http://example.com/file4",
}
errCnt := client.Download(lists)
Minimum Options
opt := &paralleldl.Options{
	Output: "/path/to/download",
}

License


The MIT License

Author

Yoshiki Nakagawa

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func SetHTTPClient

func SetHTTPClient(client *http.Client)

SetHTTPClient overrides the default HTTP client.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client represents http.Client with paralleldl options.

func New

func New(opt *Options) (*Client, error)

New returns a new Client struct.

func (*Client) Download

func (c *Client) Download(list []string) int64

Download downloads the lists resources.

type Options

type Options struct {
	// Output directory
	Output string

	// http.Client's Timeout.
	// A Timeout of zero means no timeout.
	Timeout time.Duration

	// Maximum number of concurrent requests.
	// A MaxConcurrents of zero means no limit of number of requests.
	MaxConcurrents int64
	// Maximum number of errors before giving up the whole requests.
	// A MaxErrorRequests of zero means try until all requests are done.
	MaxErrorRequests int64

	// Maximum number of retries before giving up a request.
	// A MaxAttempts of zero means try until a request is success.
	MaxAttempts int64
}

Options represents Client's parameters.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL