curl

package
v7.0.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 31, 2020 License: BSD-3-Clause Imports: 16 Imported by: 0

Documentation

Overview

Package curl implements routines to fetch files given a URL.

curl currently supports HTTP, TFTP, and local files.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrWrongScheme means the wrong mocked scheme was used.
	ErrWrongScheme = errors.New("wrong scheme")
	// ErrNoSuchHost means there is no host record in the mock.
	ErrNoSuchHost = errors.New("no such host exists")
	// ErrNoSuchFile means there is no file record in the mock.
	ErrNoSuchFile = errors.New("no such file exists on this host")
)
View Source 
var (
	// DefaultHTTPClient is the default HTTP FileScheme.
	//
	// It is not recommended to use this for HTTPS. We recommend creating an
	// http.Client that accepts only a private pool of certificates.
	DefaultHTTPClient = NewHTTPClient(http.DefaultClient)

	// DefaultTFTPClient is the default TFTP FileScheme.
	DefaultTFTPClient = NewTFTPClient(tftp.ClientMode(tftp.ModeOctet), tftp.ClientBlocksize(1450), tftp.ClientWindowsize(65535))

	// DefaultSchemes are the schemes supported by default.
	DefaultSchemes = Schemes{
		"tftp": DefaultTFTPClient,
		"http": DefaultHTTPClient,
		"file": &LocalFileClient{},
	}
)
View Source 
var (
	// ErrNoSuchScheme is returned by Schemes.Fetch and
	// Schemes.LazyFetch if there is no registered FileScheme
	// implementation for the given URL scheme.
	ErrNoSuchScheme = errors.New("no such scheme")
)

Functions

func IsURLError 

func IsURLError(err error) bool

IsURLError returns true iff err is a URLError.

func RegisterScheme 

func RegisterScheme(scheme string, fs FileScheme)

RegisterScheme calls DefaultSchemes.Register.

func RetryConnectErrors 

func RetryConnectErrors(u *url.URL, err error) bool

RetryConnectErrors retries only connect(2) errors.

func RetryHTTP 

func RetryHTTP(u *url.URL, err error) bool

RetryHTTP implements DoRetry for HTTP error codes where it makes sense.

func RetryTFTP 

func RetryTFTP(u *url.URL, err error) bool

RetryTFTP retries downloads if the error does not contain FILE_NOT_FOUND.

pack.ag/tftp does not export the necessary structs to get the code out of the error message cleanly, but it does embed FILE_NOT_FOUND in the error string.

func RetryTemporaryNetworkErrors 

func RetryTemporaryNetworkErrors(u *url.URL, err error) bool

RetryTemporaryNetworkErrors only retries temporary network errors.

This relies on Go's net.Error.Temporary definition of temporary network errors, which does not include network configuration errors. The latter are relevant for users of DHCP, for example.

Types

type DoRetry 

type DoRetry func(u *url.URL, err error) bool

DoRetry returns true if the Fetch request for the URL should be retried. err is the error that Fetch previously returned.

DoRetry lets a FileScheme filter for errors returned by Fetch which are worth retrying. If this interface is not implemented, the default for SchemeWithRetries is to always retry. DoRetry returns true to indicate a request should be retried.

func RetryOr 

func RetryOr(fn ...DoRetry) DoRetry

RetryOr returns a DoRetry function that returns true if any one of fn return true.

type File 

type File interface {
	io.ReaderAt

	// URL is the file's original URL.
	URL() *url.URL
}

File is a reference to a file fetched through this library.

func Fetch 

func Fetch(ctx context.Context, u *url.URL) (File, error)

Fetch fetchs a file via DefaultSchemes.

func LazyFetch 

func LazyFetch(u *url.URL) (File, error)

LazyFetch calls LazyFetch on DefaultSchemes.

type FileScheme 

type FileScheme interface {
	// Fetch returns a reader that gives the contents of `u`.
	//
	// It may do so by fetching `u` and placing it in a buffer, or by
	// returning an io.ReaderAt that fetchs the file.
	Fetch(ctx context.Context, u *url.URL) (io.ReaderAt, error)
}

FileScheme represents the implementation of a URL scheme and gives access to fetching files of that scheme.

For example, an http FileScheme implementation would fetch files using the HTTP protocol.

func NewTFTPClient 

func NewTFTPClient(opts ...tftp.ClientOpt) FileScheme

NewTFTPClient returns a new TFTP client based on the given tftp.ClientOpt.

type HTTPClient 

type HTTPClient struct {
	// contains filtered or unexported fields
}

HTTPClient implements FileScheme for HTTP files.

func NewHTTPClient 

func NewHTTPClient(c *http.Client) *HTTPClient

NewHTTPClient returns a new HTTP FileScheme based on the given http.Client.

func (HTTPClient) Fetch 

func (h HTTPClient) Fetch(ctx context.Context, u *url.URL) (io.ReaderAt, error)

Fetch implements FileScheme.Fetch.

type HTTPClientCodeError 

type HTTPClientCodeError struct {
	Err      error
	HTTPCode int
}

HTTPClientCodeError is returned by HTTPClient.Fetch when the server replies with a non-200 code.

func (*HTTPClientCodeError) Error 

func (h *HTTPClientCodeError) Error() string

Error implements error for HTTPClientCodeError.

func (*HTTPClientCodeError) Unwrap 

func (h *HTTPClientCodeError) Unwrap() error

Unwrap implements errors.Unwrap.

type LocalFileClient 

type LocalFileClient struct{}

LocalFileClient implements FileScheme for files on disk.

func (LocalFileClient) Fetch 

func (lfs LocalFileClient) Fetch(_ context.Context, u *url.URL) (io.ReaderAt, error)

Fetch implements FileScheme.Fetch.

type MockScheme 

type MockScheme struct {
	// scheme is the scheme name.
	Scheme string
	// contains filtered or unexported fields
}

MockScheme is a Scheme mock for testing.

func NewMockScheme 

func NewMockScheme(scheme string) *MockScheme

NewMockScheme creates a new MockScheme with the given scheme name.

func (*MockScheme) Add 

func (m *MockScheme) Add(host string, p string, content string)

Add adds a file to the MockScheme

func (*MockScheme) Fetch 

func (m *MockScheme) Fetch(ctx context.Context, u *url.URL) (io.ReaderAt, error)

Fetch implements FileScheme.Fetch.

func (*MockScheme) NumCalled 

func (m *MockScheme) NumCalled(u *url.URL) uint

NumCalled returns how many times a url has been looked up.

func (*MockScheme) SetErr 

func (m *MockScheme) SetErr(err error, count int)

SetErr sets the error which is returned on the next count calls to Fetch.

type SchemeWithRetries 

type SchemeWithRetries struct {
	Scheme FileScheme

	// DoRetry should return true to indicate the Fetch shall be retried.
	// Even if DoRetry returns true, BackOff can still determine whether to
	// stop.
	//
	// If DoRetry is nil, it will be retried if the BackOff agrees.
	DoRetry DoRetry

	// BackOff determines how often to retry and how long to wait between
	// each retry.
	BackOff backoff.BackOff
}

SchemeWithRetries wraps a FileScheme and automatically retries (with backoff) when Fetch returns a non-nil err.

func (*SchemeWithRetries) Fetch 

func (s *SchemeWithRetries) Fetch(ctx context.Context, u *url.URL) (io.ReaderAt, error)

Fetch implements FileScheme.Fetch.

type Schemes 

type Schemes map[string]FileScheme

Schemes is a map of URL scheme identifier -> implementation that can fetch a file for that scheme.

func (Schemes) Fetch 

func (s Schemes) Fetch(ctx context.Context, u *url.URL) (File, error)

Fetch fetchs the file with the given `u`. `u.Scheme` is used to select the FileScheme via `s`.

If `s` does not contain a FileScheme for `u.Scheme`, ErrNoSuchScheme is returned.

func (Schemes) LazyFetch 

func (s Schemes) LazyFetch(u *url.URL) (File, error)

LazyFetch returns a reader that will Fetch the file given by `u` when Read is called, based on `u`s scheme. See Schemes.Fetch for more details.

func (Schemes) Register 

func (s Schemes) Register(scheme string, fs FileScheme)

Register registers a scheme identified by `scheme` to be `fs`.

type TFTPClient 

type TFTPClient struct {
	// contains filtered or unexported fields
}

TFTPClient implements FileScheme for TFTP files.

func (*TFTPClient) Fetch 

func (t *TFTPClient) Fetch(_ context.Context, u *url.URL) (io.ReaderAt, error)

Fetch implements FileScheme.Fetch.

type URLError 

type URLError struct {
	URL *url.URL
	Err error
}

URLError is an error involving URLs.

func (*URLError) Error 

func (s *URLError) Error() string

Error implements error.Error.

func (*URLError) Unwrap 

func (s *URLError) Unwrap() error

Unwrap unwraps the underlying error.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.