Documentation ¶
Index ¶
- func GetContentLength(url *url.URL, client HTTPClient) (int64, error)
- func GetContentLengthViaGET(url *url.URL, client HTTPClient) (int64, error)
- func GetContentLengthViaHEAD(url *url.URL, client HTTPClient) (int64, error)
- type ByteRange
- type HTTPClient
- type Loader
- type LoaderFunc
- type RangedSource
- func (rs RangedSource) ParallelOffsetReader(ctx context.Context, parallelism int, offset int64) io.Reader
- func (rs RangedSource) ParallelReader(ctx context.Context, parallelism int) io.Reader
- func (rs RangedSource) ReadAt(p []byte, off int64) (n int, err error)
- func (rs RangedSource) Reader() io.ReadSeeker
- type Ranger
- type RangingHTTPClient
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func GetContentLength ¶ added in v0.2.0
func GetContentLength(url *url.URL, client HTTPClient) (int64, error)
GetContentLength returns the content length of the given URL, using the given HTTPClient. It first attempts to use the HEAD method, but if that fails, falls back to using the GET method.
func GetContentLengthViaGET ¶ added in v0.2.0
func GetContentLengthViaGET(url *url.URL, client HTTPClient) (int64, error)
GetContentLengthViaGET returns the content length of the given URL, using the given HTTPClient. It uses a GET request with a zeroed Range header to get the content length.
func GetContentLengthViaHEAD ¶ added in v0.2.0
func GetContentLengthViaHEAD(url *url.URL, client HTTPClient) (int64, error)
GetContentLengthViaHEAD returns the content length of the given URL, using the given HTTPClient. It uses a HEAD request to get the content length.
Types ¶
type ByteRange ¶
ByteRange represents a range of bytes available in a file
func (ByteRange) RangeHeader ¶ added in v0.2.0
RangeHeader returns the HTTP header representation of the byte range, suitable for use in the Range header, as described in https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range
type HTTPClient ¶
HTTPClient provides an interface allowing us to perform HTTP requests.
type Loader ¶
Loader implements a Load method that provides data as byte slice for a given byte range chunk.
`Load` should be safe to call from multiple goroutines.
If err is nil, the returned byte slice must always have exactly as many bytes as was asked for, i.e. `len([]byte)` returned must always be equal to `br.Length()`.
func DefaultHTTPLoader ¶ added in v0.2.0
func HTTPLoader ¶ added in v0.2.0
func HTTPLoader(url *url.URL, client HTTPClient) Loader
type LoaderFunc ¶
LoaderFunc converts a Load function into a Loader type.
type RangedSource ¶
type RangedSource struct {
// contains filtered or unexported fields
}
func NewRangedSource ¶
func NewRangedSource(length int64, loader Loader, ranger Ranger) RangedSource
func (RangedSource) ParallelOffsetReader ¶ added in v0.2.0
func (rs RangedSource) ParallelOffsetReader(ctx context.Context, parallelism int, offset int64) io.Reader
ParallelOffsetReader returns an io.Reader that reads the data in parallel, using a number of goroutines equal to the given parallelism count. Data is still returned in order. The reader will start reading at the given offset.
func (RangedSource) ParallelReader ¶ added in v0.2.0
ParallelReader returns an io.Reader that reads the data in parallel, using a number of goroutines equal to the given parallelism count. Data is still returned in order.
func (RangedSource) ReadAt ¶
func (rs RangedSource) ReadAt(p []byte, off int64) (n int, err error)
ReadAt implements the io.ReaderAt interface, while reading the data only in chunks based on the given Ranger.
func (RangedSource) Reader ¶
func (rs RangedSource) Reader() io.ReadSeeker
Reader provides a new io.ReadSeeker view for the ranged source.
type Ranger ¶
type Ranger struct {
// contains filtered or unexported fields
}
Ranger can split a file into chunks of a given size.
type RangingHTTPClient ¶
type RangingHTTPClient struct { HTTPClient // contains filtered or unexported fields }
RangingHTTPClient wraps another HTTP client to issue all requests in pre-defined chunks.
func NewParallelRangingClient ¶ added in v0.2.0
func NewParallelRangingClient(ranger Ranger, client HTTPClient, parallelism int) RangingHTTPClient
NewParallelRangingClient wraps and uses the given HTTPClient to make requests only for chunks designated by the given Ranger, but does so in parallel with the given number of goroutines. This is useful for downloading large files from cache-friendly sources in manageable chunks, with the added speed benefits of parallelism.
func NewRangingHTTPClient ¶
func NewRangingHTTPClient(ranger Ranger, client HTTPClient) RangingHTTPClient
NewRangingHTTPClient wraps and uses the given HTTPClient to make requests only for chunks designated by the given Ranger. This is useful for downloading large-files from cache-friendly sources in manageable chunks.