Documentation ¶
Overview ¶
Package chonker implements automatic ranged HTTP requests.
A ranged request is a request that is fetched in chunks using several HTTP requests. Chunks are fetched in separate goroutines by sending HTTP Range requests to the server. Chunks are then concatenated and returned as a single io.Reader. Chunks are chunkSize bytes long. A maximum of workers chunks are fetched concurrently. If the server does not support range requests, the request fails.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( ErrInvalidArgument = errors.New( "chonker: chunkSize and workers must be greater than zero", ) ErrMultipleRangesUnsupported = errors.New("chonker: multiple ranges not supported") )
var ( // ErrRangeNoOverlap is returned by ParseRange if first-byte-pos of // all of the byte-range-spec values is greater than the content size. ErrRangeNoOverlap = errors.New("chonker: ranges failed to overlap") // ErrInvalidRange is returned by ParseRange on invalid input. ErrInvalidRange = errors.New("chonker: invalid range") // ErrUnsatisfiedRange is returned by ParseContentRange if the range is not satisfied. ErrUnsatisfiedRange = errors.New("chonker: unsatisfied range") )
var ErrRangeUnsupported = errors.New("chonker: server does not support range requests")
var ( // StatsForNerds exposes Prometheus metrics for chonker requests. // Metric names are prefixed with "chonker_". // Metrics are labeled with and grouped by request host URL. StatsForNerds = metrics.NewSet() )
Functions ¶
func Do ¶
Do sends an HTTP request and returns an HTTP response, following policy (such as redirects, cookies, auth) as configured on the client. It is a wrapper around http.Client.Do that adds support for ranged requests. A ranged request is a request that is fetched in chunks using several HTTP requests. Chunks are chunkSize bytes long. A maximum of workers chunks are fetched concurrently. HTTP HEAD requests are not fetched in chunks.
Example ¶
// Serve 1KiB of random data. content := makeData(1024) // 1KiB server := makeHttptestServer(content) defer server.Close() req, err := http.NewRequest(http.MethodGet, server.URL, nil) if err != nil { panic(err) } resp, err := Do(nil, &Request{ Request: req, chunkSize: 64, workers: 8, }) if err != nil { panic(err) } defer resp.Body.Close() fmt.Println(resp.ContentLength)
Output: 1024
func NewClient ¶
NewClient returns a new http.Client configured with a http.RoundTripper transport that fetches requests in chunks.
func NewRoundTripper ¶
NewRoundTripper returns a new http.RoundTripper that fetches requests in chunks.
Types ¶
type Chunk ¶
Chunk represents a byte range.
func ParseContentRange ¶
ParseContentRange parses a Content-Range header string as per RFC 7233. ErrUnsatisfiedRange is returned if the range is not satisfied.
func ParseRange ¶
ParseRange parses a Range header string as per RFC 7233. ErrNoOverlap is returned if none of the ranges fit inside content size. This function is a copy of the parseRange function from the Go standard library net/http/fs.go with minor modifications.
func (Chunk) ContentRangeHeader ¶ added in v1.0.1
ContentRangeHeader returns a Content-Range header value. Size is the total size of the content. Calling this method on a zero-value Chunk will return an unsatisfied range. For more information on the Content-Range header, see the MDN article on the Content-Range header.
func (Chunk) RangeHeader ¶ added in v1.0.1
RangeHeader returns a RangeHeader header value. For more information on the Range header, see the MDN article on the Range header.
type Request ¶
Request is a ranged http.Request.
func NewRequest ¶
func NewRequest( method, url string, body io.Reader, chunkSize int64, workers int, ) (*Request, error)
NewRequest returns a new Request. See NewRequestWithContext for more information.
func NewRequestWithContext ¶
func NewRequestWithContext( ctx context.Context, method, url string, body io.Reader, chunkSize int64, workers int, ) (*Request, error)
NewRequestWithContext returns a new Request. It is a wrapper around http.NewRequestWithContext that adds support for ranged requests. A ranged request is a request that is fetched in chunks using several HTTP requests. Chunks are chunkSize bytes long. A maximum of workers chunks are fetched concurrently.