ranger

README

ranger - io.ReaderAt with range requests!

INSTALL

$ go get github.com/Heng-Bian/ranger

OVERVIEW

Package ranger provides an implementation of io.ReaderAt and io.ReadSeeker which makes partial document requests. Ranger ships with a range fetcher that operates on an HTTP resource using the Range: header.

USE

package main

import (
	"archive/zip"
	"io"
	"github.com/Heng-Bian/ranger"
	"net/url"
	"os"
)

func main() {
	url, _ := url.Parse("http://example.com/example.zip")

	reader, _ := ranger.NewReader(&ranger.HTTPRanger{URL: url})
	length, _ := reader.Length()
	zipreader, _ := zip.NewReader(reader, length)

	data := make([]byte, zipreader.File[0].UncompressedSize64)
	rc, _ := zipreader.File[0].Open()
	io.ReadFull(rc, data)
	rc.Close()
}

Documentation

Index

• Variables
• type Block
• type ByteRange
• type HTTPClient
• type HTTPRanger
  • func (r *HTTPRanger) ExpectedLength() (int64, error)
  • func (r *HTTPRanger) FetchRanges(ranges []ByteRange) ([]Block, error)
• type RangeFetcher
• type Reader
  • func NewReader(fetcher RangeFetcher, size ...int) (*Reader, error)
  • func (r *Reader) Read(p []byte) (int, error)
  • func (r *Reader) ReadAt(p []byte, off int64) (int, error)
  • func (r *Reader) Seek(off int64, whence int) (int64, error)

Examples

• Reader

Variables

var (
	// ErrResourceChanged is the error returned by Read when the underlying resource's integrity can no longer be verified.
	// In the case of HTTP, this usually happens when the remote document's validator has changed.
	ErrResourceChanged = errors.New("unsatisfiable range request; resource may have changed")

	// ErrResourceNotFound is returned by the first Read operation that determines that a resource is inaccessible.
	ErrResourceNotFound = errors.New("resource not found")
)

Types

type Block

type Block struct {
	Length int64
	Data   []byte
}

Block represents a block returned from a ranged read

type ByteRange

type ByteRange struct {
	Start, End int64
}

ByteRange represents a not-yet-fetched range of bytes

type HTTPClient

type HTTPClient interface {
	Do(*http.Request) (*http.Response, error)
	Get(string) (*http.Response, error)
	Head(string) (*http.Response, error)
}

HTTPClient is an interface describing the methods required from net/http.Client

type HTTPRanger

type HTTPRanger struct {
	URL    *url.URL
	Client HTTPClient
	// contains filtered or unexported fields
}

HTTPRanger is a RangeFetcher that uses the HTTP Range: header to fetch blocks.

HTTPRanger first makes a HEAD request and then between 0 and Length()/BlockSize GET requests, attempting whenever possible to optimize for a lower number of requests.

No network requests are made until the first I/O-related function call.

func (*HTTPRanger) ExpectedLength

func (r *HTTPRanger) ExpectedLength() (int64, error)

ExpectedLength returns the length, in bytes, of the ranged-over file.

func (*HTTPRanger) FetchRanges

func (r *HTTPRanger) FetchRanges(ranges []ByteRange) ([]Block, error)

FetchRanges requests ranges from the HTTP server.

type RangeFetcher

type RangeFetcher interface {
	FetchRanges([]ByteRange) ([]Block, error)
	ExpectedLength() (int64, error)
}

RangeFetcher is the interface that wraps the FetchBlocks method.

FetchBlocks fetches the specified block ranges and returns any errors encountered in doing so.

Length returns the length, in bytes, of the ranged-over source.

Initialize, called once and passed the Reader's block size, performs any necessary setup tasks for the RangeFetcher

type Reader

type Reader struct {
	Length int64
	// contains filtered or unexported fields
}

Example

url, _ := url.Parse(testServer.URL + "/b.zip")

reader, _ := NewReader(&HTTPRanger{URL: url})
length := reader.Length
zipreader, _ := zip.NewReader(reader, length)

for i, v := range zipreader.File {
	fmt.Printf("[%d]: %s (%d bytes)\n", i, v.Name, v.UncompressedSize64)
}

data := make([]byte, 16)

rc, err := zipreader.File[0].Open()
if err != nil {
	defer rc.Close()
}

io.ReadFull(rc, data)

fmt.Printf("Data from f00: `%s`\n", string(data))

Output:

[0]: f00 (640 bytes)
[1]: f01 (640 bytes)
[2]: f02 (640 bytes)
[3]: f03 (640 bytes)
[4]: f04 (640 bytes)
[5]: f05 (640 bytes)
[6]: f06 (640 bytes)
[7]: f07 (640 bytes)
[8]: f08 (640 bytes)
[9]: f09 (640 bytes)
Data from f00: `f0000000000BEGIN`

func NewReader

func NewReader(fetcher RangeFetcher, size ...int) (*Reader, error)

NewRingBuffReader returns a newly-initialized Reader, which also initializes its provided RangeFetcher. The size specifies the ring buff size It returns the new reader and an error, if any.

func (*Reader) Read

func (r *Reader) Read(p []byte) (int, error)

Read reads len(p) bytes from ranged-over source. It returns the number of bytes read and the error, if any. EOF is signaled by a zero count with err set to io.EOF.

func (*Reader) ReadAt

func (r *Reader) ReadAt(p []byte, off int64) (int, error)

ReadAt reads len(p) bytes from the ranged-over source. It returns the number of bytes read and the error, if any. ReadAt always returns a non-nil error when n < len(b). At end of file, that error is io.EOF.

func (*Reader) Seek

func (r *Reader) Seek(off int64, whence int) (int64, error)

Seek sets the offset for the next Read to offset, interpreted according to whence: 0 means relative to the origin of the file, 1 means relative to the current offset, and 2 means relative to the end. It returns the new offset and an error, if any.