httpio

README

httpio

Implement some io interfaces using an http client

Documentation

Index

• Constants
• Variables
• type Option
  • func WithClient(c *http.Client) Option
  • func WithContext(ctx context.Context) Option
  • func WithExpectHeaders(e map[string]string) Option
  • func WithHashChunkSize(s int64) Option
  • func WithMaxConcurrentReaders(r int64) Option
  • func WithURL(url string) Option
• type Options
• type ReadAtCloser
  • func NewReadAtCloser(opts ...Option) (r *ReadAtCloser, err error)
  • func (r *ReadAtCloser) ChunkSize() int64
  • func (r *ReadAtCloser) Close() error
  • func (r *ReadAtCloser) Etag() string
  • func (r *ReadAtCloser) HashURL(scheme uint) ([]hash.Hash, error)
  • func (r *ReadAtCloser) Length() int64
  • func (r *ReadAtCloser) ReadAt(b []byte, start int64) (n int, err error)
  • func (r *ReadAtCloser) URL() string
• type ReadCloser
  • func (r *ReadCloser) Close() error
  • func (r *ReadCloser) HashURL(size uint) (hash.Hash, error)
  • func (r *ReadCloser) Read(p []byte) (n int, err error)
• type RequestError
  • func (r RequestError) Error() string
  • func (r RequestError) HTTPStatus() (int, bool)

Constants

const MaxConcurrentReaders = 5

MaxConcurrentReaders limits the number of concurrent reads.

const ReadSizeLimit = 32768

ReadSizeLimit is the maximum size buffer chunk to operate on.

Variables

var (
	ErrInvalidURLHost        = errors.New("invalid url host")
	ErrInvalidURLScheme      = errors.New("invalid url scheme")
	ErrReadFailed            = errors.New("read failed")
	ErrReadFromSource        = errors.New("read from source")
	ErrRangeReadNotSupported = errors.New("range reads not supported")
	ErrRangeReadNotSatisfied = errors.New("range not satisfied")

	ErrHeaderEtag          = errors.New("etag header differs")
	ErrHeaderContentLength = errors.New("content length differs")
)

Possible errors

Types

type Option

type Option func(*Options)

Option is a func type used to pass options to the New* funcs.

func WithClient

func WithClient(c *http.Client) Option

WithClient is an Option func which allows the user to supply their own instance of an http.Client. If not supplied a new generic http.Client is created.

func WithContext

func WithContext(ctx context.Context) Option

WithContext allows supplying a context for the ReadAtCloser to use.

func WithExpectHeaders

func WithExpectHeaders(e map[string]string) Option

WithExpectHeaders is an Option func used to specify any expected response headers from the server.

func WithHashChunkSize

func WithHashChunkSize(s int64) Option

WithHashChunkSize is an Option func to specify the size to chunk content at when hashing the content.

func WithMaxConcurrentReaders

func WithMaxConcurrentReaders(r int64) Option

WithMaxConcurrentReaders is an Option to set the maximum number of concurrent go funcs performing Reads in any Reader. If not supplied the default MaxConcurrentReaders is used.

func WithURL

func WithURL(url string) Option

WithURL (REQUIRED) is an Option func used to supply the full url string; scheme, host, and path, to be read.

type Options

type Options struct {
	// contains filtered or unexported fields
}

Options contains the parts to create and use a ReadCloser or ReadAtCloser

type ReadAtCloser

type ReadAtCloser struct {
	// contains filtered or unexported fields
}

ReadAtCloser contains the required options and metadata to implement io.ReadAtCloser on a URL. Use the Options to configure the ReadAtCloser with an http.Client, URL, and any additional http.Header values that should be sent with the request.

func NewReadAtCloser

func NewReadAtCloser(opts ...Option) (r *ReadAtCloser, err error)

NewReadAtCloser validates the options provided and returns a new a *ReadAtCloser after validating the URL. The URL validation includes basic scheme and hostnane checks.

func (*ReadAtCloser) ChunkSize

func (r *ReadAtCloser) ChunkSize() int64

func (*ReadAtCloser) Close

func (r *ReadAtCloser) Close() error

Close cancels the client context and closes any idle connections.

func (*ReadAtCloser) Etag

func (r *ReadAtCloser) Etag() string

Etag returns the last read Etag from the URL being operated on by the configured ReadAtCloser.

func (*ReadAtCloser) HashURL

func (r *ReadAtCloser) HashURL(scheme uint) ([]hash.Hash, error)

HashURL takes the hash scheme as a uint (either sha256.Size or md5.Size) and the chunk size, and returns the hashed URL body in the supplied scheme as a slice of hash.Hash. When the chunk size is less than the length of the content, the URL will be read with multiple, concurrent range reads to create the slice of hash.Hash. Specifying a chunkSize <= 0 is translated to "do not chunk" and the entire content will be hashed as one chunk. The size and capacity of the returned slice of hash.Hash is equal to the number of chunks calculated based on the content length divided by the chunkSize, or 1 if chunkSize is equal to, or less than 0.

func (*ReadAtCloser) Length

func (r *ReadAtCloser) Length() int64

Length returns the reported ContentLength of the URL body.

func (*ReadAtCloser) ReadAt

func (r *ReadAtCloser) ReadAt(b []byte, start int64) (n int, err error)

ReadAt satisfies the io.ReaderAt interface. It requires the ReadAtCloser be previously configured.

func (*ReadAtCloser) URL

func (r *ReadAtCloser) URL() string

type ReadCloser

type ReadCloser struct {
	// contains filtered or unexported fields
}

ReadCloser contains the required parts to implement a io.ReadCloser on a URL.

func (*ReadCloser) Close

func (r *ReadCloser) Close() error

Close cancels the client context and closes any idle connections.

func (*ReadCloser) HashURL

func (r *ReadCloser) HashURL(size uint) (hash.Hash, error)

HashURL takes the hash scheme size (sha256.Size or md5.Size) and returns the hashed URL body in the supplied scheme as a hash.Hash interface.

func (*ReadCloser) Read

func (r *ReadCloser) Read(p []byte) (n int, err error)

Read fulfills the io.Reader interface. The ReadCloser must be previously configured. The body of the configured URL is read into p, up to len(p). If the length of p is greater than the ContentLength of the body the length returned will be ContentLength.

type RequestError

type RequestError struct {
	StatusCode int
	Url        string
}

RequestError fulfills the error type for reporting more specific errors with http requests.

func (RequestError) Error

func (r RequestError) Error() string

Error returns the string of a RequestError.

func (RequestError) HTTPStatus

func (r RequestError) HTTPStatus() (int, bool)

HTTPStatus returns the http code from the RequestError.