htfs

Documentation

Index

• Variables
• type File
  • func Open(getURL GetURLFunc, needsRenewal NeedsRenewalFunc, settings *Settings) (*File, error)
  • func (f *File) Close() error
  • func (f *File) GetHeader() http.Header
  • func (f *File) GetRequestURL() *url.URL
  • func (f *File) NumConns() int
  • func (f *File) Read(buf []byte) (int, error)
  • func (f *File) ReadAt(buf []byte, offset int64) (int, error)
  • func (f *File) Reset() error
  • func (f *File) Seek(offset int64, whence int) (int64, error)
  • func (f *File) Stat() (os.FileInfo, error)
• type FileInfo
  • func (hfi *FileInfo) IsDir() bool
  • func (hfi *FileInfo) ModTime() time.Time
  • func (hfi *FileInfo) Mode() os.FileMode
  • func (hfi *FileInfo) Name() string
  • func (hfi *FileInfo) Size() int64
  • func (hfi *FileInfo) Sys() interface{}
• type GetURLFunc
• type LogFunc
• type NeedsRenewalFunc
• type Resetter
• type ServerError
  • func (se *ServerError) Error() string
• type ServerErrorCode
• type Settings

Variables

var ErrNotFound = goerrors.New("HTTP file not found on server")

ErrNotFound is returned when the HTTP server returns 404 - it's not considered a temporary error

var ErrTooManyRenewals = goerrors.New("Giving up, getting too many renewals. Try again later or contact support.")

ErrTooManyRenewals is returned when we keep calling the GetURLFunc but it immediately return an errors marked as renewal-related by NeedsRenewalFunc. This can happen when servers are misconfigured.

Types

type File

type File struct {
	Log      LogFunc
	LogLevel int

	ConnStaleThreshold time.Duration
	MaxConns           int

	ForbidBacktracking bool
	DumpStats          bool
	// contains filtered or unexported fields
}

File allows accessing a file served by an HTTP server as if it was local (for random-access reading purposes, not writing)

func Open

func Open(getURL GetURLFunc, needsRenewal NeedsRenewalFunc, settings *Settings) (*File, error)

Open returns a new htfs.File. Note that it differs from os.Open in that it does a first request to determine the remote file's size. If that fails (after retries), an error will be returned.

func (*File) Close

func (f *File) Close() error

Close closes all connections to the distant http server used by this File

func (*File) GetHeader

func (f *File) GetHeader() http.Header

GetHeader returns the header the server responded with on our initial request. It may contain checksums which could be used for integrity checking.

func (*File) GetRequestURL

func (f *File) GetRequestURL() *url.URL

GetRequestURL returns the first good URL File made a request to.

func (*File) NumConns

func (f *File) NumConns() int

NumConns returns the number of connections currently used by the File to serve ReadAt calls

func (*File) Read

func (f *File) Read(buf []byte) (int, error)

func (*File) ReadAt

func (f *File) ReadAt(buf []byte, offset int64) (int, error)

ReadAt reads len(buf) byte from the remote file at offset. It returns the number of bytes read, and an error. In case of temporary network errors or timeouts, it will retry with truncated exponential backoff according to RetrySettings

func (*File) Reset

func (f *File) Reset() error

func (*File) Seek

func (f *File) Seek(offset int64, whence int) (int64, error)

Seek the read head within the file - it's instant and never returns an error, except if whence is one of os.SEEK_SET, os.SEEK_END, or os.SEEK_CUR. If an invalid offset is given, it will be truncated to a valid one, between [0,size).

func (*File) Stat

func (f *File) Stat() (os.FileInfo, error)

Stat returns an os.FileInfo for this particular file. Only the Size() method is useful, the rest is default values.

type FileInfo

type FileInfo struct {
	// contains filtered or unexported fields
}

FileInfo implements os.FileInfo for Files

func (*FileInfo) IsDir

func (hfi *FileInfo) IsDir() bool

func (*FileInfo) ModTime

func (hfi *FileInfo) ModTime() time.Time

func (*FileInfo) Mode

func (hfi *FileInfo) Mode() os.FileMode

func (*FileInfo) Name

func (hfi *FileInfo) Name() string

func (*FileInfo) Size

func (hfi *FileInfo) Size() int64

func (*FileInfo) Sys

func (hfi *FileInfo) Sys() interface{}

type GetURLFunc

type GetURLFunc func() (urlString string, err error)

A GetURLFunc returns a URL we can download the resource from. It's handy to have this as a function rather than a constant for signed expiring URLs

type LogFunc

type LogFunc func(msg string)

A LogFunc prints debug message

type NeedsRenewalFunc

type NeedsRenewalFunc func(res *http.Response, body []byte) bool

A NeedsRenewalFunc analyzes an HTTP response and returns true if it needs to be renewed

type Resetter

type Resetter interface {
	Reset() error
}

type ServerError

type ServerError struct {
	Host       string
	Message    string
	Code       ServerErrorCode
	StatusCode int
}

ServerError represents an error htfs has encountered when talking to a remote server.

func (*ServerError) Error

func (se *ServerError) Error() string

type ServerErrorCode

type ServerErrorCode int64

ServerErrorCode represents an error condition where some server does not support htfs - perhaps because it has no range support, or because it returned a bad HTTP status code.

const (
	// ServerErrorCodeUnknown does not map to any known errors.
	// It's used for any unexpected HTTP status codes.
	ServerErrorCodeUnknown ServerErrorCode = iota
	// ServerErrorCodeNoRangeSupport indicates that the remote
	// server does not support HTTP Range Requests:
	// https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests
	ServerErrorCodeNoRangeSupport
)

type Settings

type Settings struct {
	Client             *http.Client
	RetrySettings      *retrycontext.Settings
	Log                LogFunc
	LogLevel           int
	ForbidBacktracking bool
	DumpStats          bool
}

Settings allows passing additional settings to an File