httpcache

Documentation

Index

• type Client
  • func NewClient(cachepath string, tms int) *Client
  • func (c *Client) ForceGet(r *Request) (io.ReadCloser, error)
  • func (c *Client) ForceGetBytes(r *Request) ([]byte, error)
  • func (c *Client) Get(u string) (io.ReadCloser, error)
  • func (c *Client) GetAndStore(u, filepath string) (io.ReadCloser, error)
  • func (c *Client) GetCached(r *Request) (io.ReadCloser, error)
  • func (c *Client) GetCachedBytes(r *Request) ([]byte, error)
  • func (c *Client) PrepCacheFile(r *Request) (string, error)
• type Request
  • func AutoRequest(uri, subdir string) *Request
  • func NewRequest(url, filename, extension string) *Request
  • func (r *Request) CachePath(dataPath string) string

Types

type Client

type Client struct {
	BeforeRequest func(*http.Request)
	HTTPClient    *http.Client
	CachePath     string
	ThrottleMS    int
}

Client makes HTTP requests and caches the results

func NewClient

func NewClient(cachepath string, tms int) *Client

NewClient initializes a default client for use in reading (and optionally caching) from remote servers. cachepath should be set to the directory where cached files' subdirectories reside.

func (*Client) ForceGet

func (c *Client) ForceGet(r *Request) (io.ReadCloser, error)

ForceGet operates like GetCached except it overwrites a previously cached file if one exists

func (*Client) ForceGetBytes

func (c *Client) ForceGetBytes(r *Request) ([]byte, error)

ForceGetBytes functions just like ForceGet, but automatically reads all bytes from the reader and returns them instead of just returning the reader

func (*Client) Get

func (c *Client) Get(u string) (io.ReadCloser, error)

Get is the most basic function for a Client. No caching is done, and just the response body is returned. All external fetching eventually lands here.

func (*Client) GetAndStore

func (c *Client) GetAndStore(u, filepath string) (io.ReadCloser, error)

GetAndStore downloads an external file and stores it at the given path

func (*Client) GetCached

func (c *Client) GetCached(r *Request) (io.ReadCloser, error)

GetCached attempts to find a file for the given request, and fetches it from its source if the file isn't locally available

func (*Client) GetCachedBytes

func (c *Client) GetCachedBytes(r *Request) ([]byte, error)

GetCachedBytes functions just like GetCached, but automatically reads all bytes from the reader and returns them instead of just returning the reader

func (*Client) PrepCacheFile

func (c *Client) PrepCacheFile(r *Request) (string, error)

PrepCacheFile ensures the directory a request will store its cached file exists or else can be created, and returns the full path to the file to be cached if successful

type Request

type Request struct {
	URL          string
	Filename     string
	Extension    string
	Subdirectory string
}

A Request contains everything needed both to look for the requested data on disk and fetch it and store it

func AutoRequest

func AutoRequest(uri, subdir string) *Request

AutoRequest uses the URL to figure out filename and extension, but requires a sub-directory to avoid collisions since filename from URL can be overly simple, lacking in context, or just not very unique. Even so, we hash the URL to add a few "unique" characters to the filename.

func NewRequest

func NewRequest(url, filename, extension string) *Request

NewRequest sets up a Request object for use in a Client's various cache-enabled functions. The filename and extension are often not a direct part of the URL, or else need to be more precise than a URL provides, so we require them to be specified here.

func (*Request) CachePath

func (r *Request) CachePath(dataPath string) string

CachePath just joins the various request data strings together to determine the cacheable filename for a given HTTP fetch. If a subdirectory wasn't specified manually (e.g., r.Subdirectory = "foo"), the extension will be used to break up cached items by file type.