cache

README

cacher

=========

[WIP]

Summary

package cacher provides a http.RoundTripper implementation that works as a mostly RFC-compliant cache for http responses.

It is only suitable for use as a 'private' cache (i.e. for a web-browser or an API-client and not for a shared proxy).

Cache Backends

---

Defaults

• The built-in 'memory' cache stores responses in an in-memory map.

Memory

• plugins/lru provides an in-memory cache that will evict least-recently used entries. (original: github.com/die-net/lrucache)
• plugins/lru/twotier allows caches to be combined, for example to use lrucache above with a persistent disk-cache. (original: github.com/die-net/lrucache/twotier)

File System - Storage

Local

• github.com/gregjones/httpcache/diskcache provides a filesystem-backed cache using the diskv library.
• plugins/bbolt
• plugins/boltdb-gzip
• plugins/boltdb-ttl
• plugins/storm
• plugins/leveldb provides a filesystem-backed cache using leveldb.

Cloud

• plugins/azurestorage uses Azure Storage service. (original: github.com/PaulARoy/azurestoragecache)
• plugins/gcs uses Google cloud service engine. (original: github.com/PaulARoy/azurestoragecache)
• plugins/diskv/s3 uses Amazon S3 for storage. (original: github.com/sourcegraph/s3cache)

KV

• plugins/etcd/v2 provides etcd api v2 implentation
• plugins/etcd/v3 provides etcd api v3 implentation
• plugins/e3ch provides etcd api v3 implentation with hierarchy
• plugins/memcache provides memcache implementations, for both App Engine and 'normal' memcache servers. (Original: memcache)

RDB

• plugins/gorm provides gorm implementations, mainly for debugging and development

Getting started

Below is a basic example of usage.

func httpCacheExample() {
    numOfRequests := 0
    ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Cache-Control", fmt.Sprintf("private, max-age=10"))
        if numOfRequests == 0 {
            w.Write([]byte("Hello!"))
        } else {
            w.Write([]byte("Goodbye!"))
        }
        numOfRequests++
    }))

    httpClient := &http.Client{
        Transport: httpcache.NewMemoryCacheTransport(),
    }
    makeRequest(ts, httpClient) // "Hello!"

    // The second request is under max-age, so the cache is used rather than hitting the server
    makeRequest(ts, httpClient) // "Hello!"

    // Sleep so the max-age is passed
    time.Sleep(time.Second * 11)

    makeRequest(ts, httpClient) // "Goodbye!"
}

func makeRequest(ts *httptest.Server, httpClient *http.Client) {
    resp, _ := httpClient.Get(ts.URL)
    var buf bytes.Buffer
    io.Copy(&buf, resp.Body)
    println(buf.String())
}

License

• MIT License

Documentation

Overview

package cacher provides a http.RoundTripper implementation that works as a mostly RFC-compliant cache for http responses.

It is only suitable for use as a 'private' cache (i.e. for a web-browser or an API-client and not for a shared proxy).

Index

• Constants
• Variables
• func CachedResponse(c Cache, req *http.Request) (resp *http.Response, err error)
• func Date(respHeaders http.Header) (date time.Time, err error)
• func InitLogger(backend logger.Logger) (bool, error)
• func NewBlockingTransport(rt http.RoundTripper) http.RoundTripper
• func NewLogger() (logx logger.Logger, err error)
• func NewTimeoutClient(connectTimeout time.Duration, readWriteTimeout time.Duration) *http.Client
• func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) func(net, addr string) (c net.Conn, err error)
• type BlockingTransport
  • func (t *BlockingTransport) RoundTrip(req *http.Request) (*http.Response, error)
• type Cache
• type CustomTransport
  • func (c *CustomTransport) RoundTrip(req *http.Request) (*http.Response, error)
• type ETagTransport
  • func (t *ETagTransport) RoundTrip(req *http.Request) (*http.Response, error)
• type MemoryCache
  • func NewMemoryCache() *MemoryCache
  • func (c *MemoryCache) Action(name string, args ...interface{}) (map[string]*interface{}, error)
  • func (c *MemoryCache) Debug(action string)
  • func (c *MemoryCache) Delete(key string)
  • func (c *MemoryCache) Get(key string) (resp []byte, ok bool)
  • func (c *MemoryCache) Set(key string, resp []byte)
• type NotFound
  • func (c *NotFound) Action(name string, args ...interface{}) (map[string]*interface{}, error)
  • func (c *NotFound) Debug(action string)
  • func (c *NotFound) Delete(string)
  • func (c *NotFound) Get(string) ([]byte, bool)
  • func (c *NotFound) Set(string, []byte)
• type Transport
  • func NewMemoryCacheTransport() *Transport
  • func NewTransport(c Cache) *Transport
  • func (t *Transport) Client() *http.Client
  • func (t *Transport) Info()
  • func (t *Transport) RoundTrip(req *http.Request) (resp *http.Response, err error)

Constants

const (
	XFromCache = "X-From-Cache" // XFromCache is the header added to responses that are returned from the cache

)

Variables

var (
	RequestBufferSize  int = 4096
	ResponseBufferSize int = 4096
	Verbose            bool
	Debug              bool
)

var (
	ErrNoDateHeader = errors.New("No Date header")
)

ErrNoDateHeader indicates that the HTTP headers contained no Date header.

Functions

func CachedResponse

func CachedResponse(c Cache, req *http.Request) (resp *http.Response, err error)

CachedResponse returns the cached http.Response for req if present, and nil otherwise.

func Date

func Date(respHeaders http.Header) (date time.Time, err error)

Date parses and returns the value of the Date header.

func InitLogger

func InitLogger(backend logger.Logger) (bool, error)

func NewBlockingTransport

func NewBlockingTransport(rt http.RoundTripper) http.RoundTripper

func NewLogger

func NewLogger() (logx logger.Logger, err error)

func NewTimeoutClient

func NewTimeoutClient(connectTimeout time.Duration, readWriteTimeout time.Duration) *http.Client

func TimeoutDialer

func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) func(net, addr string) (c net.Conn, err error)

Types

type BlockingTransport

type BlockingTransport struct {
	Transport http.RoundTripper
	// contains filtered or unexported fields
}

func (*BlockingTransport) RoundTrip

func (t *BlockingTransport) RoundTrip(req *http.Request) (*http.Response, error)

type Cache

type Cache interface {
	Get(key string) (responseBytes []byte, ok bool) // Get returns the []byte representation of a cached response and a bool set to true if the value isn't empty
	Set(key string, responseBytes []byte)           // Set stores the []byte representation of a response against a key
	Delete(key string)                              // Delete removes the value associated with the key
	Debug(action string)                            // Delete removes the value associated with the key
	Action(name string, args ...interface{}) (map[string]*interface{}, error)
}

A Cache interface is used by the Transport to store and retrieve responses.

type CustomTransport

type CustomTransport struct {
	Transport http.RoundTripper
	MaxStale  int //seconds
	Debug     bool
}

func (*CustomTransport) RoundTrip

func (c *CustomTransport) RoundTrip(req *http.Request) (*http.Response, error)

type ETagTransport

type ETagTransport struct {
	ETag string
}

func (*ETagTransport) RoundTrip

func (t *ETagTransport) RoundTrip(req *http.Request) (*http.Response, error)

type MemoryCache

type MemoryCache struct {
	// contains filtered or unexported fields
}

MemoryCache is an implemtation of Cache that stores responses in an in-memory map.

func NewMemoryCache

func NewMemoryCache() *MemoryCache

NewMemoryCache returns a new Cache that will store items in an in-memory map

func (*MemoryCache) Action

func (c *MemoryCache) Action(name string, args ...interface{}) (map[string]*interface{}, error)

func (*MemoryCache) Debug

func (c *MemoryCache) Debug(action string)

func (*MemoryCache) Delete

func (c *MemoryCache) Delete(key string)

Delete removes key from the cache

func (*MemoryCache) Get

func (c *MemoryCache) Get(key string) (resp []byte, ok bool)

Get returns the []byte representation of the response and true if present, false if not

func (*MemoryCache) Set

func (c *MemoryCache) Set(key string, resp []byte)

Set saves response resp to the cache with key

type NotFound

type NotFound struct{}

func (*NotFound) Action

func (c *NotFound) Action(name string, args ...interface{}) (map[string]*interface{}, error)

func (*NotFound) Debug

func (c *NotFound) Debug(action string)

func (*NotFound) Delete

func (c *NotFound) Delete(string)

func (*NotFound) Get

func (c *NotFound) Get(string) ([]byte, bool)

func (*NotFound) Set

func (c *NotFound) Set(string, []byte)

type Transport

type Transport struct {
	// The RoundTripper interface actually used to make requests
	// If nil, http.DefaultTransport is used
	// httpstats.NewTransport(t)
	Transport http.RoundTripper
	Cache     Cache
	// If true, responses returned from the cache will be given an extra header, X-From-Cache
	MarkCachedResponses bool
	OnDuplicateAbort    bool
	Stats               bool
	StatsEngine         string
	StatsEndpoint       string
	Debug               bool
	Timeout             time.Duration
	Dial                time.Duration
	// contains filtered or unexported fields
}

Transport is an implementation of http.RoundTripper that will return values from a cache where possible (avoiding a network request) and will additionally add validators (etag/if-modified-since) to repeated requests allowing servers to return 304 / Not Modified

func NewMemoryCacheTransport

func NewMemoryCacheTransport() *Transport

NewMemoryCacheTransport returns a new Transport using the in-memory cache implementation

func NewTransport

func NewTransport(c Cache) *Transport

NewTransport returns a new Transport with the provided Cache implementation and MarkCachedResponses set to true

func (*Transport) Client

func (t *Transport) Client() *http.Client

Client returns an *http.Client that caches responses.

func (*Transport) Info

func (t *Transport) Info()

func (*Transport) RoundTrip

func (t *Transport) RoundTrip(req *http.Request) (resp *http.Response, err error)

RoundTrip takes a Request and returns a Response

If there is a fresh Response already in cache, then it will be returned without connecting to the server.

If there is a stale Response, then any validators it contains will be set on the new request to give the server a chance to respond with NotModified. If this happens, then the cached Response will be returned.