recache

README

recache

recursive compressed caching library

TODO: copy some introduction text and diagrams from the paper

Documentation

Index

• Variables
• type Cache
  • func NewCache(opts CacheOptions) (c *Cache)
  • func (c *Cache) EvictAll(t time.Duration)
  • func (c *Cache) NewFrontend(get Getter) *Frontend
• type CacheOptions
• type Frontend
  • func (f *Frontend) Evict(t time.Duration, k Key)
  • func (f *Frontend) EvictAll(t time.Duration)
  • func (f *Frontend) EvictByFunc(t time.Duration, fn func(Key) (bool, error)) error
  • func (f *Frontend) Get(k Key) (*Record, error)
  • func (f *Frontend) WriteHTTP(k Key, w http.ResponseWriter, r *http.Request) (n int64, err error)
• type Getter
• type Key
• type Record
  • func (r *Record) DecodeJSON(dst interface{}) (err error)
  • func (r *Record) Decompress() io.Reader
  • func (r *Record) ETag() string
  • func (r *Record) ETagDecompressed() string
  • func (r *Record) NewReader() io.Reader
  • func (r *Record) SHA1() [sha1.Size]byte
  • func (r *Record) WriteTo(w io.Writer) (n int64, err error)
• type RecordWriter
  • func (rw *RecordWriter) Bind(f *Frontend, k Key) (*Record, error)
  • func (rw *RecordWriter) BindJSON(f *Frontend, k Key, dst interface{}) (err error)
  • func (rw *RecordWriter) Include(f *Frontend, k Key) (err error)
  • func (rw *RecordWriter) ReadFrom(r io.Reader) (n int64, err error)
  • func (rw *RecordWriter) Write(p []byte) (n int, err error)

Variables

var (

	// Global deflate compression level configuration.
	//
	// Can only be changed before the first Cache is constructed and must not be
	// mutated after.
	CompressionLevel = flate.DefaultCompression
)

var (
	// Indicates no components have been written and no error has been returned
	// in a call to Getter. This is not allowed.
	ErrEmptyRecord = errors.New("empty record created")
)

Types

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Unified storage for cached records with specific eviction parameters

func NewCache

func NewCache(opts CacheOptions) (c *Cache)

Create new cache with specified memory and LRU eviction limits. After either of these are exceeded, the least recently used cache records will be evicted, until the requirements are satisfied again. Note that this eviction is eventual and not immediate for optimisation purposes.

Pass in zero values to ignore either or both eviction limits.

func (*Cache) EvictAll

func (c *Cache) EvictAll(t time.Duration)

Evict all records from cache after t amount of time, if the matched are still in the cache by then.

If t = 0, any matched record(s) are evicted immediately.

t can be used to decrease record turnover on often evicted records, thereby decreasing fresh data fetches and improving performance.

Any subsequent scheduled eviction calls on matching records with a greater t value than is currently left from a previous scheduled eviction on the record will have no effect.

A scheduled eviction with a smaller timer than currently left on the record will replace the existing timer.

func (*Cache) NewFrontend

func (c *Cache) NewFrontend(get Getter) *Frontend

Create new Frontend for accessing the cache. A Frontend must only be created using this method.

get() will be used for generating fresh cache records for the given key by the cache engine. These records will be stored by the cache engine and must not be modified after Get() returns. Get() must be thread-safe.

type CacheOptions

type CacheOptions struct {
	// Maximum amount of memory the cache can consume without forcing eviction
	MemoryLimit uint

	// Maximum last use time of record without forcing eviction
	LRULimit time.Duration
}

Options for new cache creation

type Frontend

type Frontend struct {
	// contains filtered or unexported fields
}

A frontend for accessing the cache contents

func (*Frontend) Evict

func (f *Frontend) Evict(t time.Duration, k Key)

Evict a record by key after t amount of time, if the matched are still in the cache by then.

If t = 0, any matched record(s) are evicted immediately.

t can be used to decrease record turnover on often evicted records, thereby decreasing fresh data fetches and improving performance.

Any subsequent scheduled eviction calls on matching records with a greater t value than is currently left from a previous scheduled eviction on the record will have no effect.

A scheduled eviction with a smaller timer than currently left on the record will replace the existing timer.

func (*Frontend) EvictAll

func (f *Frontend) EvictAll(t time.Duration)

Evict all records from frontend after t amount of time, if the matched are still in the cache by then.

If t = 0, any matched record(s) are evicted immediately.

t can be used to decrease record turnover on often evicted records, thereby decreasing fresh data fetches and improving performance.

Any subsequent scheduled eviction calls on matching records with a greater t value than is currently left from a previous scheduled eviction on the record will have no effect.

A scheduled eviction with a smaller timer than currently left on the record will replace the existing timer.

func (*Frontend) EvictByFunc

func (f *Frontend) EvictByFunc(t time.Duration, fn func(Key) (bool, error),
) error

Evict records from frontend using matcher function fn after t amount of time,

if the matched are still in the cache by then.

If t = 0, any matched record(s) are evicted immediately.

t can be used to decrease record turnover on often evicted records, thereby decreasing fresh data fetches and improving performance.

Any subsequent scheduled eviction calls on matching records with a greater t value than is currently left from a previous scheduled eviction on the record will have no effect.

A scheduled eviction with a smaller timer than currently left on the record will replace the existing timer.

func (*Frontend) Get

func (f *Frontend) Get(k Key) (*Record, error)

Retrieve or generate data by key and return cache Record

func (*Frontend) WriteHTTP

func (f *Frontend) WriteHTTP(k Key, w http.ResponseWriter, r *http.Request,
) (n int64, err error)

Retrieve or generate data by key and write it to w. Writes ETag to w and returns 304 on ETag match without writing data. Sets "Content-Encoding" header to "deflate", if client support deflate compressions

type Getter

type Getter func(Key, *RecordWriter) error

Generates fresh cache records for the given key by writing to RecordWriter. Getter must be thread-safe.

type Key

type Key interface{}

Value used to store entries in the cache. Must be a type suitable for being a key in a Go map.

type Record

type Record struct {
	// contains filtered or unexported fields
}

Data storage unit in the cache. Linked to a single Key on a Frontend.

func (*Record) DecodeJSON

func (r *Record) DecodeJSON(dst interface{}) (err error)

Convenience method for efficiently decoding stream contents as JSON into the destination variable.

dst: pointer to destination variable

func (*Record) Decompress

func (r *Record) Decompress() io.Reader

Create a new io.Reader for the Decompressped content of this stream

func (*Record) ETag

func (r *Record) ETag() string

Return strong ETag of content, if served as a compressed stream

func (*Record) ETagDecompressed

func (r *Record) ETagDecompressed() string

Return strong ETag of content, if served as a decompressed stream

func (*Record) NewReader

func (r *Record) NewReader() io.Reader

Create a new io.Reader for this stream. Multiple instances of such an io.Reader can exist and be read concurrently.

func (*Record) SHA1

func (r *Record) SHA1() [sha1.Size]byte

Return SHA1 hash of the content

func (*Record) WriteTo

func (r *Record) WriteTo(w io.Writer) (n int64, err error)

Implements io.WrWriteTo

type RecordWriter

type RecordWriter struct {
	// contains filtered or unexported fields
}

Provides utility methods for building record buffers and recursive record trees

func (*RecordWriter) Bind

func (rw *RecordWriter) Bind(f *Frontend, k Key) (*Record, error)

Bind to record from passed frontend by key and return the retrieved record.

The record generated by rw will automatically be evicted from its parent cache on eviction of the included record.

func (*RecordWriter) BindJSON

func (rw *RecordWriter) BindJSON(
	f *Frontend,
	k Key,
	dst interface{},
) (err error)

Bind to record from passed frontend by key and decode it as JSON into dst.

The record generated by rw will automatically be evicted from its parent cache on eviction of the included record.

func (*RecordWriter) Include

func (rw *RecordWriter) Include(f *Frontend, k Key) (err error)

Include data from passed frontend by key and bind it to rw. The record generated by rw will automatically be evicted from its parent cache on eviction of the included record.

func (*RecordWriter) ReadFrom

func (rw *RecordWriter) ReadFrom(r io.Reader) (n int64, err error)

Read non-compressed data from r and write it to the record for storage

func (*RecordWriter) Write

func (rw *RecordWriter) Write(p []byte) (n int, err error)

Write non-compressed data to the record for storage