filecache

README

filecache

A simple file cache of pure Golang

Install

First, you need to install the package:

go get -u github.com/ucwong/filecache

Overview

A file cache can be created with either the NewDefaultCache() function to get a cache with the defaults set, or NewCache() to get a new cache with 0 values for everything; you will not be able to store items in this cache until the values are changed; specifically, at a minimum, you should set the MaxItems field to be > 0.

Let's start with a basic example; we'll create a basic cache and give it a maximum item size of 128M:

import (
      "github.com/ucwong/filecache"
)

...

cache := filecache.NewDefaultCache()
cache.MaxSize = 128 * filecache.Megabyte
cache.Start()

...

cache.Stop()

The Kilobyte, Megabyte, and Gigabyte constants are provided as a convience when setting cache sizes.

When cache.Start() is called, a goroutine is launched in the background that routinely checks the cache for expired items. The delay between runs is specified as the number of seconds given by cache.Every ("every cache.Every seconds, check for expired items"). There are three criteria used to determine whether an item in the cache should be expired; they are:

1. Has the file been modified on disk? (The cache stores the last time of modification at the time of caching, and compares that to the file's current last modification time).
2. Has the file been in the cache for longer than the maximum allowed time?
3. Is the cache at capacity? When a file is being cached, a check is made to see if the cache is currently filled. If it is, the item that was last accessed the longest ago is expired and the new item takes its place. When loading items asynchronously, this check might miss the fact that the cache will be at capacity; the background scanner performs a check after its regular checks to ensure that the cache is not at capacity.

The background scanner can be disabled by setting cache.Every to 0; if so, cache expiration is only done when the cache is at capacity.

Once the cache is no longer needed, a call to cache.Stop() will close down the channels and signal the background scanner that it should stop.

Documentation

Index

• Constants
• Variables
• type CacheInfo
• type CacheItemPair
• type CacheItemPairList
  • func (p CacheItemPairList) Len() int
  • func (p CacheItemPairList) Less(i, j int) bool
  • func (p CacheItemPairList) Swap(i, j int)
• type FileCache
  • func NewDefaultCache() *FileCache
  • func (cache *FileCache) Active() bool
  • func (cache *FileCache) Cache(name string, content []byte)
  • func (cache *FileCache) CacheNow(name string) (err error)
  • func (cache *FileCache) FileSize() (totalSize int64)
  • func (cache *FileCache) GetItem(name string) (content []byte, ok bool)
  • func (cache *FileCache) GetItemString(name string) (content string, ok bool)
  • func (cache *FileCache) InCache(name string) bool
  • func (cache *FileCache) MostAccessed(count int64) []*cacheItem
  • func (cache *FileCache) ReadFile(name string) (content []byte, err error)
  • func (cache *FileCache) ReadFileString(name string) (content string, err error)
  • func (cache *FileCache) Remove(name string) (ok bool, err error)
  • func (cache *FileCache) Size() int
  • func (cache *FileCache) Start() error
  • func (cache *FileCache) Stop()
  • func (cache *FileCache) StoredFiles() (fileList []string)
  • func (cache *FileCache) WriteFile(w io.Writer, name string) (err error)
  • func (cache *FileCache) WriteItem(w io.Writer, name string) (err error)

Constants

const (
	Kilobyte = 1024
	Megabyte = 1024 * 1024
	Gigabyte = 1024 * 1024 * 1024
)

File size constants for use with FileCache.MaxSize. For example, cache.MaxSize = 64 * Megabyte

Variables

var (
	InvalidCacheItem = errors.New("invalid cache item")
	ItemIsDirectory  = errors.New("can't cache a directory")
	ItemNotInCache   = errors.New("item not in cache")
	ItemTooLarge     = errors.New("item too large for cache")
	WriteIncomplete  = errors.New("incomplete write of cache item")
)

var (
	SquelchItemNotInCache bool = true

	DefaultExpireItem int = 300 // 5 minutes

	// Max size for each item
	DefaultMaxSize int64 = 16 * Megabyte

	// Max amount of items
	DefaultMaxItems int = 32

	// Check interval by seconds
	DefaultEvery int = 60 // 1 minute

	// Mumber of items to buffer adding to the file cache.
	NewCachePipeSize int = runtime.NumCPU() * 8
)

Types

type CacheInfo

type CacheInfo struct {
	// contains filtered or unexported fields
}

type CacheItemPair

type CacheItemPair struct {
	Key         string
	AccessCount uint64
}

CacheItemPair maps key to access counter

type CacheItemPairList

type CacheItemPairList []CacheItemPair

func (CacheItemPairList) Len

func (p CacheItemPairList) Len() int

func (CacheItemPairList) Less

func (p CacheItemPairList) Less(i, j int) bool

func (CacheItemPairList) Swap

func (p CacheItemPairList) Swap(i, j int)

type FileCache

type FileCache struct {
	MaxItems   int   // Maximum number of files to cache
	MaxSize    int64 // Maximum file size to store
	ExpireItem int   // Seconds a file should be cached for
	Every      int   // Run an expiration check Every seconds
	// contains filtered or unexported fields
}

FileCache represents a cache in memory. An ExpireItem value of 0 means that items should not be expired based on time in memory.

func NewDefaultCache

func NewDefaultCache() *FileCache

NewDefaultCache returns a new FileCache with sane defaults.

func (*FileCache) Active

func (cache *FileCache) Active() bool

Active returns true if the cache has been started, and false otherwise.

func (*FileCache) Cache

func (cache *FileCache) Cache(name string, content []byte)

Cache will store the file named by 'name' to the cache. This function doesn't return anything as it passes the file onto the incoming pipe; the file will be cached asynchronously. Errors will not be returned.

func (*FileCache) CacheNow

func (cache *FileCache) CacheNow(name string) (err error)

CacheNow immediately caches the file named by 'name'.

func (*FileCache) FileSize

func (cache *FileCache) FileSize() (totalSize int64)

FileSize returns the sum of the file sizes stored in the cache

func (*FileCache) GetItem

func (cache *FileCache) GetItem(name string) (content []byte, ok bool)

GetItem returns the content of the item and a bool if name is present. GetItem should be used when you are certain an object is in the cache, or if you want to use the cache only.

func (*FileCache) GetItemString

func (cache *FileCache) GetItemString(name string) (content string, ok bool)

GetItemString is the same as GetItem, except returning a string.

func (*FileCache) InCache

func (cache *FileCache) InCache(name string) bool

InCache returns true if the item is in the cache.

func (*FileCache) MostAccessed

func (cache *FileCache) MostAccessed(count int64) []*cacheItem

MostAccessed returns the most accessed items in this cache cache

func (*FileCache) ReadFile

func (cache *FileCache) ReadFile(name string) (content []byte, err error)

ReadFile retrieves the file named by 'name'. If the file is not in the cache, load the file and cache the file in the background. If the file was not in the cache and the read was successful, the error ItemNotInCache is returned to indicate that the item was pulled from the filesystem and not the cache, unless the SquelchItemNotInCache global option is set; in that case, returns no error.

func (*FileCache) ReadFileString

func (cache *FileCache) ReadFileString(name string) (content string, err error)

ReadFileString is the same as ReadFile, except returning a string.

func (*FileCache) Remove

func (cache *FileCache) Remove(name string) (ok bool, err error)

RemoveItem immediately removes the item from the cache if it is present. It returns a boolean indicating whether anything was removed, and an error if an error has occurred.

func (*FileCache) Size

func (cache *FileCache) Size() int

Size returns the number of entries in the cache.

func (*FileCache) Start

func (cache *FileCache) Start() error

Start activates the file cache; it will start up the background caching and automatic cache expiration goroutines and initialise the internal data structures.

func (*FileCache) Stop

func (cache *FileCache) Stop()

Stop turns off the file cache. This closes the concurrent caching mechanism, destroys the cache, and the background scanner that it should stop. If there are any items or cache operations ongoing while Stop() is called, it is undefined how they will behave.

func (*FileCache) StoredFiles

func (cache *FileCache) StoredFiles() (fileList []string)

StoredFiles returns the list of files stored in the cache.

func (*FileCache) WriteFile

func (cache *FileCache) WriteFile(w io.Writer, name string) (err error)

WriteFile writes the file named by 'name' to the specified io.Writer. If the file is in the cache, it is loaded from the cache; otherwise, it is read from the filesystem and the file is cached in the background.

func (*FileCache) WriteItem

func (cache *FileCache) WriteItem(w io.Writer, name string) (err error)

WriteItem writes the cache item to the specified io.Writer.