filecache

README

filecache caches build artifacts in files

It is a copy of the Go internal cmd/go/internal/cache which is used for go build caching.

The default setup from the environment (GOCACHE) has been removed, but the trimming defaults (1 hour mtime resolution, evict files older than 5 days) has been kept. It is also possible to override these with TrimWithLimits.

Documentation

Overview

Package filecache implements an artifact cache.

It is copied from Go's cmd/go/internal/cache, cleared the Go-specific environment settings for default cache, Go version-salted hash, and added TrimWithLimits.

Index

• Constants
• type ActionID
  • func NewActionID(p []byte) ActionID
• type Cache
  • func Open(dir string) (*Cache, error)
  • func (c *Cache) Get(id ActionID) (Entry, error)
  • func (c *Cache) GetBytes(id ActionID) ([]byte, Entry, error)
  • func (c *Cache) GetFile(id ActionID) (file string, entry Entry, err error)
  • func (c *Cache) OutputFile(out OutputID) string
  • func (c *Cache) Put(id ActionID, file io.ReadSeeker) (OutputID, int64, error)
  • func (c *Cache) PutBytes(id ActionID, data []byte) error
  • func (c *Cache) SetMTimeInterval(d time.Duration)
  • func (c *Cache) Trim()
  • func (c *Cache) TrimWithLimit(trimInterval, trimLimit time.Duration)
• type Entry
• type Hash
  • func NewHash() Hash
  • func (h Hash) SumID() ID
• type ID
• type OutputID

Constants

const (
	DefaultMTimeInterval = 1 * time.Hour
	DefaultTrimInterval  = 24 * time.Hour
	DefaultTrimLimit     = 5 * 24 * time.Hour
)

Time constants for cache expiration.

We set the mtime on a cache file on each use, but at most one per mtimeInterval (1 hour), to avoid causing many unnecessary inode updates. The mtimes therefore roughly reflect "time of last use" but may in fact be older by at most an hour.

We scan the cache for entries to delete at most once per trimInterval (1 day).

When we do scan the cache, we delete entries that have not been used for at least trimLimit (5 days). Statistics gathered from a month of usage by Go developers found that essentially all reuse of cached entries happened within 5 days of the previous reuse. See golang.org/issue/22990.

const HashSize = sha256.Size

Types

type ActionID

type ActionID ID

An ActionID is a cache action key, the hash of a complete description of a repeatable computation (command line, environment variables, input file contents, executable contents).

func NewActionID

func NewActionID(p []byte) ActionID

NewActionID returns the hashed bytes.

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

A Cache is a package cache, backed by a file system directory tree.

func Open

func Open(dir string) (*Cache, error)

Open opens and returns the cache in the given directory.

It is safe for multiple processes on a single machine to use the same cache directory in a local file system simultaneously. They will coordinate using operating system file locks and may duplicate effort but will not corrupt the cache.

However, it is NOT safe for multiple processes on different machines to share a cache directory (for example, if the directory were stored in a network file system). File locking is notoriously unreliable in network file systems and may not suffice to protect the cache.

func (*Cache) Get

func (c *Cache) Get(id ActionID) (Entry, error)

Get looks up the action ID in the cache, returning the corresponding output ID and file size, if any. Note that finding an output ID does not guarantee that the saved file for that output ID is still available.

func (*Cache) GetBytes

func (c *Cache) GetBytes(id ActionID) ([]byte, Entry, error)

GetBytes looks up the action ID in the cache and returns the corresponding output bytes. GetBytes should only be used for data that can be expected to fit in memory.

func (*Cache) GetFile

func (c *Cache) GetFile(id ActionID) (file string, entry Entry, err error)

GetFile looks up the action ID in the cache and returns the name of the corresponding data file.

func (*Cache) OutputFile

func (c *Cache) OutputFile(out OutputID) string

OutputFile returns the name of the cache file storing output with the given OutputID.

func (*Cache) Put

func (c *Cache) Put(id ActionID, file io.ReadSeeker) (OutputID, int64, error)

Put stores the given output in the cache as the output for the action ID. It may read file twice. The content of file must not change between the two passes.

func (*Cache) PutBytes

func (c *Cache) PutBytes(id ActionID, data []byte) error

PutBytes stores the given bytes in the cache as the output for the action ID.

func (*Cache) SetMTimeInterval

func (c *Cache) SetMTimeInterval(d time.Duration)

SetMTimeInterval set the time precision for updating file access times. The default is 1 hour.

func (*Cache) Trim

func (c *Cache) Trim()

Trim removes old cache entries that are likely not to be reused. It uses the default trim interval and limit.

func (*Cache) TrimWithLimit

func (c *Cache) TrimWithLimit(trimInterval, trimLimit time.Duration)

TrimLimited removes old cache entries that are likely not to be reused.

For each duration, <=0 means Default.

type Entry

type Entry struct {
	OutputID OutputID
	Size     int64
	Time     time.Time
}

type Hash

type Hash struct {
	hash.Hash
}

func NewHash

func NewHash() Hash

func (Hash) SumID

func (h Hash) SumID() ID

type ID

type ID [HashSize]byte

type OutputID

type OutputID ID

An OutputID is a cache output key, the hash of an output of a computation.