cache

Documentation

Index

• func PathIsDir(directory string) bool
• type Local
  • func NewLocal(label string) *Local
  • func (c *Local) Commit(location string) (string, error)
  • func (c *Local) Exists(key string) (string, error)
  • func (c *Local) Get(key string) (string, bool, error)
  • func (c *Local) Purge(location string) error
  • func (l *Local) Register(flags kflags.FlagSet, prefix string) *Local
  • func (c *Local) Rollback(location string) error
• type Store

Functions

func PathIsDir

func PathIsDir(directory string) bool

PathIsDir returns true if the specified path is a directory.

Types

type Local

type Local struct {
	Root string
}

Local implements a local file system based cache.

func NewLocal

func NewLocal(label string) *Local

NewLocal will return a Local cache pointing to the OS specific directory where files are cached. label is the name of the subdirectory, it should not be empty.

Example:

l := NewLocal("gnome")

Will return /home/username/.cache/gnome as the location for the cache.

func (*Local) Commit

func (c *Local) Commit(location string) (string, error)

Commits a cache location so others can start retrieving it.

When Get() encounters a key that was not in cache already, it returns a temporary location where to store the data. This location is not available for others to use until it is committed with a Commit() call.

It is recommended that Commit() is only called once the data to be cached has successfully been generated and verified - once it can be treated as a fully complete read only resource.

If your code, however, can deal with inconsistent/incomplete data in cache, or with concurrent write access to data in a common shared directory, you can call Commit() immediately after Get().

The `location` string must be a value returned by Get().

func (*Local) Exists

func (c *Local) Exists(key string) (string, error)

Returns true if a cache entry by the specified key exists already.

Note that this is no guarantee that the cache key will exist by the time you use it. If you need a cache key in a concurrent environment, you should always use Get(), Commit(), and Purge().

func (*Local) Get

func (c *Local) Get(key string) (string, bool, error)

Get returns a directory name where to store data corresponding to key.

key is a unique identifier for the data to be stored (or retrieved) from cache. It can be, for example, the URL for a large download, or the id of an object in a database.

Get will return three parameters:

• the path to a directory, where to store or read the data corresponding to this key.
• a boolean indicating if there was already a directory associated with this key (true), or if a new directory was created from scratch (false). When true, you should expect data previously stored to be ready and available in the directory, but see below.
• an error, normally nil, indicating any error that happened while trying to create this cached object.

For a cache to be effective in an environment supporting parallel builds:

• only complete and usable artifacts should be stored in cache.
• data in cache should never be modified.

To ensure both:

1. When using the cache, call Get()
2. If the data is there already, just use it.
3. If not there, download it, generate it, ... and store it directly in the directory returned by Get().
4. Once the download, build, ... process is completed successfully, call Commit(), described below. In case of error, call Purge().

Between 3 and 4, the Local implementation guarantees that no other job will see the partial results being built or downloaded, the key will become available from cache (and results stored) only after Commit() is called.

If you need to make changes to values in Local(), use Clone().

func (*Local) Purge

func (c *Local) Purge(location string) error

Purge removes and frees up the resources associated with a location generated by Get.

Purge is used in two different cases:

1. To remove an existing entry from cache. Invoke Get for the key, pass the location to Purge() to eliminate it from cache.
2. To remove any resource associated with a failed attempt at filling a cache entry. Let's say Get() returned a new location. Your code is preparing data to be stored in cache, but the build fails. To free up resources in this cache entry, call Purge(location).

Note that once a resource is Commit()ted, the location changes. So if Purge() is invoked on a resource just committed, it will just fail without doing anything. It is thus recommended that a defer c.Purge(location) is set up every time a new cache entry is being filled, like with:

location, ready, err := cache.Get("downloaded-libc")
if !ready {
    defer cache.Purge(location)
    // retry downloading file.
    if err != nil {
         return err // cache.Purge() will be invoked, and clean up.
    }
    cache.Commit(location)
    // cache.Purge, when invoked by the defer, is now a noop.
}

func (*Local) Register

func (l *Local) Register(flags kflags.FlagSet, prefix string) *Local

func (*Local) Rollback

func (c *Local) Rollback(location string) error

Rollback purges the directory if it has not been committed.

type Store

type Store interface {
	// Get creates or returns an existing location where data corresponding to key is stored.
	//
	// The first value returned is a path, the path where data is to be stored, or has been stored before.
	// The second value returned is true if the path was existing already, false otherwise.
	// An error is returned if for some reason the key could not be created or looked up.
	//
	// If the location returned by Get was newly created, this location must either be Purged,
	// Rolled back, or Committed, otherwise your code will leak cache keys.
	//
	// If a location was already existing, data should be accessed in a read only way, unless
	// the application is capable of handling concurrent writes on its own data.
	//
	// Both Commit and Rollback can safely be called on an existing location, in which case
	// nothing will be done. This is convenient to use with defer.
	//
	// Purge can be called on existing as well as new locations, and will result in the data
	// being forever deleted.
	Get(key string) (string, bool, error)

	// Exists check if there is an entry in the cache corresponding to key.
	//
	// If there is, it returns the path on disk. If there is not, it returns the empty string.
	// Error is returned if the location could not be found.
	Exists(key string) (string, error)

	// Commit ensures that the changes made to the cache at location are saved.
	//
	// Calling Commit multiple times on the same location, committed or not, is a noop.
	//
	// If multiple Get() calls for the same key are performed, and all are Committed in parallel,
	// only one of the commits will be saved, which one is undetermined.
	Commit(location string) (string, error)

	// Purge will remove the cache entry at location.
	// Purge works on both committed and uncommitted locations.
	//
	// Use Purge when either a cache key has to be removed, or when an uncommitted key needs to be removed.
	Purge(location string) error

	// Rollback will leave a committed location in place, but will purge a location that has not been committed yet.
	// If the location has been committed, Rollback is a noop.
	//
	// Use Rollback when a cache key was retrieved with Get, an error occurred, and you want to leave an
	// old cache entry in place (if it exists) or delete the entry if nothing was ever committed.
	Rollback(location string) error
}