localcache

README

Local file-based atomic cache manager

I repeatedly find myself writing partitioned keyed caching systems, akin to Go's module cache. To DRY myself I created this.

It provides:

• Partitioned cache entries: <root>/<partition>/<key>
• Atomic creation, replacement and deletion of single files.
• Atomic creation, replacement and deletion of directory hierarchies.

Usage

cache, err := localcache.New("myapp")

// Create a temporarily unaddressable file in the cache.
tx, f, err := cache.Create("some-key")
// Write to f

// Commit the file to the cache.
err = cache.Commit(tx)

// Load the cache entry back.
data, err := cache.ReadFile("some-key")

// Open the cache entry for reading.
data, err := cache.Open("some-key")

// Create a temporarily unaddressable directory with the same key as the previous file.
tx, dir, err := cache.Mkdir("some-key")

// Atomically replace the previous file with the new directory.
err := cache.Commit(tx)

Implementation

The cache manager maintains transactionality/atomicity by relying on two aspects of Unix filesystems:

1. File renames are atomic.
2. Symlinks can be atomically overwritten by a rename.

The process is then:

1. Create a file or directory F = <partition>/<hash>.<timestamp>.
2. User writes to the file or populates the directory.
3. Create a symlink L = <partition>/<hash>.<timestamp> -> F
4. Rename L to <partition>/<hash>, the final "committed" name for the entry.

eg.

Code	Filesystem
tx, f, err := cache.Create("my-key") f.WriteString("hello") f.Close()	5e/5e78…f732.67e7996297ee
Step 1 cache.Commit(tx)	5e/5e78…f732.67e799629823 -> 5e78…f732.67e7996297ee 5e/5e78…f732.67e7996297ee
Step 2 cache.Commit(tx)	5e/5e78…f732 -> 5e78…f732.67e7996297ee 5e/5e78…f732.67e7996297ee

Documentation

Index

• type Cache
  • func New(name string) (*Cache, error)
  • func NewForTesting(t testing.TB) *Cache
  • func (c *Cache) Commit(tx Transaction) (string, error)
  • func (c *Cache) Create(key string) (Transaction, *os.File, error)
  • func (c *Cache) CreateOrRead(key string) (Transaction, *os.File, error)
  • func (c *Cache) IfExists(key string) string
  • func (c *Cache) Mkdir(key string) (Transaction, string, error)
  • func (c *Cache) Open(key string) (*os.File, error)
  • func (c *Cache) Purge(older time.Duration) error
  • func (c *Cache) ReadFile(key string) ([]byte, error)
  • func (c *Cache) Remove(key string) error
  • func (c *Cache) Rollback(tx Transaction) error
  • func (c *Cache) RollbackOnError(tx Transaction, err *error)
  • func (c *Cache) RollbackOrCommit(tx Transaction, err *error)
  • func (c *Cache) WriteFile(key string, data []byte) (err error)
• type Transaction
  • func (t Transaction) Valid() bool

Types

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Cache type.

func New

func New(name string) (*Cache, error)

New creates a new cache "name" under the user's cache directory.

func NewForTesting

func NewForTesting(t testing.TB) *Cache

NewForTesting creates a new Cache for testing.

The Cache will be removed on test completion.

func (*Cache) Commit

func (c *Cache) Commit(tx Transaction) (string, error)

Commit atomically commits an in-flight file or directory creation Transaction to the Cache.

func (*Cache) Create

func (c *Cache) Create(key string) (Transaction, *os.File, error)

Create a file in the Cache.

Commit() must be called with the returned Transaction to atomically add the created file to the Cache.

tx, f, err := cache.Create("my-key")
err = f.Close()
err = cache.Commit(tx)

func (*Cache) CreateOrRead

func (c *Cache) CreateOrRead(key string) (Transaction, *os.File, error)

CreateOrRead creates a key if it doesn't exist, or opens it for reading if it does.

Use Transaction.Valid() to check if the key was created.

func (*Cache) IfExists

func (c *Cache) IfExists(key string) string

IfExists returns the path to a cache entry if it exists, or empty string if it does not.

func (*Cache) Mkdir

func (c *Cache) Mkdir(key string) (Transaction, string, error)

Mkdir creates a directory in the cache.

Commit() must be called with the returned Transaction to atomically add the created directory to the Cache.

tx, dir, err := cache.Mkdir("my-key")
err = cache.Commit(tx)

func (*Cache) Open

func (c *Cache) Open(key string) (*os.File, error)

Open a file or directory in the Cache.

func (*Cache) Purge

func (c *Cache) Purge(older time.Duration) error

Purge all entries older than the given age.

func (*Cache) ReadFile

func (c *Cache) ReadFile(key string) ([]byte, error)

ReadFile identified by key.

func (*Cache) Remove

func (c *Cache) Remove(key string) error

Remove cache entry atomically.

func (*Cache) Rollback

func (c *Cache) Rollback(tx Transaction) error

Rollback reverts an in-flight file or directory creation Transaction.

func (*Cache) RollbackOnError

func (c *Cache) RollbackOnError(tx Transaction, err *error)

RollbackOnError is a convenience method for use with defer.

It will Rollback on error, however Commit must be called manually.

defer cache.RollbackOnError(tx, &err)

func (*Cache) RollbackOrCommit

func (c *Cache) RollbackOrCommit(tx Transaction, err *error)

RollbackOrCommit is a convenience method for use with defer.

It will Rollback on error or otherwise Commit.

defer cache.RollbackOrCommit(tx, &err)

func (*Cache) WriteFile

func (c *Cache) WriteFile(key string, data []byte) (err error)

WriteFile writes a byte slice to a file in the cache.

type Transaction

type Transaction string

Transaction key for an uncommitted cache entry.

func (Transaction) Valid

func (t Transaction) Valid() bool

Valid returns true if the Transaction is valid.