filestore

README

Filestore

a versioning local file storage system that avoids copying files

A local filestore allows you to copy files to a storage location on harddisk, keep track of changes manually (under your control), and retrieve older versions of files that have been stored. It stores unmodified copies and remembers metadata in an Sqlite3 database with FTS search.

Usage

Not available yet. This package is in early alpha state and ought not be used.

Dependencies

Filestore uses sqlite3 with the driver by mattn and this will not change. You probably shouldn't use it if you use another Sqlite driver.

Implementation

The filestore is implemented following the K.I.S.S. principle. The source code should be easy to read and unsurprising. Please note that file changes are detected by computing a Blake2b checksum over whole files. This is not fast at all for large files and the Filestore ought not be used when performance is needed for checking whether a copy needs to be made. You should use a different solution with OS-level change tracking if you have this requirement.

If you find bugs or problems in a non-alpha version, please let me know by raising an issue.

Documentation

Index

• Constants
• Variables
• type FileVersion
• type Filestore
  • func NewFilestore(root string, options flags.Bits) *Filestore
  • func (fs *Filestore) Add(path, info, version string) error
  • func (fs *Filestore) Checksum(path string) (string, error)
  • func (fs *Filestore) Close() error
  • func (fs *Filestore) FuzzySearch(term string, limit int) ([]FileVersion, error)
  • func (fs *Filestore) Get(path string) (FileVersion, error)
  • func (fs *Filestore) Has(file string) bool
  • func (fs *Filestore) Open() error
  • func (fs *Filestore) Restore(version FileVersion, dst string) error
  • func (fs *Filestore) RestoreAtSource(version FileVersion) error
  • func (fs *Filestore) Root() string
  • func (fs *Filestore) Search(term string, limit int) ([]FileVersion, error)
  • func (fs *Filestore) SimpleSearch(words []string, limit int) ([]FileVersion, error)
  • func (fs *Filestore) Versions(path string, limit int) ([]FileVersion, error)
  • func (fs *Filestore) VersionsAfter(path string, after time.Time, limit int) ([]FileVersion, error)

Constants

const Compress = flags.Flag0 // if option is set, then files are compressed with Snappy

Variables

var ErrDirectoryIsFile = errors.New("directory cannot be created because it is a file")

var ErrInvalidDate = errors.New("filestore entry contains invalid date")

var ErrNotOpen = errors.New("filestore is not open")

Types

type FileVersion

type FileVersion struct {
	ID       int64     // file version ID (internal)
	Name     string    // the name of the file, including suffix
	Path     string    // the path from which the version was sourced (os path)
	Local    string    // the path to the file content on disk in the local filestore (os path)
	Info     string    // the info string
	Fuzzy    string    // fuzzy into string
	Version  string    // the version string
	From     time.Time // the datetime on which this version was added
	Checksum string    // the hex-encoded Blake2b checksum of the file contents of this version
}

FileVersion represents a particular version of a file.

type Filestore

type Filestore struct {
	Dir     string     // the root directory under which versions are stored
	Options flags.Bits // flag options for configuring the filestore
	// contains filtered or unexported fields
}

Filestore stores different versions of a file on the local hard disk and allows you to retrieve them by path or global FileID.

func NewFilestore

func NewFilestore(root string, options flags.Bits) *Filestore

NewFilestore returns a new filestore based on the given root directory and options.

func (*Filestore) Add

func (fs *Filestore) Add(path, info, version string) error

Add adds a file with given path or updates the existing entries for the file. The file is versioned and a version stored with the given info, tag strings and semantic version.

func (*Filestore) Checksum

func (fs *Filestore) Checksum(path string) (string, error)

Checksum computes a 512 byte Blake2b checksum of a given file.

func (*Filestore) Close

func (fs *Filestore) Close() error

Close closes the filestore and frees associated resources.

func (*Filestore) FuzzySearch

func (fs *Filestore) FuzzySearch(term string, limit int) ([]FileVersion, error)

FuzzySearch performs an FTS5 term search on the database directly. This requires some knowledge of the database organization and FTS5 queries.

func (*Filestore) Get

func (fs *Filestore) Get(path string) (FileVersion, error)

Get returns the latest version of a file at path, or an error if the file is not in the filestore.

func (*Filestore) Has

func (fs *Filestore) Has(file string) bool

Has returns true if versions of the file given by the filepath exist, false otherwise.

func (*Filestore) Open

func (fs *Filestore) Open() error

Open opens the filestore and prepares it for access.

func (*Filestore) Restore

func (fs *Filestore) Restore(version FileVersion, dst string) error

Restore restores the given file version to destination directory dst.

func (*Filestore) RestoreAtSource

func (fs *Filestore) RestoreAtSource(version FileVersion) error

RestoreAtSource restores the version into the original source destination path from which it was created. If a file already exists at this place (normally the case), it will be overwritten.

func (*Filestore) Root

func (fs *Filestore) Root() string

Root returns the root directory, ending in a directory separator unless it is an empty relative directory (== the current directory).

func (*Filestore) Search

func (fs *Filestore) Search(term string, limit int) ([]FileVersion, error)

Search performs an FTS5 term search on the database directly. This requires some knowledge of the database organization and FTS5 queries.

func (*Filestore) SimpleSearch

func (fs *Filestore) SimpleSearch(words []string, limit int) ([]FileVersion, error)

SimpleSearch returns FileVersion entries for all file info strings starting with terms, combined with OR but sorted from more to less matching entries.

func (*Filestore) Versions

func (fs *Filestore) Versions(path string, limit int) ([]FileVersion, error)

Versions returns FileVersion entries for all versions of a file. Nil is returned if there are no versions.

func (*Filestore) VersionsAfter

func (fs *Filestore) VersionsAfter(path string, after time.Time, limit int) ([]FileVersion, error)

VersionsAfter returns FileVersion entries for all versions of a file after the given date. Nil is returned if there are no versions.