catfs

package
v0.4.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 30, 2019 License: AGPL-3.0 Imports: 32 Imported by: 12

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrIsClosed is returned when an operation is performed on an already
	// closed file.
	ErrIsClosed = errors.New("File handle is closed")
)
View Source 
var ErrReadOnly = errors.New("fs is read only")

ErrReadOnly is returned when a file system was created in read only mode and a modifying operation was called on it.

Functions

This section is empty.

Types

type Change 

type Change struct {
	// Path is the node that was changed
	Path string

	// IsPinned tells you if the content is pinned at this stage
	IsPinned bool

	// IsExplicty tells you if the content is pinned explicitly.
	IsExplicit bool

	// Change describes what was changed
	Change string

	// MovedTo indicates that the node at this Path was moved to
	// another location and that there is no node at this location now.
	MovedTo string

	// WasPreviouslyAt is filled when the node was moved
	// and was previously at another location.
	WasPreviouslyAt string

	// Head is the commit after the change
	Head *Commit

	// Next is the commit before the change
	Next *Commit
}

Change describes a single change to a node between two versions

type Commit 

type Commit struct {
	// Hash is the id of this commit
	Hash h.Hash
	// Msg describes the committed contents
	Msg string
	// Tags is a user defined list of tags
	// (tags like HEAD, CURR and INIT are assigned dynamically as exception)
	Tags []string
	// Date is the time when the commit was made
	Date time.Time
	// Index is the index of the commit:
	Index int64
}

Commit gives information about a single commit.

type Diff 

type Diff struct {
	// Added is a list of nodes that were added newly
	Added []StatInfo `json:"added"`

	// Removed is a list of nodes that were removed
	Removed []StatInfo `json:"removed"`

	// Ignored is a list of nodes that were not considered
	Ignored []StatInfo `json:"ignored"`

	// Missing is a list of nodes that the remoe side is missing
	Missing []StatInfo `json:"missing"`

	// Moved is a list of nodes that changed path
	Moved []DiffPair `json:"moved"`

	// Merged is a list of nodes that can be merged automatically
	Merged []DiffPair `json:"merged"`

	// Conflict is a list of nodes that cannot be merged automatically
	Conflict []DiffPair `json:"conflict"`
}

Diff is a list of things that changed between to commits

type DiffPair 

type DiffPair struct {
	Src StatInfo `json:"src"`
	Dst StatInfo `json:"dst"`
}

DiffPair is a pair of nodes. It is returned by MakeDiff(), where the source is a node on the remote side and the dst node is a node on our side.

type ErrNoSuchHash 

type ErrNoSuchHash struct {
	// contains filtered or unexported fields
}

ErrNoSuchHash should be returned whenever the backend is unable to find an object referenced to by this hash.

func (ErrNoSuchHash) Error 

func (eh ErrNoSuchHash) Error() string

type ExplicitPin 

type ExplicitPin struct {
	Path   string
	Commit string
}

ExplicitPin is a pair of path and commit id.

type FS 

type FS struct {
	// contains filtered or unexported fields
}

FS (short for Filesystem) is the central API entry for everything related to paths. It exposes a POSIX-like interface where path are mapped to the actual underlying hashes and the associated metadata.

Additionally it supports version control commands like MakeCommit(), Checkout() etc. The API is file-centric, i.e. directories are created on the fly for some operations like Stage(). Empty directories can be created via Mkdir() though.

func NewFilesystem 

func NewFilesystem(backend FsBackend, dbPath string, owner string, readOnly bool, fsCfg *config.Config) (*FS, error)

NewFilesystem creates a new CATFS filesystem. This filesystem stores all its data in a Merkle DAG and is fully versioned.

func (*FS) ApplyPatch 

func (fs *FS) ApplyPatch(data []byte) error

ApplyPatch reads the binary patch coming from MakePatch and tries to apply it.

func (*FS) Cat 

func (fs *FS) Cat(path string) (mio.Stream, error)

Cat will open a file read-only and expose it's underlying data as stream. If no such path is known or it was deleted, nil is returned as stream.

func (*FS) Checkout 

func (fs *FS) Checkout(rev string, force bool) error

Checkout reverts all state to the commit referenced by `rev`. If `force` is true a non-empty staging area will be overwritten.

func (*FS) Close 

func (fs *FS) Close() error

Close will clean up internal storage.

func (*FS) CommitInfo added in v0.2.0 

func (fs *FS) CommitInfo(rev string) (*Commit, error)

CommitInfo returns detailed info about a certain commit.

func (*FS) Copy 

func (fs *FS) Copy(src, dst string) error

Copy will copy the file or directory at `src` to `dst`. If it does not exist, an error will be returned.

func (*FS) Curr added in v0.2.0 

func (fs *FS) Curr() (string, error)

Curr translates the "curr" symbol to a ref.

func (*FS) DeletedNodes added in v0.4.0 

func (fs *FS) DeletedNodes(root string) ([]*StatInfo, error)

DeletedNodes returns all nodes under `root` that were deleted. This does not include files that were moved. Note that you cannot pass the paths of those files to methods like Cat(), since they will refuse to work on deleted files.

func (*FS) Export 

func (fs *FS) Export(w io.Writer) error

Export will export a serialized version of the filesystem to `w`.

func (*FS) FilesByContent 

func (fs *FS) FilesByContent(contents []h.Hash) (map[string]StatInfo, error)

FilesByContent returns all stat info for the content hashes referenced in `contents`. The return value is a map with the content hash as key and a StatInfo describing the exact file content.

func (*FS) Filter added in v0.4.0 

func (fs *FS) Filter(root, query string) ([]*StatInfo, error)

Filter implements a quick and easy way to search over all files by using a query that checks if it is part of the path.

func (*FS) HaveStagedChanges added in v0.4.0 

func (fs *FS) HaveStagedChanges() (bool, error)

HaveStagedChanges returns true if there are changes that were not committed yet.

func (*FS) Head 

func (fs *FS) Head() (string, error)

Head translates the "head" symbol to a ref.

func (*FS) History 

func (fs *FS) History(path string) ([]Change, error)

History returns all modifications of a node with one entry per commit.

func (*FS) Import 

func (fs *FS) Import(r io.Reader) error

Import will read a previously FS dump from `r`.

func (*FS) IsCached added in v0.4.1 

func (fs *FS) IsCached(path string) (bool, error)

IsCached will return true when the file is cached locally.

func (*FS) IsPinned 

func (fs *FS) IsPinned(path string) (bool, bool, error)

IsPinned returns true for files and directories that are pinned. A directory only counts as pinned if all files and directories in it are also pinned.

func (*FS) LastPatchIndex 

func (fs *FS) LastPatchIndex() (int64, error)

LastPatchIndex will return the current version of this filesystem regarding patch state.

func (*FS) List 

func (fs *FS) List(root string, maxDepth int) ([]*StatInfo, error)

List returns stat info for each node below (and including) root. Nodes deeper than maxDepth will not be shown. If maxDepth is a negative number, all nodes will be shown.

func (*FS) Log 

func (fs *FS) Log(head string, fn func(c *Commit) error) error

Log returns a list of commits starting with the staging commit until the initial commit. For each commit, metadata is collected and fn is called. The log starts at the revision pointed to by `head`. If `head` is an empty string, "curr" is assumed.

func (*FS) MakeCommit 

func (fs *FS) MakeCommit(msg string) error

MakeCommit bundles all staged changes into one commit described by `msg`. If no changes were made since the last call to MakeCommit() ErrNoConflict is returned.

func (*FS) MakeDiff 

func (fs *FS) MakeDiff(remote *FS, headRevOwn, headRevRemote string) (*Diff, error)

MakeDiff will return a diff between `headRevOwn` and `headRevRemote`. `remote` is the filesystem `headRevRemote` belongs to and may be the same as `fs`.

func (*FS) MakePatch 

func (fs *FS) MakePatch(fromRev string, folders []string, remoteName string) ([]byte, error)

MakePatch creates a binary patch with all file changes starting with `fromRev`. Note that commit information is not exported, only individual file and directory changes.

The byte structured returned by this method may change at any point and may not be relied upon.

The `remoteName` is the name of the remote we're creating the patch for. It's only used for display purpose in the commit message.

func (*FS) Mkdir 

func (fs *FS) Mkdir(dir string, createParents bool) error

Mkdir creates a new empty directory at `dir`, possibly creating all intermediate parents if `createParents` is set.

func (*FS) Move 

func (fs *FS) Move(src, dst string) error

Move will move the file or directory at `src` to `dst`. If it does not exist, an error will be returned.

func (*FS) Open 

func (fs *FS) Open(path string) (*Handle, error)

Open returns a file like object that can be used for modifying a file in memory. If you want to have seekable read-only stream, use Cat(), it has less overhead.

func (*FS) Pin 

func (fs *FS) Pin(path, rev string, explicit bool) error

Pin will pin the file or directory at `path` explicitly.

func (*FS) Remove 

func (fs *FS) Remove(path string) error

Remove removes the file or directory at `path`.

func (*FS) RemoveTag 

func (fs *FS) RemoveTag(name string) error

RemoveTag removes a previously created tag.

func (*FS) Repin added in v0.4.0 

func (fs *FS) Repin(root string) error

Repin goes over all files in the filesystem and identifies files that need to be unpinned. Only files that are not explicitly pinned, are touched. If a file is explicitly pinned, it will survive the repinning process in any case. The repinning is steered by two config variables:

- fs.repin.quota: Maximum amount of pinned storage (excluding explicit pins) - fs.repin.depth: How many versions of a file to keep at least. This trumps quota.

func (*FS) Reset 

func (fs *FS) Reset(path, rev string) error

Reset restores the state of `path` to the state in `rev`.

func (*FS) ScheduleGCRun 

func (fs *FS) ScheduleGCRun()

ScheduleGCRun runs GC run at the next possible time. This method does not block until the run is finished.

func (*FS) Stage 

func (fs *FS) Stage(path string, r io.ReadSeeker) error

Stage reads all data from `r` and stores as content of the node at `path`. If `path` already exists, it will be updated.

func (*FS) Stat 

func (fs *FS) Stat(path string) (*StatInfo, error)

Stat delivers detailed information about the node at `path`.

func (*FS) Sync 

func (fs *FS) Sync(remote *FS, options ...SyncOption) error

Sync will synchronize the state of two filesystems. If one of filesystems have unstaged changes, they will be committted first. If our filesystem was changed by Sync(), a new merge commit will also be created.

func (*FS) Tag 

func (fs *FS) Tag(rev, name string) error

Tag saves a human readable name for the revision pointed to by `rev`. There are three pre-defined tags available:

- HEAD: The last full commit. - CURR: The current commit (== staging commit) - INIT: the initial commit.

The tagname is case-insensitive.

func (*FS) Tar added in v0.2.0 

func (fs *FS) Tar(root string, w io.Writer, filter func(node *StatInfo) bool) error

Tar produces a tar archive from the file or directory at `root` and writes the output to `w`. If you want compression, supply a gzip writer.

func (*FS) Touch 

func (fs *FS) Touch(path string) error

Touch creates an empty file at `path` if it does not exist yet. If it exists, it's mod time is being updated to the current time.

func (*FS) Truncate 

func (fs *FS) Truncate(path string, size uint64) error

Truncate cuts of the output of the file at `path` to `size`. `size` should be between 0 and the size of the file, all other values will be ignored.

Note that this is not implemented as an actual IO operation. It is possible to go back to a bigger size until the actual content was changed via Stage().

func (*FS) Undelete added in v0.4.0 

func (fs *FS) Undelete(root string) error

Undelete tries to recover a file or directory that was previously deleted. This will fail when being called on a regular file or directory. You can obtain deleted paths by using DeletedNodes()

func (*FS) Unpin 

func (fs *FS) Unpin(path, rev string, explicit bool) error

Unpin will unpin the file or directory at `path` explicitly.

type FsBackend 

type FsBackend interface {
	// Cat should find the object referenced to by `hash` and
	// make its data available as mio.Stream.
	Cat(hash h.Hash) (mio.Stream, error)

	// Add should read all data in `r` and return the hash under
	// which it can be accessed on later.
	Add(r io.Reader) (h.Hash, error)

	// Pin gives the object at `hash` a "pin".
	// (i.e. it marks the file to be stored indefinitely in local storage)
	// When pinning an explicit pin with an implicit pin, the explicit pin
	// will stay. Upgrading from implicit to explicit is possible though.
	Pin(hash h.Hash) error

	// Unpin removes a previously added pin.
	// If an object is already unpinned this is a no op.
	Unpin(hash h.Hash) error

	// IsPinned checks if the file is pinned.
	IsPinned(hash h.Hash) (bool, error)

	// IsCached checks if the file contents are available locally.
	IsCached(hash h.Hash) (bool, error)
}

FsBackend is the interface that needs to be implemented by the data management layer.

type Handle 

type Handle struct {
	// contains filtered or unexported fields
}

Handle is a emulation of a os.File handle, as returned by os.Open() It supports the usual operations like open, read, write and seek. Take note that the flushing operation currently is quite expensive.

func (*Handle) Close 

func (hdl *Handle) Close() error

Close will finalize the file. It should not be used after. This will call flush if it did not happen yet.

func (*Handle) Flush 

func (hdl *Handle) Flush() error

Flush makes sure to write the current state to the backend. Please remember that this method is currently pretty expensive.

func (*Handle) Path 

func (hdl *Handle) Path() string

Path returns the absolute path of the file.

func (*Handle) Read 

func (hdl *Handle) Read(buf []byte) (int, error)

Read will try to fill `buf` as much as possible. The seek pointer will be advanced by the number of bytes written. Take care, `buf` might still have contents, even if io.EOF was returned.

func (*Handle) Seek 

func (hdl *Handle) Seek(offset int64, whence int) (int64, error)

Seek will jump to the `offset` relative to `whence`. There next read and write operation will start from this point.

func (*Handle) Truncate 

func (hdl *Handle) Truncate(size uint64) error

Truncate truncates the file to a specific length.

func (*Handle) Write 

func (hdl *Handle) Write(buf []byte) (int, error)

Write will write the contents of `buf` to the current position. It will return the number of currently written bytes.

type MemFsBackend 

type MemFsBackend struct {
	// contains filtered or unexported fields
}

MemFsBackend is a mock structure that implements FsBackend.

func NewMemFsBackend 

func NewMemFsBackend() *MemFsBackend

NewMemFsBackend returns a MemFsBackend (useful for writing tests)

func (*MemFsBackend) Add 

func (mb *MemFsBackend) Add(r io.Reader) (h.Hash, error)

Add implements FsBackend.Add by storing the data in memory.

func (*MemFsBackend) Cat 

func (mb *MemFsBackend) Cat(hash h.Hash) (mio.Stream, error)

Cat implements FsBackend.Cat by querying memory.

func (*MemFsBackend) IsCached added in v0.4.1 

func (mb *MemFsBackend) IsCached(hash h.Hash) (bool, error)

IsCached implements FsBackend.IsCached by checking if the file exists. If yes, the file is cached always.

func (*MemFsBackend) IsPinned 

func (mb *MemFsBackend) IsPinned(hash h.Hash) (bool, error)

IsPinned implements FsBackend.IsPinned by querying a marker in memory.

func (*MemFsBackend) Pin 

func (mb *MemFsBackend) Pin(hash h.Hash) error

Pin implements FsBackend.Pin by storing a marker in memory.

func (*MemFsBackend) Unpin 

func (mb *MemFsBackend) Unpin(hash h.Hash) error

Unpin implements FsBackend.Unpin by removing a marker in memory.

type Pinner 

type Pinner struct {
	// contains filtered or unexported fields
}

Pinner remembers which hashes are pinned and if they are pinned explicitly. Its API can be used to safely change the pinning state. It assumes that it is the only entitiy the pins & unpins nodes.

func NewPinner 

func NewPinner(lkr *c.Linker, bk FsBackend) (*Pinner, error)

NewPinner creates a new pin cache at `pinDbPath`, possibly erroring out. `lkr` and `bk` are used to make PinNode() and UnpinNode() work.

func (*Pinner) Close 

func (pc *Pinner) Close() error

Close the pinning cache.

func (*Pinner) IsNodePinned 

func (pc *Pinner) IsNodePinned(nd n.Node) (bool, bool, error)

IsNodePinned checks if all `nd` is pinned and if so, exlusively. If `nd` is a directory, it will only return true if all children are also pinned (same for second return value).

func (*Pinner) IsPinned 

func (pc *Pinner) IsPinned(inode uint64, hash h.Hash) (bool, bool, error)

IsPinned returns two boolean values indicating the pin status of `inode` and `hash`. If the first value is true, the content is pinned, if the second is true it is pinned explicitly.

func (*Pinner) Pin 

func (pc *Pinner) Pin(inode uint64, hash h.Hash, explicit bool) error

Pin will remember the node at `inode` with hash `hash` as `explicit`ly pinned.

func (*Pinner) PinNode 

func (pc *Pinner) PinNode(nd n.Node, explicit bool) error

PinNode tries to pin the node referenced by `nd`. The difference to calling Pin(nd.BackendHash()) is, that this method will pin directories recursively, if given.

If the file is already pinned exclusively and you want to pin it non-exclusive, this will be a no-op. In this case you have to unpin it first exclusively.

func (*Pinner) Unpin 

func (pc *Pinner) Unpin(inode uint64, hash h.Hash, explicit bool) error

Unpin pins the content at `inode` and `hash`. If the pin was explicit, `explicit` must be true to make this work.

func (*Pinner) UnpinNode 

func (pc *Pinner) UnpinNode(nd n.Node, explicit bool) error

UnpinNode is the exact opposite of PinNode.

type StatInfo 

type StatInfo struct {
	// Path is the full path to the file
	Path string

	// TreeHash is the hash of the node in the DAG
	TreeHash h.Hash
	// ContentHash is the actual hash of the content
	// (used to test for content equality)
	ContentHash h.Hash
	// BackendHash is the hash under which the file is reachable
	// in the backend.
	BackendHash h.Hash

	// User is the name of the user that modified this node last.
	User string
	// Size in bytes
	Size uint64
	// Inode is a unique number specific to this node
	Inode uint64
	// Depth is the hierarchy level inside of this node (root has 0)
	Depth int
	// ModTime is the last modification timestamp
	ModTime time.Time

	// IsDir tells you if this node is a dir
	IsDir bool
	// IsPinned tells you if this node is pinned (either implicit or explicit)
	IsPinned bool
	// IsExplicit is true when the user pinned this node on purpose
	IsExplicit bool
}

StatInfo describes the metadata of a single node. The concept is comparable to the POSIX stat() call.

type SyncOption added in v0.4.0 

type SyncOption func(cfg *vcs.SyncOptions)

SyncOption is a option that can be passed to Sync.

func SyncOptConflictStrategy added in v0.4.0 

func SyncOptConflictStrategy(strategy string) SyncOption

SyncOptConflictStrategy overwrites the conflict strategy (see also fs.sync.conflict_strategy which acts as default)

func SyncOptConflictgStrategyPerFolder added in v0.4.0 

func SyncOptConflictgStrategyPerFolder(strategies map[string]string) SyncOption

SyncOptConflictgStrategyPerFolder allows you to set a specific conflict resolution strategy for specific folders. The key of the map is the folder, the key is the conflict strategy name.

func SyncOptMessage added in v0.4.0 

func SyncOptMessage(msg string) SyncOption

SyncOptMessage sets the commit message that will be given to MakeCommit() on a sync commit.

func SyncOptReadOnlyFolders added in v0.4.0 

func SyncOptReadOnlyFolders(folders []string) SyncOption

SyncOptReadOnlyFolders allows you to set a set of folders that will be protected from modifications by the sync.

Source Files

View all Source files

Directories

Path Synopsis
mio
Package mio (short for memory input/output) implements the layered io stack of brig.
 Package mio (short for memory input/output) implements the layered io stack of brig.
chunkbuf
compress
encrypt
Package encrypt implements the encryption layer of brig.
 Package encrypt implements the encryption layer of brig.
overlay
tool
Package nodes implements all nodes and defines basic operations on it.
 Package nodes implements all nodes and defines basic operations on it.
capnp
vcs
capnp

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.