README
¶
UtahFS
UtahFS is a state-of-the art encrypted storage solution, meant to be similar to Dropbox. It has a FUSE binding that creates a synthetic drive on the user's computer that they can interact with like an external hard-drive. Files stored in the drive are uploaded to a cloud storage provider, which means the drive will never run out of space and minimizes the likelihood of any files being lost. However the files are encrypted such that the cloud storage provider knows almost nothing about what's being stored.
Features
- Interchangeable Storage Providers. For storing data in the cloud, UtahFS uses Object Storage, which is cheap and commodified. Example providers include: AWS S3, Google Cloud Storage, Backblaze B2, and Wasabi.
- Very Very Strong Encryption. The method of encryption hides the number of files, file names, file contents, individual file size, and prevents any modifications (including rollbacks!). The only information which is clearly leaked is the maximum archive size: archives grow to fit new data, but won't shrink if that data is deleted. Instead, that space is left allocated and will be re-used if needed in the future.
- Local Hardware allows Multiple Users and Improved Performance. If the user has server-like hardware on their LAN (Raspberry Pi / Intel NUC), this can be used to coordinate multiple users operating in the same archive. It can also dramatically improve the performance of uploads, because a user can upload large amounts of already-encrypted data to the server over the fast local network, and let the server take over the much slower upload to the cloud provider.
- Archive Mode. The client can be configured to guard against deleting or overwriting existing files, while still allowing new files to be created and old files to be moved around. This helps protect against accidental data loss.
- Oblivious RAM (ORAM). ORAM can be used to hide the access pattern of data from the cloud storage provider, so the provider only sees the amount data is accessed. (Access pattern: Which pieces of data are being accessed, and whether the access was a read or write.)
How to Use
Setup documentation can be found in the docs/ folder.
Future Work
- Reliability strategies for the WAL. Changes are buffered in a local Write-Ahead Log (WAL) before being sent to the cloud storage provider. If the disk that the WAL is stored on fails, it could become very difficult to return the archive to a usable, partially-regressed state.
Documentation
¶
Overview ¶
Package utahfs provides a FUSE binding where files are stored encrypted in the cloud.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewArchive ¶
func NewArchive(bfs *BlockFilesystem) (fuseutil.FileSystem, error)
NewArchive wraps NewFilesystem but refuses to delete or overwrite data.
It allows new files to be created, and old files to be moved / renamed / appended to. Empty directories may be deleted, but no files may be deleted or overwritten. This is just enforced by the FUSE binding, not by an actual access management system. Data stored is compatible with NewFilesystem.
func NewFilesystem ¶
func NewFilesystem(bfs *BlockFilesystem) (fuseutil.FileSystem, error)
NewFilesystem returns a FUSE binding that internally stores data in a block-based filesystem.
Types ¶
type BlockFile ¶
type BlockFile struct {
// contains filtered or unexported fields
}
BlockFile implements read-write functionality for a variable-size file over a skiplist of fixed-size blocks.
type BlockFilesystem ¶
type BlockFilesystem struct {
// contains filtered or unexported fields
}
BlockFilesystem implements large files as skiplists over fixed-size blocks stored in an object storage service.
func NewBlockFilesystem ¶
func NewBlockFilesystem(store *persistent.AppStorage, numPtrs, dataSize int64, splitPtrs bool) (*BlockFilesystem, error)
NewBlockFilesystem returns a new block-based filesystem. Blocks will have `numPtrs` pointers in their skiplist and contain at most `dataSize` bytes of application data.
Recommended values:
numPtrs = 12, dataSize = 32*1024
This system manages two pieces of global state:
- trash - Points to the first block of the trash list: a linked list of blocks which have been discarded and are free for re-allocation.
- next - The next unallocated pointer. A block with this pointer is created only if the trash list is empty.
`splitPtrs` is true if the pointers section of a block should be stored separately from the data section, and false if they should be stored together. Storing them separately can improve seek performance.
func (*BlockFilesystem) Create ¶
func (bfs *BlockFilesystem) Create(ctx context.Context, dt persistent.DataType) (uint64, *BlockFile, error)
Create creates a new file. It returns the pointer to the file and an open copy.
func (*BlockFilesystem) Open ¶
func (bfs *BlockFilesystem) Open(ctx context.Context, ptr uint64, dt persistent.DataType) (*BlockFile, error)
Open returns a handle to an existing file.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package cache implements a capped-size in-memory cache that randomly evicts elements when it reaches max size.
|
Package cache implements a capped-size in-memory cache that randomly evicts elements when it reaches max size. |
|
cmd
|
|
|
utahfs-client
Command utahfs-client provides a FUSE binding, backed by an encrypted object storage provider.
|
Command utahfs-client provides a FUSE binding, backed by an encrypted object storage provider. |
|
utahfs-server
Command utahfs-server acts as a cache to improve the performance of a UtahFS instance, and helps coordinate multiple users.
|
Command utahfs-server acts as a cache to improve the performance of a UtahFS instance, and helps coordinate multiple users. |
|
utahfs-web
Command utahfs-web provides access to a UtahFS repository through a Web UI.
|
Command utahfs-web provides access to a UtahFS repository through a Web UI. |
|
Package persistent implements several compatible object-storage backends, and additional functionality that can be layered upon them.
|
Package persistent implements several compatible object-storage backends, and additional functionality that can be layered upon them. |