Documentation
¶
Overview ¶
Package vfs is for storing files on the cozy, including binary ones like photos and movies. The range of possible operations with this endpoint goes from simple ones, like uploading a file, to more complex ones, like renaming a directory. It also ensure that an instance is not exceeding its quota, and keeps a trash to recover files recently deleted.
Index ¶
- Constants
- Variables
- func Allows(fs VFS, pset permission.Set, v permission.Verb, fd Fetcher) error
- func CheckAvailableDiskSpace(fs VFS, doc *FileDoc) (newsize, maxsize, capsize int64, err error)
- func ConflictName(fs VFS, dirID, name string, isFile bool) string
- func ContentDisposition(disposition, filename string) string
- func DirExists(fs VFS, name string) (bool, error)
- func DiskQuotaAfterDestroy(fs VFS, diskUsageBeforeWrite, destroyed int64)
- func Exists(fs VFS, name string) (bool, error)
- func ExtractMimeAndClass(contentType string) (mime, class string)
- func ExtractMimeAndClassFromFilename(name string) (mime, class string)
- func FilterNotSynchronizedDocs(fs VFS, clientID string, changes *couchdb.ChangesResponse) error
- func FindVersionsToClean(db Prefixer, fileID string, candidate *Version) (ActionForCandidateVersion, []*Version, error)
- func MergeMetadata(doc *FileDoc, meta Metadata)
- func OptionsAllowCreationInTrash(opts []CreateOptions) bool
- func PushDiskQuotaAlert(fs VFS, exceeded bool)
- func RegisterDiskQuotaAlertCallback(cb func(domain string, exceeded bool))
- func Remove(fs VFS, name string, push func(TrashJournal) error) error
- func RemoveAll(fs VFS, name string, push func(TrashJournal) error) error
- func SameReferences(a, b []couchdb.DocReference) bool
- func ServeFileContent(fs VFS, doc *FileDoc, version *Version, filename, disposition string, ...) error
- func ServePDFIcon(w http.ResponseWriter, req *http.Request, fs VFS, doc *FileDoc) error
- func ServePDFPage(fs VFS, doc *FileDoc, disposition string, page int, req *http.Request, ...) error
- func ServePDFPreview(w http.ResponseWriter, req *http.Request, fs VFS, doc *FileDoc) error
- func SetMetaFromVersion(file *FileDoc, version *Version)
- func Stat(fs VFS, name string) (os.FileInfo, error)
- func Walk(fs Indexer, root string, walkFn WalkFn) error
- func WalkAlreadyLocked(fs Indexer, dir *DirDoc, walkFn WalkFn) error
- func WalkByID(fs Indexer, fileID string, walkFn WalkFn) error
- type ActionForCandidateVersion
- type Archive
- func (a *Archive) Clone() couchdb.Doc
- func (a *Archive) DocType() string
- func (a *Archive) GetEntries(fs VFS) ([]ArchiveEntry, error)
- func (a *Archive) ID() string
- func (a *Archive) Rev() string
- func (a *Archive) Serve(fs VFS, w http.ResponseWriter) error
- func (a *Archive) SetID(_ string)
- func (a *Archive) SetRev(_ string)
- type ArchiveEntry
- type AudioExtractor
- type Avatarer
- type CozyMetadataPatch
- type CreateOptions
- type DirDoc
- func Mkdir(fs VFS, name string, tags []string) (*DirDoc, error)
- func MkdirAll(fs VFS, name string) (*DirDoc, error)
- func ModifyDirMetadata(fs VFS, olddoc *DirDoc, patch *DocPatch) (*DirDoc, error)
- func NewDirDoc(index Indexer, name, dirID string, tags []string) (*DirDoc, error)
- func NewDirDocWithParent(name string, parent *DirDoc, tags []string) (*DirDoc, error)
- func NewDirDocWithPath(name, dirID, dirPath string, tags []string) (*DirDoc, error)
- func RestoreDir(fs VFS, olddoc *DirDoc) (*DirDoc, error)
- func TrashDir(fs VFS, olddoc *DirDoc) (*DirDoc, error)
- func (d *DirDoc) AddNotSynchronizedOn(refs ...couchdb.DocReference)
- func (d *DirDoc) AddReferencedBy(ri ...couchdb.DocReference)
- func (d *DirDoc) Clone() couchdb.Doc
- func (d *DirDoc) DocType() string
- func (d *DirDoc) Fetch(field string) []string
- func (d *DirDoc) ID() string
- func (d *DirDoc) IsDir() bool
- func (d *DirDoc) IsEmpty(fs VFS) (bool, error)
- func (d *DirDoc) ModTime() time.Time
- func (d *DirDoc) Mode() os.FileMode
- func (d *DirDoc) Name() string
- func (d *DirDoc) Parent(fs VFS) (*DirDoc, error)
- func (d *DirDoc) Path(fs FilePather) (string, error)
- func (d *DirDoc) RemoveNotSynchronizedOn(refs ...couchdb.DocReference)
- func (d *DirDoc) RemoveReferencedBy(ri ...couchdb.DocReference)
- func (d *DirDoc) Rev() string
- func (d *DirDoc) SetID(id string)
- func (d *DirDoc) SetRev(rev string)
- func (d *DirDoc) Size() int64
- func (d *DirDoc) Sys() interface{}
- type DirIterator
- type DirOrFileDoc
- type DiskThresholder
- type DocPatch
- type ExifExtractor
- type Fetcher
- type File
- type FileDoc
- func CreateFileDocCopy(doc *FileDoc, newDirID, copyName string) *FileDoc
- func ModifyFileMetadata(fs VFS, olddoc *FileDoc, patch *DocPatch) (*FileDoc, error)
- func NewFileDoc(name, dirID string, size int64, md5Sum []byte, mimeType, class string, ...) (*FileDoc, error)
- func RestoreFile(fs VFS, olddoc *FileDoc) (*FileDoc, error)
- func TrashFile(fs VFS, olddoc *FileDoc) (*FileDoc, error)
- func (f *FileDoc) AddReferencedBy(ri ...couchdb.DocReference)
- func (f *FileDoc) Clone() couchdb.Doc
- func (f *FileDoc) DocType() string
- func (f *FileDoc) Fetch(field string) []string
- func (f *FileDoc) ID() string
- func (f *FileDoc) IsDir() bool
- func (f *FileDoc) ModTime() time.Time
- func (f *FileDoc) Mode() os.FileMode
- func (f *FileDoc) Name() string
- func (f *FileDoc) Parent(fs VFS) (*DirDoc, error)
- func (f *FileDoc) Path(fp FilePather) (string, error)
- func (f *FileDoc) RemoveReferencedBy(ri ...couchdb.DocReference)
- func (f *FileDoc) ResetFullpath()
- func (f *FileDoc) Rev() string
- func (f *FileDoc) SetID(id string)
- func (f *FileDoc) SetRev(rev string)
- func (f *FileDoc) Size() int64
- func (f *FileDoc) Sys() interface{}
- type FilePather
- type FilesCozyMetadata
- type Fs
- type FsckContentMismatch
- type FsckLog
- type FsckLogType
- type ImageExtractor
- type Indexer
- type IteratorOptions
- type MetaExtractor
- type Metadata
- type Page
- type Prefixer
- type ShortcutExtractor
- type Store
- type ThumbFiler
- type Thumbser
- type TrashJournal
- type Tree
- type TreeFile
- type UploadedByEntry
- type VFS
- type Version
- func (v *Version) Clone() couchdb.Doc
- func (v *Version) DocType() string
- func (v *Version) ID() string
- func (v *Version) Included() []jsonapi.Object
- func (v *Version) Links() *jsonapi.LinksList
- func (v *Version) Relationships() jsonapi.RelationshipMap
- func (v *Version) Rev() string
- func (v *Version) SetID(id string)
- func (v *Version) SetRev(rev string)
- type WalkFn
Constants ¶
const ( // IndexMissingRoot is used when the index does not have a root object IndexMissingRoot FsckLogType = "index_missing_root" // IndexMissingTrash is used when the index does not have a trash folder IndexMissingTrash FsckLogType = "index_missing_trash" // IndexOrphanTree used when a part of the tree is detached from the main // root of the index. IndexOrphanTree FsckLogType = "index_orphan_tree" // IndexBadFullpath used when a directory does not have the correct path // field given its position in the index. IndexBadFullpath FsckLogType = "index_bad_fullpath" // FSMissing used when a file data is missing on the filesystem from its // index entry. FSMissing FsckLogType = "filesystem_missing" // IndexMissing is used when the index entry is missing from a file data. IndexMissing FsckLogType = "index_missing" // TypeMismatch is used when a document type does not match in the index and // underlying filesystem. TypeMismatch FsckLogType = "type_mismatch" // ContentMismatch is used when a document content checksum does not match // with the one in the underlying fs. ContentMismatch FsckLogType = "content_mismatch" // FileMissing is used when a version is present for a file that is not in // the index. FileMissing FsckLogType = "file_missing" // IndexFileWithPath is used when a file has a path in the index (only // directories should have one). IndexFileWithPath = "index_file_with_path" // IndexDuplicateName is used when two files or directories have the same // name inside the same folder (ie they have the same path). IndexDuplicateName = "index_duplicate_name" // TrashedNotInTrash is used when a file has trashed: true but its parent // directory is not the trash, not is in the trash. TrashedNotInTrash = "trashed_not_in_trash" // NotTrashedInTrash is used when a file has trashed: false but its parent // directory is the trash or a directory in the trash. NotTrashedInTrash = "not_trashed_in_trash" // ConflictInIndex is used when there is a conflict in CouchDB with 2 // branches of revisions. ConflictInIndex = "conflict_in_index" // ThumbnailWithNoFile is used when there is a thumbnail but not the file // that was used to create it. ThumbnailWithNoFile = "thumbnail_with_no_file" )
const ( // TrashDirName is the path of the trash directory TrashDirName = "/.cozy_trash" // ThumbsDirName is the path of the directory for thumbnails ThumbsDirName = "/.thumbs" // WebappsDirName is the path of the directory in which apps are stored WebappsDirName = "/.cozy_apps" // KonnectorsDirName is the path of the directory in which konnectors source // are stored KonnectorsDirName = "/.cozy_konnectors" // OrphansDirName is the path of the directory used to store data-files added // in the index from a filesystem-check (fsck) OrphansDirName = "/.cozy_orphans" // VersionsDirName is the path of the directory where old versions of files // are persisted. VersionsDirName = "/.cozy_versions" )
const DocTypeVersion = "1"
DocTypeVersion represents the doctype version. Each time this document structure is modified, update this value
const ForbiddenFilenameChars = "/\x00\n\r"
ForbiddenFilenameChars is the list of forbidden characters in a filename.
const MaxDepth = 512
MaxDepth is the maximum amount of recursion allowed for the recursive walk process.
const MetadataExtractorVersion = 2
MetadataExtractorVersion is the version number of the metadata extractor. It will be used later to know which files can be re-examined to get more metadata when the extractor is improved.
const ZipMime = "application/zip"
ZipMime is the content-type for zip archives
Variables ¶
var ( // ErrParentDoesNotExist is used when the parent directory does not // exist ErrParentDoesNotExist = errors.New("Parent directory with given DirID does not exist") // ErrForbiddenDocMove is used when trying to move a document in an // illicit destination ErrForbiddenDocMove = errors.New("Forbidden document move") // ErrIllegalFilename is used when the given filename is not allowed ErrIllegalFilename = errors.New("Invalid filename: empty or contains an illegal character") // ErrIllegalPath is used when the path has too many levels ErrIllegalPath = errors.New("Invalid path: too many levels") // ErrIllegalMime is used when the mime-type of a file is invalid ErrIllegalMime = errors.New("Invalid Content-Type") // ErrIllegalTime is used when a time given (creation or // modification) is not allowed ErrIllegalTime = errors.New("Invalid time given") // ErrInvalidHash is used when the given hash does not match the // calculated one ErrInvalidHash = errors.New("Invalid hash") // ErrContentLengthMismatch is used when the content-length does not // match the calculated one ErrContentLengthMismatch = errors.New("Content length does not match") // ErrConflict is used when the access to a file or directory is in // conflict with another ErrConflict = errors.New("Conflict access to same file or directory") // ErrFileInTrash is used when the file is already in the trash ErrFileInTrash = errors.New("File or directory is already in the trash") // ErrFileNotInTrash is used when the file is not in the trash ErrFileNotInTrash = errors.New("File or directory is not in the trash") // ErrParentInTrash is used when trying to upload a file to a directory // that is trashed ErrParentInTrash = errors.New("Parent directory is in the trash") // ErrNonAbsolutePath is used when the given path is not absolute // while it is required to be ErrNonAbsolutePath = errors.New("Path should be absolute") // ErrDirNotEmpty is used to inform that the directory is not // empty ErrDirNotEmpty = errors.New("Directory is not empty") // ErrWrongCouchdbState is given when couchdb gives us an unexpected value ErrWrongCouchdbState = errors.New("Wrong couchdb reduce value") // ErrFileTooBig is used when there is no more space left on the filesystem ErrFileTooBig = errors.New("The file is too big and exceeds the disk quota") // ErrMaxFileSize is used when a file is larger than the filesystem's maximum file size ErrMaxFileSize = errors.New("The file is too big and exceeds the filesystem maximum file size") // ErrFsckFailFast is used when the FSCK is stopped by the fail-fast option ErrFsckFailFast = errors.New("FSCK has been stopped on first failure") // ErrWrongToken is used when a key is not found on the store ErrWrongToken = errors.New("Wrong download token") // ErrInvalidMetadataID is used when the metadata cannot be found from a MetadatID parameter ErrInvalidMetadataID = errors.New("Invalid or expired MetadataID") )
var DeleteNote = func(db prefixer.Prefixer, noteID string) {}
var ErrIteratorDone = errors.New("No more element in the iterator")
ErrIteratorDone is returned by the Next() method of the iterator when the iterator is actually done.
var ErrSkipDir = errors.New("skip directories")
ErrSkipDir is used in WalkFn as an error to skip the current directory. It is not returned by any function of the package.
var ErrWalkOverflow = errors.New("vfs: walk overflow")
ErrWalkOverflow is used in the walk process when the maximum amount of recursivity allowed is reached when browsing the index tree.
var RevokeSharingFunc = func(db prefixer.Prefixer, sharingID string) { logger.WithNamespace("vfs").WithField("critical", "true"). Errorf("RevokeSharingFunc called without having been overridden!") }
RevokeSharingFunc does nothing. It will will be overridden from the sharing package.
var ThumbnailFormatNames = []string{
"tiny",
"small",
"medium",
"large",
}
ThumbnailFormatNames is the list of supported thumbnail formats
Functions ¶
func Allows ¶
func Allows(fs VFS, pset permission.Set, v permission.Verb, fd Fetcher) error
Allows check if a permSet allows verb on given file
func CheckAvailableDiskSpace ¶
func ConflictName ¶
ConflictName generates a new name for a file/folder in conflict with another that has the same path. A conflicted file `foo` will be renamed foo (2), then foo (3), etc.
func ContentDisposition ¶
ContentDisposition creates an HTTP header value for Content-Disposition
func DirExists ¶
DirExists returns wether or not the specified path exist in the file system and is associated with a directory.
func DiskQuotaAfterDestroy ¶
DiskQuotaAfterDestroy is a helper function that can be used after files or directories have be erased from the disk in order to register that the disk quota alert has fall behind (or not).
func ExtractMimeAndClass ¶
ExtractMimeAndClass returns a mime and class value from the specified content-type. For now it only takes the first segment of the type as the class and the whole type as mime.
func ExtractMimeAndClassFromFilename ¶
ExtractMimeAndClassFromFilename is a shortcut of ExtractMimeAndClass used to generate the mime and class from a filename.
func FilterNotSynchronizedDocs ¶
func FilterNotSynchronizedDocs(fs VFS, clientID string, changes *couchdb.ChangesResponse) error
FilterNotSynchronizedDocs filters a changes feed to replace documents in not_synchronized_on directories with deleted: true entries.
func FindVersionsToClean ¶
func FindVersionsToClean(db Prefixer, fileID string, candidate *Version) (ActionForCandidateVersion, []*Version, error)
FindVersionsToClean returns a bool to say if the candidate version must be cleaned, a list of old versions to clean, and an error. The rules to know the versions to clean or keep are: - the tagged versions are kept - two versions must not be too close in time - there is a maximal number of versions.
func MergeMetadata ¶
MergeMetadata takes a metadata map and merges it in the FileDoc
func OptionsAllowCreationInTrash ¶
func OptionsAllowCreationInTrash(opts []CreateOptions) bool
OptionsAllowCreationInTrash returns true if one of the given option says so.
func PushDiskQuotaAlert ¶
PushDiskQuotaAlert can be used to notify when the VFS reaches, or fall behind, its quota alert of 90% of its total capacity.
func RegisterDiskQuotaAlertCallback ¶
RegisterDiskQuotaAlertCallback allows to register a callback function called when the instance reaches, a fall behind, 90% of its quota capacity.
func Remove ¶
func Remove(fs VFS, name string, push func(TrashJournal) error) error
Remove removes the specified named file or directory.
func RemoveAll ¶
func RemoveAll(fs VFS, name string, push func(TrashJournal) error) error
RemoveAll removes the specified name file or directory and its content.
func SameReferences ¶
func SameReferences(a, b []couchdb.DocReference) bool
SameReferences returns true if the two sets reference the same documents.
func ServeFileContent ¶
func ServeFileContent(fs VFS, doc *FileDoc, version *Version, filename, disposition string, req *http.Request, w http.ResponseWriter) error
ServeFileContent replies to a http request using the content of a file given its FileDoc.
It uses internally http.ServeContent and benefits from it by offering support to Range, If-Modified-Since and If-None-Match requests. It uses the revision of the file as the Etag value for non-ranged requests
The content disposition is inlined.
func ServePDFIcon ¶
ServePDFIcon will send the icon image for a PDF.
func ServePDFPage ¶
func ServePDFPage(fs VFS, doc *FileDoc, disposition string, page int, req *http.Request, w http.ResponseWriter) error
ServePDFPage replies to an http request with a single page from a PDF file.
func ServePDFPreview ¶
ServePDFPreview will send the preview image for a PDF.
func SetMetaFromVersion ¶
SetMetaFromVersion takes the metadata from the version and copies them to the file document.
func WalkAlreadyLocked ¶
WalkAlreadyLocked walks the file tree rooted on the given directory. It is the responsibility of the caller to ensure the VFS is already locked (read).
Types ¶
type ActionForCandidateVersion ¶
type ActionForCandidateVersion int
const ( DoNothingForCandidateVersion ActionForCandidateVersion = iota KeepCandidateVersion CleanCandidateVersion )
type Archive ¶
type Archive struct { Name string `json:"name"` Secret string `json:"-"` IDs []string `json:"ids"` Files []string `json:"files"` Pages []Page `json:"pages"` // contains filtered or unexported fields }
Archive is the data to create a zip archive
func (*Archive) GetEntries ¶
func (a *Archive) GetEntries(fs VFS) ([]ArchiveEntry, error)
GetEntries returns all files and folders in the archive as ArchiveEntry.
type ArchiveEntry ¶
type ArchiveEntry struct { Dir *DirDoc File *FileDoc Page int // contains filtered or unexported fields }
ArchiveEntry is an utility struct to store a file or doc to be placed in the archive.
type AudioExtractor ¶
type AudioExtractor struct {
// contains filtered or unexported fields
}
AudioExtractor is used to extract album/artist/etc. from audio
func NewAudioExtractor ¶
func NewAudioExtractor() *AudioExtractor
NewAudioExtractor returns an extractor for audio
func (*AudioExtractor) Abort ¶
func (e *AudioExtractor) Abort(err error)
Abort is called when the extractor can be discarded
func (*AudioExtractor) Close ¶
func (e *AudioExtractor) Close() error
Close is called when all the bytes has been pushed, to finalize the extraction
func (*AudioExtractor) Result ¶
func (e *AudioExtractor) Result() Metadata
Result is called to get the extracted metadata
func (*AudioExtractor) Start ¶
func (e *AudioExtractor) Start()
Start is used in a goroutine to start the metadata extraction
type Avatarer ¶
type Avatarer interface { CreateAvatar(contentType string) (io.WriteCloser, error) ServeAvatarContent(w http.ResponseWriter, req *http.Request) error }
Avatarer defines an interface to define an avatar filesystem.
type CozyMetadataPatch ¶
type CozyMetadataPatch struct {
Favorite *bool `json:"favorite,omitempty"`
}
CozyMetadataPatch is a struct containing the modifiable fields for a CozyMetadata.
type CreateOptions ¶
type CreateOptions int
CreateOptions is used for options on the create file operation
const ( // AllowCreationInTrash is an option to allow bypassing the rule that // forbids the creation of file in the trash. AllowCreationInTrash CreateOptions = 1 + iota )
type DirDoc ¶
type DirDoc struct { // Type of document. Useful to (de)serialize and filter the data // from couch. Type string `json:"type"` // Qualified file identifier DocID string `json:"_id,omitempty"` // Directory revision DocRev string `json:"_rev,omitempty"` // Directory name DocName string `json:"name,omitempty"` // Parent directory identifier DirID string `json:"dir_id,omitempty"` RestorePath string `json:"restore_path,omitempty"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` Tags []string `json:"tags,omitempty"` // Directory path on VFS. // Fullpath should always be present. It is marked "omitempty" because // DirDoc is the base of the DirOrFile struct. Fullpath string `json:"path,omitempty"` ReferencedBy []couchdb.DocReference `json:"referenced_by,omitempty"` NotSynchronizedOn []couchdb.DocReference `json:"not_synchronized_on,omitempty"` Metadata Metadata `json:"metadata,omitempty"` CozyMetadata *FilesCozyMetadata `json:"cozyMetadata,omitempty"` }
DirDoc is a struct containing all the informations about a directory. It implements the couchdb.Doc and jsonapi.Object interfaces.
func MkdirAll ¶
MkdirAll creates a directory named path, along with any necessary parents, and returns nil, or else returns an error.
func ModifyDirMetadata ¶
ModifyDirMetadata modify the metadata associated to a directory. It can be used to rename or move the directory in the VFS.
func NewDirDocWithParent ¶
NewDirDocWithParent returns an instance of DirDoc from a parent document. The given name is validated.
func NewDirDocWithPath ¶
NewDirDocWithPath returns an instance of DirDoc its directory ID and path. The given name is validated.
func RestoreDir ¶
RestoreDir is used to restore a trashed directory given its document
func (*DirDoc) AddNotSynchronizedOn ¶
func (d *DirDoc) AddNotSynchronizedOn(refs ...couchdb.DocReference)
AddNotSynchronizedOn adds not_synchronized_on to the directory
func (*DirDoc) AddReferencedBy ¶
func (d *DirDoc) AddReferencedBy(ri ...couchdb.DocReference)
AddReferencedBy adds referenced_by to the directory
func (*DirDoc) Path ¶
func (d *DirDoc) Path(fs FilePather) (string, error)
Path is used to generate the file path
func (*DirDoc) RemoveNotSynchronizedOn ¶
func (d *DirDoc) RemoveNotSynchronizedOn(refs ...couchdb.DocReference)
RemoveNotSynchronizedOn removes one or several not_synchronized_on to the directory
func (*DirDoc) RemoveReferencedBy ¶
func (d *DirDoc) RemoveReferencedBy(ri ...couchdb.DocReference)
RemoveReferencedBy removes one or several referenced_by to the directory
type DirIterator ¶
DirIterator is the interface that an iterator over a specific directory should implement. The Next method will return a ErrIteratorDone when the iterator is over and does not have element anymore.
func NewIterator ¶
func NewIterator(db prefixer.Prefixer, dir *DirDoc, opt *IteratorOptions) DirIterator
NewIterator return a new iterator.
type DirOrFileDoc ¶
type DirOrFileDoc struct { *DirDoc // fields from FileDoc not contained in DirDoc ByteSize int64 `json:"size,string"` MD5Sum []byte `json:"md5sum,omitempty"` Mime string `json:"mime,omitempty"` Class string `json:"class,omitempty"` Executable bool `json:"executable,omitempty"` Trashed bool `json:"trashed,omitempty"` Encrypted bool `json:"encrypted,omitempty"` InternalID string `json:"internal_vfs_id,omitempty"` }
DirOrFileDoc is a union struct of FileDoc and DirDoc. It is useful to unmarshal documents from couch.
func (*DirOrFileDoc) Clone ¶
func (fd *DirOrFileDoc) Clone() couchdb.Doc
Clone is part of the couchdb.Doc interface
func (*DirOrFileDoc) Refine ¶
func (fd *DirOrFileDoc) Refine() (*DirDoc, *FileDoc)
Refine returns either a DirDoc or FileDoc pointer depending on the type of the DirOrFileDoc
type DiskThresholder ¶
type DiskThresholder interface { // DiskQuota returns the total number of bytes allowed to be stored in the // VFS. If minus or equal to zero, it is considered without limit. DiskQuota() int64 }
DiskThresholder it an interface that can be implemeted to known how many space is available on the disk.
type DocPatch ¶
type DocPatch struct { Name *string `json:"name,omitempty"` DirID *string `json:"dir_id,omitempty"` RestorePath *string `json:"restore_path,omitempty"` Tags *[]string `json:"tags,omitempty"` UpdatedAt *time.Time `json:"updated_at,omitempty"` Executable *bool `json:"executable,omitempty"` Encrypted *bool `json:"encrypted,omitempty"` Class *string `json:"class,omitempty"` CozyMetadata CozyMetadataPatch `json:"cozyMetadata"` }
DocPatch is a struct containing modifiable fields from file and directory documents.
type ExifExtractor ¶
type ExifExtractor struct {
// contains filtered or unexported fields
}
ExifExtractor is used to extract EXIF metadata from jpegs
func NewExifExtractor ¶
func NewExifExtractor(createdAt time.Time, withImageExtractor bool) *ExifExtractor
NewExifExtractor returns an extractor for EXIF metadata
func (*ExifExtractor) Abort ¶
func (e *ExifExtractor) Abort(err error)
Abort is called when the extractor can be discarded
func (*ExifExtractor) Close ¶
func (e *ExifExtractor) Close() error
Close is called when all the bytes has been pushed, to finalize the extraction
func (*ExifExtractor) Result ¶
func (e *ExifExtractor) Result() Metadata
Result is called to get the extracted metadata
func (*ExifExtractor) Start ¶
func (e *ExifExtractor) Start()
Start is used in a goroutine to start the metadata extraction
type Fetcher ¶
type Fetcher interface { permission.Fetcher Path(fs FilePather) (string, error) Parent(fs VFS) (*DirDoc, error) // contains filtered or unexported methods }
Fetcher extends on permission.Fetcher with hierarchy functions
type File ¶
File is a reader, writer, seeker, closer iterface representing an opened file for reading or writing.
type FileDoc ¶
type FileDoc struct { // Type of document. Useful to (de)serialize and filter the data // from couch. Type string `json:"type"` // Qualified file identifier DocID string `json:"_id,omitempty"` // File revision DocRev string `json:"_rev,omitempty"` // File name DocName string `json:"name,omitempty"` // Parent directory identifier DirID string `json:"dir_id,omitempty"` RestorePath string `json:"restore_path,omitempty"` CreatedAt time.Time `json:"created_at"` UpdatedAt time.Time `json:"updated_at"` ByteSize int64 `json:"size,string"` // Serialized in JSON as a string, because JS has some issues with big numbers MD5Sum []byte `json:"md5sum,omitempty"` Mime string `json:"mime,omitempty"` Class string `json:"class,omitempty"` Executable bool `json:"executable"` Trashed bool `json:"trashed"` Encrypted bool `json:"encrypted"` Tags []string `json:"tags,omitempty"` Metadata Metadata `json:"metadata,omitempty"` ReferencedBy []couchdb.DocReference `json:"referenced_by,omitempty"` CozyMetadata *FilesCozyMetadata `json:"cozyMetadata,omitempty"` // InternalID is an identifier that can be used by the VFS, but must no be // used by clients. For example, it can be used to know the location in // Swift of a file. InternalID string `json:"internal_vfs_id,omitempty"` // contains filtered or unexported fields }
FileDoc is a struct containing all the informations about a file. It implements the couchdb.Doc and jsonapi.Object interfaces.
func CreateFileDocCopy ¶
func ModifyFileMetadata ¶
ModifyFileMetadata modify the metadata associated to a file. It can be used to rename or move the file in the VFS.
func NewFileDoc ¶
func NewFileDoc(name, dirID string, size int64, md5Sum []byte, mimeType, class string, cdate time.Time, executable, trashed, encrypted bool, tags []string) (*FileDoc, error)
NewFileDoc is the FileDoc constructor. The given name is validated.
func RestoreFile ¶
RestoreFile is used to restore a trashed file given its document
func (*FileDoc) AddReferencedBy ¶
func (f *FileDoc) AddReferencedBy(ri ...couchdb.DocReference)
AddReferencedBy adds referenced_by to the file
func (*FileDoc) Path ¶
func (f *FileDoc) Path(fp FilePather) (string, error)
Path is used to generate the file path
func (*FileDoc) RemoveReferencedBy ¶
func (f *FileDoc) RemoveReferencedBy(ri ...couchdb.DocReference)
RemoveReferencedBy removes one or several referenced_by to the file
func (*FileDoc) ResetFullpath ¶
func (f *FileDoc) ResetFullpath()
ResetFullpath clears the fullpath, so it can be recomputed with Path()
type FilePather ¶
FilePather is an interface for computing the fullpath of a filedoc
func NewFilePatherWithCache ¶
func NewFilePatherWithCache(fs Indexer) FilePather
NewFilePatherWithCache creates a FilePather that will cache the calls to CouchDB for finding the parent directories.
type FilesCozyMetadata ¶
type FilesCozyMetadata struct { metadata.CozyMetadata // Instance URL where the file has been created CreatedOn string `json:"createdOn,omitempty"` // Date of the last upload of a new content UploadedAt *time.Time `json:"uploadedAt,omitempty"` // Information about the last time the content was uploaded UploadedBy *UploadedByEntry `json:"uploadedBy,omitempty"` // Instance URL where the content has been changed the last time UploadedOn string `json:"uploadedOn,omitempty"` }
FilesCozyMetadata is an extended version of cozyMetadata with some specific fields.
func NewCozyMetadata ¶
func NewCozyMetadata(instance string) *FilesCozyMetadata
NewCozyMetadata initializes a new FilesCozyMetadata struct
func (*FilesCozyMetadata) Clone ¶
func (fcm *FilesCozyMetadata) Clone() *FilesCozyMetadata
Clone clones a FileCozyMetadata struct
func (*FilesCozyMetadata) ToJSONDoc ¶
func (fcm *FilesCozyMetadata) ToJSONDoc() map[string]interface{}
ToJSONDoc returns a map with the cozyMetadata to be used inside a JSONDoc (typically for sendings sharing updates).
func (*FilesCozyMetadata) UpdatedByApp ¶
func (fcm *FilesCozyMetadata) UpdatedByApp(entry *metadata.UpdatedByAppEntry)
UpdatedByApp updates the list of UpdatedByApps entries with the new entry. It ensures that each entry has a unique slug+instance, and the new entry will be in the last position.
type Fs ¶
type Fs interface { prefixer.Prefixer InitFs() error Delete() error // Maximum file size MaxFileSize() int64 // OpenFile return a file handler for reading associated with the given file // document. The file handler implements io.ReadCloser and io.Seeker. OpenFile(doc *FileDoc) (File, error) // OpenFileVersion returns a file handler for reading the content of an old // version of the given file. OpenFileVersion(doc *FileDoc, version *Version) (File, error) // CreateDir is used to create a new directory from its document. CreateDir(doc *DirDoc) error // CreateFile creates a new file or update the content of an existing file. // The first argument contains the document of the new or update version of // the file. The second argument is the optional old document of the old // version of the file. // // Warning: you MUST call the Close() method and check for its error. CreateFile(newdoc, olddoc *FileDoc, opts ...CreateOptions) (File, error) // CopyFile creates a fresh copy of the source file with the given newdoc // attributes (e.g. a new name) CopyFile(olddoc, newdoc *FileDoc) error // DissociateFile creates a copy of the source file with the name and // directory of the destination file doc, and then remove the source file // with all of its version. It is used by the sharings to change the ID // of the document to avoid later conflicts. DissociateFile(src, dst *FileDoc) error // DissociateDir is like DissociateFile but for directories. DissociateDir(src, dst *DirDoc) error // DestroyDirContent destroys all directories and files contained in a // directory. DestroyDirContent(doc *DirDoc, push func(TrashJournal) error) error // DestroyDirAndContent destroys all directories and files contained in a // directory and the directory itself. DestroyDirAndContent(doc *DirDoc, push func(TrashJournal) error) error // DestroyFile destroys a file from the trash. DestroyFile(doc *FileDoc) error // EnsureErased remove the files in Swift if they still exist. EnsureErased(journal TrashJournal) error // RevertFileVersion restores the content of a file from an old version. // The current version of the content is not lost, but saved as another // version. RevertFileVersion(doc *FileDoc, version *Version) error // CleanOldVersion deletes an old version of a file. CleanOldVersion(fileID string, version *Version) error // ClearOldVersions deletes all the old versions of all files ClearOldVersions() error // ImportFileVersion returns a file handler that can be used to write a // version. ImportFileVersion(version *Version, content io.ReadCloser) error // CopyFileFromOtherFS creates or updates a file by copying the content of // a file in another Cozy. It is used for sharings, to optimize I/O when // two instances are on the same stack. CopyFileFromOtherFS(olddoc, newdoc *FileDoc, srcFS Fs, srcDoc *FileDoc) error // Fsck return the list of inconsistencies in the VFS Fsck(func(log *FsckLog), bool) (err error) CheckFilesConsistency(func(*FsckLog), bool) error }
Fs is an interface providing a set of high-level methods to interact with the file-system binaries and metadata.
type FsckContentMismatch ¶
type FsckContentMismatch struct { SizeIndex int64 `json:"size_index"` SizeFile int64 `json:"size_file"` MD5SumIndex []byte `json:"md5sum_index"` MD5SumFile []byte `json:"md5sum_file"` }
FsckContentMismatch is a struct used by the FSCK where CouchDB and Swift haven't the same information about a file content (md5sum and size).
type FsckLog ¶
type FsckLog struct { Type FsckLogType `json:"type"` FileDoc *TreeFile `json:"file_doc,omitempty"` DirDoc *TreeFile `json:"dir_doc,omitempty"` VersionDoc *Version `json:"version_doc,omitempty"` IsFile bool `json:"is_file"` IsVersion bool `json:"is_version"` ContentMismatch *FsckContentMismatch `json:"content_mismatch,omitempty"` ExpectedFullpath string `json:"expected_fullpath,omitempty"` }
FsckLog is a struct for an inconsistency in the VFS
type ImageExtractor ¶
type ImageExtractor struct {
// contains filtered or unexported fields
}
ImageExtractor is used to extract width/height from images
func NewImageExtractor ¶
func NewImageExtractor(createdAt time.Time) *ImageExtractor
NewImageExtractor returns an extractor for images
func (*ImageExtractor) Abort ¶
func (e *ImageExtractor) Abort(err error)
Abort is called when the extractor can be discarded
func (*ImageExtractor) Close ¶
func (e *ImageExtractor) Close() error
Close is called when all the bytes has been pushed, to finalize the extraction
func (*ImageExtractor) Result ¶
func (e *ImageExtractor) Result() Metadata
Result is called to get the extracted metadata
func (*ImageExtractor) Start ¶
func (e *ImageExtractor) Start()
Start is used in a goroutine to start the metadata extraction
type Indexer ¶
type Indexer interface { InitIndex() error FilePather // DiskUsage computes the total size of the files contained in the VFS, // including versions. DiskUsage() (int64, error) // FilesUsage computes the total size of the files contained in the VFS, // excluding versions. FilesUsage() (int64, error) // VersionsUsage computes the total size of the old file versions contained // in the VFS, not including latest version. VersionsUsage() (int64, error) // TrashUsage computes the total size of the files contained in the trash. TrashUsage() (int64, error) // DirSize returns the size of a directory, including files in // subdirectories. DirSize(doc *DirDoc) (int64, error) // CreateFileDoc creates and add in the index a new file document. CreateFileDoc(doc *FileDoc) error // CreateNamedFileDoc creates and add in the index a new file document with // its id already set. CreateNamedFileDoc(doc *FileDoc) error // UpdateFileDoc is used to update the document of a file. It takes the // new file document that you want to create and the old document, // representing the current revision of the file. UpdateFileDoc(olddoc, newdoc *FileDoc) error // DeleteFileDoc removes from the index the specified file document. DeleteFileDoc(doc *FileDoc) error // CreateDirDoc creates and add in the index a new directory document. CreateDirDoc(doc *DirDoc) error // CreateNamedDirDoc creates and add in the index a new directory document // with its id already set. CreateNamedDirDoc(doc *DirDoc) error // UpdateDirDoc is used to update the document of a directory. It takes the // new directory document that you want to create and the old document, // representing the current revision of the directory. UpdateDirDoc(olddoc, newdoc *DirDoc) error // DeleteDirDoc removes from the index the specified directory document. DeleteDirDoc(doc *DirDoc) error // DeleteDirDocAndContent removes from the index the specified directory as // well all its children. It returns the list of the children files that // were removed. DeleteDirDocAndContent(doc *DirDoc, onlyContent bool) ([]*FileDoc, int64, error) // MoveDir is an internal call to update the fullpath of the subdirectories // of a renamed/moved directory. It is exported to allow the sharing // indexer to call this method on the couchdb indexer of the VFS. MoveDir(oldpath, newpath string) error // DirByID returns the directory document information associated with the // specified identifier. DirByID(fileID string) (*DirDoc, error) // DirByPath returns the directory document information associated with the // specified path. DirByPath(name string) (*DirDoc, error) // FileByID returns the file document information associated with the // specified identifier. FileByID(fileID string) (*FileDoc, error) // FileByPath returns the file document information associated with the // specified path. FileByPath(name string) (*FileDoc, error) // DirOrFileByID returns the document from its identifier without knowing in // advance its type. One of the returned argument is not nil. DirOrFileByID(fileID string) (*DirDoc, *FileDoc, error) // DirOrFileByPath returns the document from its path without knowing in // advance its type. One of the returned argument is not nil. DirOrFileByPath(name string) (*DirDoc, *FileDoc, error) // DirIterator returns an iterator over the children of the specified // directory. DirIterator(doc *DirDoc, opts *IteratorOptions) DirIterator // DirBatch returns a batch of documents DirBatch(*DirDoc, couchdb.Cursor) ([]DirOrFileDoc, error) DirLength(*DirDoc) (int, error) DirChildExists(dirID, filename string) (bool, error) BatchDelete([]couchdb.Doc) error // CreateVersion adds a version to the CouchDB index. CreateVersion(*Version) error // DeleteVersion removes a version from the CouchDB index. DeleteVersion(*Version) error AllVersions() ([]*Version, error) BatchDeleteVersions([]*Version) error ListNotSynchronizedOn(clientID string) ([]DirDoc, error) CheckIndexIntegrity(func(*FsckLog), bool) error CheckTreeIntegrity(*Tree, func(*FsckLog), bool) error BuildTree(each ...func(*TreeFile)) (tree *Tree, err error) }
Indexer is an interface providing a common set of method for indexing layer of our VFS.
An indexer is typically responsible for storing and indexing the files and directories metadata, as well as caching them if necessary.
func NewCouchdbIndexer ¶
NewCouchdbIndexer creates an Indexer instance based on couchdb to store files and directories metadata and index them.
type IteratorOptions ¶
IteratorOptions contains the options of the iterator.
type MetaExtractor ¶
type MetaExtractor interface { io.WriteCloser Abort(error) Result() Metadata }
MetaExtractor is an interface for extracting metadata from a file
func NewMetaExtractor ¶
func NewMetaExtractor(doc *FileDoc) *MetaExtractor
NewMetaExtractor returns an extractor for metadata if the mime type has one, or null else
type Metadata ¶
type Metadata map[string]interface{}
Metadata is a list of metadata specific to each mimetype: id3 for music, exif for jpegs, etc.
func NewMetadata ¶
func NewMetadata() Metadata
NewMetadata returns a new metadata object, with the version field set
func (Metadata) RemoveCertifiedMetadata ¶
RemoveCertifiedMetadata returns a metadata map where the keys that are certified have been removed. It can be useful for sharing, as certified metadata are only valid localy.
func (Metadata) RemoveFavoriteMetadata ¶
RemoveFavoriteMetadata returns a metadata map where the favorite key has been removed. It can be useful for sharing, as favorite metadata are only valid localy.
type Prefixer ¶
Prefixer interface describes a prefixer that can also give the context for the targeted instance.
type ShortcutExtractor ¶
type ShortcutExtractor struct {
// contains filtered or unexported fields
}
ShortcutExtractor is used to extract information from .url files
func NewShortcutExtractor ¶
func NewShortcutExtractor(instance string, target map[string]interface{}) *ShortcutExtractor
NewShortcutExtractor returns an extractor for .url files
func (*ShortcutExtractor) Abort ¶
func (e *ShortcutExtractor) Abort(err error)
Abort is called when the extractor can be discarded
func (*ShortcutExtractor) Close ¶
func (e *ShortcutExtractor) Close() error
Close is called when all the bytes has been pushed, to finalize the extraction
func (*ShortcutExtractor) Result ¶
func (e *ShortcutExtractor) Result() Metadata
Result is called to get the extracted metadata
func (*ShortcutExtractor) Start ¶
func (e *ShortcutExtractor) Start()
Start is used in a goroutine to start the metadata extraction
type Store ¶
type Store interface { AddFile(db prefixer.Prefixer, filePath string) (string, error) AddThumb(db prefixer.Prefixer, fileID string) (string, error) AddThumbs(db prefixer.Prefixer, fileIDs []string) (map[string]string, error) AddVersion(db prefixer.Prefixer, versionID string) (string, error) AddArchive(db prefixer.Prefixer, archive *Archive) (string, error) AddMetadata(db prefixer.Prefixer, metadata *Metadata) (string, error) GetFile(db prefixer.Prefixer, key string) (string, error) GetThumb(db prefixer.Prefixer, key string) (string, error) GetVersion(db prefixer.Prefixer, key string) (string, error) GetArchive(db prefixer.Prefixer, key string) (*Archive, error) GetMetadata(db prefixer.Prefixer, key string) (*Metadata, error) }
Store is essentially a place to store transient objects between two HTTP requests: it can be a secret for downloading a file, a list of files in an archive, or a metadata object for an upcoming upload.
type ThumbFiler ¶
ThumbFiler defines a interface to handle the creation of thumbnails. It is an io.Writer that can be aborted in case of error, or committed in case of success.
type Thumbser ¶
type Thumbser interface { ThumbExists(img *FileDoc, format string) (ok bool, err error) CreateThumb(img *FileDoc, format string) (ThumbFiler, error) RemoveThumbs(img *FileDoc, formats []string) error ServeThumbContent(w http.ResponseWriter, req *http.Request, img *FileDoc, format string) error CreateNoteThumb(id, mime, format string) (ThumbFiler, error) OpenNoteThumb(id, format string) (io.ReadCloser, error) RemoveNoteThumb(id string, formats []string) error ServeNoteThumbContent(w http.ResponseWriter, req *http.Request, id string) error }
Thumbser defines an interface to define a thumbnail filesystem.
type TrashJournal ¶
TrashJournal is a list of files that have been deleted of CouchDB when the trash was cleared, but removing them from Swift is slow and should be done later via the trash-files worker.
type Tree ¶
type Tree struct { Root *TreeFile DirsMap map[string]*TreeFile Orphans map[string][]*TreeFile Files map[string]struct{} }
Tree is returned by the BuildTree method on the indexes. It contains a pointer to the root element of the tree, a map of directories indexed by their ID, and a map of a potential list of orphan file or directories indexed by their DirID.
type TreeFile ¶
type TreeFile struct { DirOrFileDoc FilesChildren []*TreeFile `json:"children,omitempty"` FilesChildrenSize int64 `json:"children_size,omitempty"` DirsChildren []*TreeFile `json:"directories,omitempty"` IsDir bool `json:"is_dir"` IsOrphan bool `json:"is_orphan"` HasCycle bool `json:"has_cycle"` HasPath bool `json:"-"` }
TreeFile represent a subset of a file/directory structure that can be used in a tree-like representation of the index.
type UploadedByEntry ¶
type UploadedByEntry struct { Slug string `json:"slug,omitempty"` Version string `json:"version,omitempty"` Client map[string]string `json:"oauthClient,omitempty"` }
UploadedByEntry is a struct with information on the app that has done the last upload of a file.
type VFS ¶
type VFS interface { Indexer DiskThresholder Fs // UseSharingIndexer returns a new Fs with an overloaded indexer that can // be used for the special purpose of the sharing. UseSharingIndexer(Indexer) VFS // GetIndexer returns the indexer without the overloaded operations from // VFSAfero / VFSSwift. Its result can be used for FilePatherWithCache with // a VFS that is already locked. GetIndexer() Indexer }
VFS is composed of the Indexer and Fs interface. It is the common interface used throughout the stack to access the VFS.
type Version ¶
type Version struct { DocID string `json:"_id,omitempty"` DocRev string `json:"_rev,omitempty"` UpdatedAt time.Time `json:"updated_at"` ByteSize int64 `json:"size,string"` MD5Sum []byte `json:"md5sum"` Tags []string `json:"tags"` Metadata Metadata `json:"metadata,omitempty"` CozyMetadata FilesCozyMetadata `json:"cozyMetadata,omitempty"` Rels struct { File struct { Data struct { ID string `json:"_id"` Type string `json:"_type"` } `json:"data"` } `json:"file"` } `json:"relationships"` }
Version is used for storing the metadata about previous versions of file contents. The content its-self is stored on the local file system or in Swift.
func FindVersion ¶
FindVersion returns the version for the given id
func NewVersion ¶
NewVersion returns a version from a given FileDoc. It is often used just before modifying the content of this file. Note that the _id is precomputed as it can be useful to use it for a storage location before the version is saved in CouchDB.
func VersionsFor ¶
VersionsFor returns the list of the versions for a given file identifier.
func (*Version) Relationships ¶
func (v *Version) Relationships() jsonapi.RelationshipMap
Relationships is part of jsonapi.Object interface
Source Files
¶
Directories
¶
Path | Synopsis |
---|---|
Package vfsafero is the implementation of the Virtual File System by using afero.
|
Package vfsafero is the implementation of the Virtual File System by using afero. |
Package vfsswift is the implementation of the Virtual File System by using Swift from the OpenStack project.
|
Package vfsswift is the implementation of the Virtual File System by using Swift from the OpenStack project. |