vfs

package
v1.50.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 10, 2020 License: MIT Imports: 29 Imported by: 0

Documentation

Overview

Package vfs provides a virtual filing system layer over rclone's native objects.

It attempts to behave in a similar way to Go's filing system manipulation code in the os package. The same named function should behave in an identical fashion. The objects also obey Go's standard interfaces.

Note that paths don't start or end with /, so the root directory may be referred to as "". However Stat strips slashes so you can use paths with slashes in.

It also includes directory caching

The vfs package returns Error values to signal precisely which error conditions have ocurred. It may also return general errors it receives. It tries to use os Error values (eg os.ErrExist) where possible.

Index

Constants

This section is empty.

Variables

View Source 
var (
	ENOENT  = os.ErrNotExist
	EEXIST  = os.ErrExist
	EPERM   = os.ErrPermission
	EINVAL  = os.ErrInvalid
	ECLOSED = os.ErrClosed
)

Errors which have exact counterparts in os

View Source 
var DefaultOpt = Options{
	NoModTime:         false,
	NoChecksum:        false,
	NoSeek:            false,
	DirCacheTime:      5 * 60 * time.Second,
	PollInterval:      time.Minute,
	ReadOnly:          false,
	Umask:             0,
	UID:               ^uint32(0),
	GID:               ^uint32(0),
	DirPerms:          os.FileMode(0777),
	FilePerms:         os.FileMode(0666),
	CacheMode:         CacheModeOff,
	CacheMaxAge:       3600 * time.Second,
	CachePollInterval: 60 * time.Second,
	ChunkSize:         128 * fs.MebiByte,
	ChunkSizeLimit:    -1,
	CacheMaxSize:      -1,
	CaseInsensitive:   runtime.GOOS == "windows" || runtime.GOOS == "darwin",
	WriteWait:         1000 * time.Millisecond,
	ReadWait:          5 * time.Millisecond,
}

DefaultOpt is the default values uses for Opt

View Source 
var Help = `
### Directory Cache

Using the ` + "`--dir-cache-time`" + ` flag, you can set how long a
directory should be considered up to date and not refreshed from the
backend. Changes made locally in the mount may appear immediately or
invalidate the cache. However, changes done on the remote will only
be picked up once the cache expires if the backend configured does not
support polling for changes. If the backend supports polling, changes
will be picked up on within the polling interval.

Alternatively, you can send a ` + "`SIGHUP`" + ` signal to rclone for
it to flush all directory caches, regardless of how old they are.
Assuming only one rclone instance is running, you can reset the cache
like this:

    kill -SIGHUP $(pidof rclone)

If you configure rclone with a [remote control](/rc) then you can use
rclone rc to flush the whole directory cache:

    rclone rc vfs/forget

Or individual files or directories:

    rclone rc vfs/forget file=path/to/file dir=path/to/dir

### File Buffering

The ` + "`--buffer-size`" + ` flag determines the amount of memory,
that will be used to buffer data in advance.

Each open file descriptor will try to keep the specified amount of
data in memory at all times. The buffered data is bound to one file
descriptor and won't be shared between multiple open file descriptors
of the same file.

This flag is a upper limit for the used memory per file descriptor.
The buffer will only use memory for data that is downloaded but not
not yet read. If the buffer is empty, only a small amount of memory
will be used.
The maximum memory used by rclone for buffering can be up to
` + "`--buffer-size * open files`" + `.

### File Caching

These flags control the VFS file caching options.  The VFS layer is
used by rclone mount to make a cloud storage system work more like a
normal file system.

You'll need to enable VFS caching if you want, for example, to read
and write simultaneously to a file.  See below for more details.

Note that the VFS cache works in addition to the cache backend and you
may find that you need one or the other or both.

    --cache-dir string                   Directory rclone will use for caching.
    --vfs-cache-max-age duration         Max age of objects in the cache. (default 1h0m0s)
    --vfs-cache-mode string              Cache mode off|minimal|writes|full (default "off")
    --vfs-cache-poll-interval duration   Interval to poll the cache for stale objects. (default 1m0s)
    --vfs-cache-max-size int             Max total size of objects in the cache. (default off)

If run with ` + "`-vv`" + ` rclone will print the location of the file cache.  The
files are stored in the user cache file area which is OS dependent but
can be controlled with ` + "`--cache-dir`" + ` or setting the appropriate
environment variable.

The cache has 4 different modes selected by ` + "`--vfs-cache-mode`" + `.
The higher the cache mode the more compatible rclone becomes at the
cost of using disk space.

Note that files are written back to the remote only when they are
closed so if rclone is quit or dies with open files then these won't
get written back to the remote.  However they will still be in the on
disk cache.

If using --vfs-cache-max-size note that the cache may exceed this size
for two reasons.  Firstly because it is only checked every
--vfs-cache-poll-interval.  Secondly because open files cannot be
evicted from the cache.

#### --vfs-cache-mode off

In this mode the cache will read directly from the remote and write
directly to the remote without caching anything on disk.

This will mean some operations are not possible

  * Files can't be opened for both read AND write
  * Files opened for write can't be seeked
  * Existing files opened for write must have O_TRUNC set
  * Files open for read with O_TRUNC will be opened write only
  * Files open for write only will behave as if O_TRUNC was supplied
  * Open modes O_APPEND, O_TRUNC are ignored
  * If an upload fails it can't be retried

#### --vfs-cache-mode minimal

This is very similar to "off" except that files opened for read AND
write will be buffered to disks.  This means that files opened for
write will be a lot more compatible, but uses the minimal disk space.

These operations are not possible

  * Files opened for write only can't be seeked
  * Existing files opened for write must have O_TRUNC set
  * Files opened for write only will ignore O_APPEND, O_TRUNC
  * If an upload fails it can't be retried

#### --vfs-cache-mode writes

In this mode files opened for read only are still read directly from
the remote, write only and read/write files are buffered to disk
first.

This mode should support all normal file system operations.

If an upload fails it will be retried up to --low-level-retries times.

#### --vfs-cache-mode full

In this mode all reads and writes are buffered to and from disk.  When
a file is opened for read it will be downloaded in its entirety first.

This may be appropriate for your needs, or you may prefer to look at
the cache backend which does a much more sophisticated job of caching,
including caching directory hierarchies and chunks of files.

In this mode, unlike the others, when a file is written to the disk,
it will be kept on the disk after it is written to the remote.  It
will be purged on a schedule according to ` + "`--vfs-cache-max-age`" + `.

This mode should support all normal file system operations.

If an upload or download fails it will be retried up to
--low-level-retries times.

### Case Sensitivity

Linux file systems are case-sensitive: two files can differ only
by case, and the exact case must be used when opening a file.

Windows is not like most other operating systems supported by rclone.
File systems in modern Windows are case-insensitive but case-preserving:
although existing files can be opened using any case, the exact case used
to create the file is preserved and available for programs to query.
It is not allowed for two files in the same directory to differ only by case.

Usually file systems on MacOS are case-insensitive. It is possible to make MacOS
file systems case-sensitive but that is not the default

The "--vfs-case-insensitive" mount flag controls how rclone handles these
two cases. If its value is "false", rclone passes file names to the mounted
file system as is. If the flag is "true" (or appears without a value on
command line), rclone may perform a "fixup" as explained below.

The user may specify a file name to open/delete/rename/etc with a case
different than what is stored on mounted file system. If an argument refers
to an existing file with exactly the same name, then the case of the existing
file on the disk will be used. However, if a file name with exactly the same
name is not found but a name differing only by case exists, rclone will
transparently fixup the name. This fixup happens only when an existing file
is requested. Case sensitivity of file names created anew by rclone is
controlled by an underlying mounted file system.

Note that case sensitivity of the operating system running rclone (the target)
may differ from case sensitivity of a file system mounted by rclone (the source).
The flag controls whether "fixup" is performed to satisfy the target.

If the flag is not provided on command line, then its default value depends
on the operating system where rclone runs: "true" on Windows and MacOS, "false"
otherwise. If the flag is provided without a value, then it is "true".
`

Help contains text describing file and directory caching to add to the command help.

Functions

This section is empty.

Types

type CacheMode 

type CacheMode byte

CacheMode controls the functionality of the cache 

const (
	CacheModeOff     CacheMode = iota // cache nothing - return errors for writes which can't be satisfied
	CacheModeMinimal                  // cache only the minimum, eg read/write opens
	CacheModeWrites                   // cache all files opened with write intent
	CacheModeFull                     // cache all files opened in any mode
)

CacheMode options

func (*CacheMode) Set 

func (l *CacheMode) Set(s string) error

Set a CacheMode

func (CacheMode) String 

func (l CacheMode) String() string

String turns a CacheMode into a string

func (*CacheMode) Type 

func (l *CacheMode) Type() string

Type of the value

type Dir 

type Dir struct {
	// contains filtered or unexported fields
}

Dir represents a directory entry

func (*Dir) Create 

func (d *Dir) Create(name string, flags int) (*File, error)

Create makes a new file node

func (*Dir) DirEntry 

func (d *Dir) DirEntry() (entry fs.DirEntry)

DirEntry returns the underlying fs.DirEntry

func (*Dir) ForgetAll 

func (d *Dir) ForgetAll()

ForgetAll ensures the directory and all its children are purged from the cache.

It does not invalidate or clear the cache of the parent directory.

func (*Dir) ForgetPath 

func (d *Dir) ForgetPath(relativePath string, entryType fs.EntryType)

ForgetPath clears the cache for itself and all subdirectories if they match the given path. The path is specified relative from the directory it is called from. The cache of the parent directory is marked as stale, but not cleared otherwise. It is not possible to traverse the directory tree upwards, i.e. you cannot clear the cache for the Dir's ancestors or siblings.

func (*Dir) Inode 

func (d *Dir) Inode() uint64

Inode returns the inode number - satisfies Node interface

func (*Dir) IsDir 

func (d *Dir) IsDir() bool

IsDir returns true for Dir - satisfies Node interface

func (*Dir) IsFile 

func (d *Dir) IsFile() bool

IsFile returns false for Dir - satisfies Node interface

func (*Dir) Mkdir 

func (d *Dir) Mkdir(name string) (*Dir, error)

Mkdir creates a new directory

func (*Dir) ModTime 

func (d *Dir) ModTime() time.Time

ModTime returns the modification time of the directory

func (*Dir) Mode 

func (d *Dir) Mode() (mode os.FileMode)

Mode bits of the directory - satisfies Node interface

func (*Dir) Name 

func (d *Dir) Name() (name string)

Name (base) of the directory - satisfies Node interface

func (*Dir) Node 

func (d *Dir) Node() Node

Node returns the Node assocuated with this - satisfies Noder interface

func (*Dir) Open 

func (d *Dir) Open(flags int) (fd Handle, err error)

Open the directory according to the flags provided

func (*Dir) Path 

func (d *Dir) Path() (name string)

Path of the directory - satisfies Node interface

func (*Dir) ReadDirAll 

func (d *Dir) ReadDirAll() (items Nodes, err error)

ReadDirAll reads the contents of the directory sorted

func (*Dir) Remove 

func (d *Dir) Remove() error

Remove the directory

func (*Dir) RemoveAll 

func (d *Dir) RemoveAll() error

RemoveAll removes the directory and any contents recursively

func (*Dir) RemoveName 

func (d *Dir) RemoveName(name string) error

RemoveName removes the entry with the given name from the receiver, which must be a directory. The entry to be removed may correspond to a file (unlink) or to a directory (rmdir).

func (*Dir) Rename 

func (d *Dir) Rename(oldName, newName string, destDir *Dir) error

Rename the file

func (*Dir) SetModTime 

func (d *Dir) SetModTime(modTime time.Time) error

SetModTime sets the modTime for this dir

func (*Dir) Size 

func (d *Dir) Size() int64

Size of the directory

func (*Dir) Stat 

func (d *Dir) Stat(name string) (node Node, err error)

Stat looks up a specific entry in the receiver.

Stat should return a Node corresponding to the entry. If the name does not exist in the directory, Stat should return ENOENT.

Stat need not to handle the names "." and "..".

func (*Dir) String 

func (d *Dir) String() string

String converts it to printablee

func (*Dir) Sync 

func (d *Dir) Sync() error

Sync the directory

Note that we don't do anything except return OK

func (*Dir) Sys 

func (d *Dir) Sys() interface{}

Sys returns underlying data source (can be nil) - satisfies Node interface

func (*Dir) Truncate 

func (d *Dir) Truncate(size int64) error

Truncate changes the size of the named file.

func (*Dir) VFS 

func (d *Dir) VFS() *VFS

VFS returns the instance of the VFS

type DirHandle 

type DirHandle struct {
	// contains filtered or unexported fields
}

DirHandle represents an open directory

func (DirHandle) Chdir 

func (h DirHandle) Chdir() error

func (DirHandle) Chmod 

func (h DirHandle) Chmod(mode os.FileMode) error

func (DirHandle) Chown 

func (h DirHandle) Chown(uid, gid int) error

func (*DirHandle) Close 

func (fh *DirHandle) Close() (err error)

Close closes the handle

func (DirHandle) Fd 

func (h DirHandle) Fd() uintptr

func (DirHandle) Flush 

func (h DirHandle) Flush() (err error)

func (DirHandle) Name 

func (h DirHandle) Name() string

func (*DirHandle) Node 

func (fh *DirHandle) Node() Node

Node returns the Node assocuated with this - satisfies Noder interface

func (DirHandle) Read 

func (h DirHandle) Read(b []byte) (n int, err error)

func (DirHandle) ReadAt 

func (h DirHandle) ReadAt(b []byte, off int64) (n int, err error)

func (*DirHandle) Readdir 

func (fh *DirHandle) Readdir(n int) (fis []os.FileInfo, err error)

Readdir reads the contents of the directory associated with file and returns a slice of up to n FileInfo values, as would be returned by Lstat, in directory order. Subsequent calls on the same file will yield further FileInfos.

If n > 0, Readdir returns at most n FileInfo structures. In this case, if Readdir returns an empty slice, it will return a non-nil error explaining why. At the end of a directory, the error is io.EOF.

If n <= 0, Readdir returns all the FileInfo from the directory in a single slice. In this case, if Readdir succeeds (reads all the way to the end of the directory), it returns the slice and a nil error. If it encounters an error before the end of the directory, Readdir returns the FileInfo read until that point and a non-nil error.

func (*DirHandle) Readdirnames 

func (fh *DirHandle) Readdirnames(n int) (names []string, err error)

Readdirnames reads and returns a slice of names from the directory f.

If n > 0, Readdirnames returns at most n names. In this case, if Readdirnames returns an empty slice, it will return a non-nil error explaining why. At the end of a directory, the error is io.EOF.

If n <= 0, Readdirnames returns all the names from the directory in a single slice. In this case, if Readdirnames succeeds (reads all the way to the end of the directory), it returns the slice and a nil error. If it encounters an error before the end of the directory, Readdirnames returns the names read until that point and a non-nil error.

func (DirHandle) Release 

func (h DirHandle) Release() (err error)

func (DirHandle) Seek 

func (h DirHandle) Seek(offset int64, whence int) (ret int64, err error)

func (*DirHandle) Stat 

func (fh *DirHandle) Stat() (fi os.FileInfo, err error)

Stat returns info about the current directory

func (*DirHandle) String 

func (fh *DirHandle) String() string

String converts it to printable

func (DirHandle) Sync 

func (h DirHandle) Sync() error

func (DirHandle) Truncate 

func (h DirHandle) Truncate(size int64) error

func (DirHandle) Write 

func (h DirHandle) Write(b []byte) (n int, err error)

func (DirHandle) WriteAt 

func (h DirHandle) WriteAt(b []byte, off int64) (n int, err error)

func (DirHandle) WriteString 

func (h DirHandle) WriteString(s string) (n int, err error)

type Error 

type Error byte

Error describes low level errors in a cross platform way. 

const (
	OK Error = iota
	ENOTEMPTY
	ESPIPE
	EBADF
	EROFS
	ENOSYS
)

Low level errors

func (Error) Error 

func (e Error) Error() string

Error renders the error as a string

type File 

type File struct {
	// contains filtered or unexported fields
}

File represents a file

func (*File) Dir 

func (f *File) Dir() *Dir

Dir returns the directory this file is in

func (*File) DirEntry 

func (f *File) DirEntry() (entry fs.DirEntry)

DirEntry returns the underlying fs.DirEntry - may be nil

func (*File) Inode 

func (f *File) Inode() uint64

Inode returns the inode number - satisfies Node interface

func (*File) IsDir 

func (f *File) IsDir() bool

IsDir returns false for File - satisfies Node interface

func (*File) IsFile 

func (f *File) IsFile() bool

IsFile returns true for File - satisfies Node interface

func (*File) ModTime 

func (f *File) ModTime() (modTime time.Time)

ModTime returns the modified time of the file

if NoModTime is set then it returns the mod time of the directory

func (*File) Mode 

func (f *File) Mode() (mode os.FileMode)

Mode bits of the file or directory - satisfies Node interface

func (*File) Name 

func (f *File) Name() (name string)

Name (base) of the directory - satisfies Node interface

func (*File) Node 

func (f *File) Node() Node

Node returns the Node assocuated with this - satisfies Noder interface

func (*File) Open 

func (f *File) Open(flags int) (fd Handle, err error)

Open a file according to the flags provided 

O_RDONLY open the file read-only.
O_WRONLY open the file write-only.
O_RDWR   open the file read-write.

O_APPEND append data to the file when writing.
O_CREATE create a new file if none exists.
O_EXCL   used with O_CREATE, file must not exist
O_SYNC   open for synchronous I/O.
O_TRUNC  if possible, truncate file when opene

We ignore O_SYNC and O_EXCL

func (*File) Path 

func (f *File) Path() string

Path returns the full path of the file

func (*File) Remove 

func (f *File) Remove() error

Remove the file

func (*File) RemoveAll 

func (f *File) RemoveAll() error

RemoveAll the file - same as remove for files

func (*File) SetModTime 

func (f *File) SetModTime(modTime time.Time) error

SetModTime sets the modtime for the file

func (*File) Size 

func (f *File) Size() int64

Size of the file

func (*File) String 

func (f *File) String() string

String converts it to printable

func (*File) Sync 

func (f *File) Sync() error

Sync the file

Note that we don't do anything except return OK

func (*File) Sys 

func (f *File) Sys() interface{}

Sys returns underlying data source (can be nil) - satisfies Node interface

func (*File) Truncate 

func (f *File) Truncate(size int64) (err error)

Truncate changes the size of the named file.

func (*File) VFS 

func (f *File) VFS() *VFS

VFS returns the instance of the VFS

type Handle 

type Handle interface {
	OsFiler
	// Additional methods useful for FUSE filesystems
	Flush() error
	Release() error
	Node() Node
}

Handle is the interface statisified by open files or directories. It is the methods on *os.File, plus a few more useful for FUSE filingsystems. Not all of them are supported.

type Node 

type Node interface {
	os.FileInfo
	IsFile() bool
	Inode() uint64
	SetModTime(modTime time.Time) error
	Sync() error
	Remove() error
	RemoveAll() error
	DirEntry() fs.DirEntry
	VFS() *VFS
	Open(flags int) (Handle, error)
	Truncate(size int64) error
	Path() string
}

Node represents either a directory (*Dir) or a file (*File)

type Noder 

type Noder interface {
	fmt.Stringer
	Node() Node
}

Noder represents something which can return a node

type Nodes 

type Nodes []Node

Nodes is a slice of Node

func (Nodes) Len 

func (ns Nodes) Len() int

Sort functions

func (Nodes) Less 

func (ns Nodes) Less(i, j int) bool

func (Nodes) Swap 

func (ns Nodes) Swap(i, j int)

type Options 

type Options struct {
	NoSeek            bool          // don't allow seeking if set
	NoChecksum        bool          // don't check checksums if set
	ReadOnly          bool          // if set VFS is read only
	NoModTime         bool          // don't read mod times for files
	DirCacheTime      time.Duration // how long to consider directory listing cache valid
	PollInterval      time.Duration
	Umask             int
	UID               uint32
	GID               uint32
	DirPerms          os.FileMode
	FilePerms         os.FileMode
	ChunkSize         fs.SizeSuffix // if > 0 read files in chunks
	ChunkSizeLimit    fs.SizeSuffix // if > ChunkSize double the chunk size after each chunk until reached
	CacheMode         CacheMode
	CacheMaxAge       time.Duration
	CacheMaxSize      fs.SizeSuffix
	CachePollInterval time.Duration
	CaseInsensitive   bool
	WriteWait         time.Duration // time to wait for in-sequence write
	ReadWait          time.Duration // time to wait for in-sequence read
}

Options is options for creating the vfs

type OsFiler 

type OsFiler interface {
	Chdir() error
	Chmod(mode os.FileMode) error
	Chown(uid, gid int) error
	Close() error
	Fd() uintptr
	Name() string
	Read(b []byte) (n int, err error)
	ReadAt(b []byte, off int64) (n int, err error)
	Readdir(n int) ([]os.FileInfo, error)
	Readdirnames(n int) (names []string, err error)
	Seek(offset int64, whence int) (ret int64, err error)
	Stat() (os.FileInfo, error)
	Sync() error
	Truncate(size int64) error
	Write(b []byte) (n int, err error)
	WriteAt(b []byte, off int64) (n int, err error)
	WriteString(s string) (n int, err error)
}

OsFiler is the methods on *os.File

type RWFileHandle 

type RWFileHandle struct {
	*os.File
	// contains filtered or unexported fields
}

RWFileHandle is a handle that can be open for read and write.

It will be open to a temporary file which, when closed, will be transferred to the remote.

func (*RWFileHandle) Close 

func (fh *RWFileHandle) Close() error

Close closes the file

func (*RWFileHandle) Flush 

func (fh *RWFileHandle) Flush() error

Flush is called each time the file or directory is closed. Because there can be multiple file descriptors referring to a single opened file, Flush can be called multiple times.

func (*RWFileHandle) Node 

func (fh *RWFileHandle) Node() Node

Node returns the Node assocuated with this - satisfies Noder interface

func (*RWFileHandle) Read 

func (fh *RWFileHandle) Read(b []byte) (n int, err error)

Read bytes from the file

func (*RWFileHandle) ReadAt 

func (fh *RWFileHandle) ReadAt(b []byte, off int64) (n int, err error)

ReadAt bytes from the file at off

func (*RWFileHandle) Release 

func (fh *RWFileHandle) Release() error

Release is called when we are finished with the file handle

It isn't called directly from userspace so the error is ignored by the kernel

func (*RWFileHandle) Seek 

func (fh *RWFileHandle) Seek(offset int64, whence int) (ret int64, err error)

Seek to new file position

func (*RWFileHandle) Size 

func (fh *RWFileHandle) Size() int64

Size returns the size of the underlying file

func (*RWFileHandle) Stat 

func (fh *RWFileHandle) Stat() (os.FileInfo, error)

Stat returns info about the file

func (*RWFileHandle) String 

func (fh *RWFileHandle) String() string

String converts it to printable

func (*RWFileHandle) Sync 

func (fh *RWFileHandle) Sync() error

Sync commits the current contents of the file to stable storage. Typically, this means flushing the file system's in-memory copy of recently written data to disk.

func (*RWFileHandle) Truncate 

func (fh *RWFileHandle) Truncate(size int64) (err error)

Truncate file to given size

func (*RWFileHandle) Write 

func (fh *RWFileHandle) Write(b []byte) (n int, err error)

Write bytes to the file

func (*RWFileHandle) WriteAt 

func (fh *RWFileHandle) WriteAt(b []byte, off int64) (n int, err error)

WriteAt bytes to the file at off

func (*RWFileHandle) WriteString 

func (fh *RWFileHandle) WriteString(s string) (n int, err error)

WriteString a string to the file

type ReadFileHandle 

type ReadFileHandle struct {
	// contains filtered or unexported fields
}

ReadFileHandle is an open for read file handle on a File

func (ReadFileHandle) Chdir 

func (h ReadFileHandle) Chdir() error

func (ReadFileHandle) Chmod 

func (h ReadFileHandle) Chmod(mode os.FileMode) error

func (ReadFileHandle) Chown 

func (h ReadFileHandle) Chown(uid, gid int) error

func (*ReadFileHandle) Close 

func (fh *ReadFileHandle) Close() error

Close closes the file

func (ReadFileHandle) Fd 

func (h ReadFileHandle) Fd() uintptr

func (*ReadFileHandle) Flush 

func (fh *ReadFileHandle) Flush() error

Flush is called each time the file or directory is closed. Because there can be multiple file descriptors referring to a single opened file, Flush can be called multiple times.

func (ReadFileHandle) Name 

func (h ReadFileHandle) Name() string

func (*ReadFileHandle) Node 

func (fh *ReadFileHandle) Node() Node

Node returns the Node assocuated with this - satisfies Noder interface

func (*ReadFileHandle) Read 

func (fh *ReadFileHandle) Read(p []byte) (n int, err error)

Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered. Even if Read returns n < len(p), it may use all of p as scratch space during the call. If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more.

When Read encounters an error or end-of-file condition after successfully reading n > 0 bytes, it returns the number of bytes read. It may return the (non-nil) error from the same call or return the error (and n == 0) from a subsequent call. An instance of this general case is that a Reader returning a non-zero number of bytes at the end of the input stream may return either err == EOF or err == nil. The next Read should return 0, EOF.

Callers should always process the n > 0 bytes returned before considering the error err. Doing so correctly handles I/O errors that happen after reading some bytes and also both of the allowed EOF behaviors.

Implementations of Read are discouraged from returning a zero byte count with a nil error, except when len(p) == 0. Callers should treat a return of 0 and nil as indicating that nothing happened; in particular it does not indicate EOF.

Implementations must not retain p.

func (*ReadFileHandle) ReadAt 

func (fh *ReadFileHandle) ReadAt(p []byte, off int64) (n int, err error)

ReadAt reads len(p) bytes into p starting at offset off in the underlying input source. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered.

When ReadAt returns n < len(p), it returns a non-nil error explaining why more bytes were not returned. In this respect, ReadAt is stricter than Read.

Even if ReadAt returns n < len(p), it may use all of p as scratch space during the call. If some data is available but not len(p) bytes, ReadAt blocks until either all the data is available or an error occurs. In this respect ReadAt is different from Read.

If the n = len(p) bytes returned by ReadAt are at the end of the input source, ReadAt may return either err == EOF or err == nil.

If ReadAt is reading from an input source with a seek offset, ReadAt should not affect nor be affected by the underlying seek offset.

Clients of ReadAt can execute parallel ReadAt calls on the same input source.

Implementations must not retain p.

func (ReadFileHandle) Readdir 

func (h ReadFileHandle) Readdir(n int) ([]os.FileInfo, error)

func (ReadFileHandle) Readdirnames 

func (h ReadFileHandle) Readdirnames(n int) (names []string, err error)

func (*ReadFileHandle) Release 

func (fh *ReadFileHandle) Release() error

Release is called when we are finished with the file handle

It isn't called directly from userspace so the error is ignored by the kernel

func (*ReadFileHandle) Seek 

func (fh *ReadFileHandle) Seek(offset int64, whence int) (n int64, err error)

Seek the file - returns ESPIPE if seeking isn't possible

func (*ReadFileHandle) Size 

func (fh *ReadFileHandle) Size() int64

Size returns the size of the underlying file

func (*ReadFileHandle) Stat 

func (fh *ReadFileHandle) Stat() (os.FileInfo, error)

Stat returns info about the file

func (*ReadFileHandle) String 

func (fh *ReadFileHandle) String() string

String converts it to printable

func (ReadFileHandle) Sync 

func (h ReadFileHandle) Sync() error

func (ReadFileHandle) Truncate 

func (h ReadFileHandle) Truncate(size int64) error

func (ReadFileHandle) Write 

func (h ReadFileHandle) Write(b []byte) (n int, err error)

func (ReadFileHandle) WriteAt 

func (h ReadFileHandle) WriteAt(b []byte, off int64) (n int, err error)

func (ReadFileHandle) WriteString 

func (h ReadFileHandle) WriteString(s string) (n int, err error)

type VFS 

type VFS struct {
	Opt Options
	// contains filtered or unexported fields
}

VFS represents the top level filing system

func New 

func New(f fs.Fs, opt *Options) *VFS

New creates a new VFS and root directory. If opt is nil, then DefaultOpt will be used

func (*VFS) CleanUp 

func (vfs *VFS) CleanUp() error

CleanUp deletes the contents of the on disk cache

func (*VFS) FlushDirCache 

func (vfs *VFS) FlushDirCache()

FlushDirCache empties the directory cache

func (*VFS) Fs added in v1.6.1 

func (vfs *VFS) Fs() fs.Fs

Fs returns the Fs passed into the New call

func (*VFS) OpenFile 

func (vfs *VFS) OpenFile(name string, flags int, perm os.FileMode) (fd Handle, err error)

OpenFile a file according to the flags and perm provided

func (*VFS) Rename 

func (vfs *VFS) Rename(oldName, newName string) error

Rename oldName to newName

func (*VFS) Root 

func (vfs *VFS) Root() (*Dir, error)

Root returns the root node

func (*VFS) SetCacheMode 

func (vfs *VFS) SetCacheMode(cacheMode CacheMode)

SetCacheMode change the cache mode

func (*VFS) Shutdown 

func (vfs *VFS) Shutdown()

Shutdown stops any background go-routines

func (*VFS) Stat 

func (vfs *VFS) Stat(path string) (node Node, err error)

Stat finds the Node by path starting from the root

It is the equivalent of os.Stat - Node contains the os.FileInfo interface.

func (*VFS) StatParent 

func (vfs *VFS) StatParent(name string) (dir *Dir, leaf string, err error)

StatParent finds the parent directory and the leaf name of a path

func (*VFS) Statfs 

func (vfs *VFS) Statfs() (total, used, free int64)

Statfs returns into about the filing system if known

The values will be -1 if they aren't known

This information is cached for the DirCacheTime interval

func (*VFS) WaitForWriters 

func (vfs *VFS) WaitForWriters(timeout time.Duration)

WaitForWriters sleeps until all writers have finished or time.Duration has elapsed

type WriteFileHandle 

type WriteFileHandle struct {
	// contains filtered or unexported fields
}

WriteFileHandle is an open for write handle on a File

func (WriteFileHandle) Chdir 

func (h WriteFileHandle) Chdir() error

func (WriteFileHandle) Chmod 

func (h WriteFileHandle) Chmod(mode os.FileMode) error

func (WriteFileHandle) Chown 

func (h WriteFileHandle) Chown(uid, gid int) error

func (*WriteFileHandle) Close 

func (fh *WriteFileHandle) Close() error

Close closes the file

func (WriteFileHandle) Fd 

func (h WriteFileHandle) Fd() uintptr

func (*WriteFileHandle) Flush 

func (fh *WriteFileHandle) Flush() error

Flush is called on each close() of a file descriptor. So if a filesystem wants to return write errors in close() and the file has cached dirty data, this is a good place to write back data and return any errors. Since many applications ignore close() errors this is not always useful.

NOTE: The flush() method may be called more than once for each open(). This happens if more than one file descriptor refers to an opened file due to dup(), dup2() or fork() calls. It is not possible to determine if a flush is final, so each flush should be treated equally. Multiple write-flush sequences are relatively rare, so this shouldn't be a problem.

Filesystems shouldn't assume that flush will always be called after some writes, or that if will be called at all.

func (WriteFileHandle) Name 

func (h WriteFileHandle) Name() string

func (*WriteFileHandle) Node 

func (fh *WriteFileHandle) Node() Node

Node returns the Node assocuated with this - satisfies Noder interface

func (*WriteFileHandle) Offset 

func (fh *WriteFileHandle) Offset() (offset int64)

Offset returns the offset of the file pointer

func (*WriteFileHandle) Read 

func (fh *WriteFileHandle) Read(p []byte) (n int, err error)

Read reads up to len(p) bytes into p.

func (*WriteFileHandle) ReadAt 

func (fh *WriteFileHandle) ReadAt(p []byte, off int64) (n int, err error)

ReadAt reads len(p) bytes into p starting at offset off in the underlying input source. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered.

func (WriteFileHandle) Readdir 

func (h WriteFileHandle) Readdir(n int) ([]os.FileInfo, error)

func (WriteFileHandle) Readdirnames 

func (h WriteFileHandle) Readdirnames(n int) (names []string, err error)

func (*WriteFileHandle) Release 

func (fh *WriteFileHandle) Release() error

Release is called when we are finished with the file handle

It isn't called directly from userspace so the error is ignored by the kernel

func (WriteFileHandle) Seek 

func (h WriteFileHandle) Seek(offset int64, whence int) (ret int64, err error)

func (*WriteFileHandle) Stat 

func (fh *WriteFileHandle) Stat() (os.FileInfo, error)

Stat returns info about the file

func (*WriteFileHandle) String 

func (fh *WriteFileHandle) String() string

String converts it to printable

func (*WriteFileHandle) Sync 

func (fh *WriteFileHandle) Sync() error

Sync commits the current contents of the file to stable storage. Typically, this means flushing the file system's in-memory copy of recently written data to disk.

func (*WriteFileHandle) Truncate 

func (fh *WriteFileHandle) Truncate(size int64) (err error)

Truncate file to given size

func (*WriteFileHandle) Write 

func (fh *WriteFileHandle) Write(p []byte) (n int, err error)

Write writes len(p) bytes from p to the underlying data stream. It returns the number of bytes written from p (0 <= n <= len(p)) and any error encountered that caused the write to stop early. Write must return a non-nil error if it returns n < len(p). Write must not modify the slice data, even temporarily.

Implementations must not retain p.

func (*WriteFileHandle) WriteAt 

func (fh *WriteFileHandle) WriteAt(p []byte, off int64) (n int, err error)

WriteAt writes len(p) bytes from p to the underlying data stream at offset off. It returns the number of bytes written from p (0 <= n <= len(p)) and any error encountered that caused the write to stop early. WriteAt must return a non-nil error if it returns n < len(p).

If WriteAt is writing to a destination with a seek offset, WriteAt should not affect nor be affected by the underlying seek offset.

Clients of WriteAt can execute parallel WriteAt calls on the same destination if the ranges do not overlap.

Implementations must not retain p.

func (*WriteFileHandle) WriteString 

func (fh *WriteFileHandle) WriteString(s string) (n int, err error)

WriteString a string to the file

Source Files

View all Source files

Directories

Path Synopsis
Test the VFS to exhaustion, specifically looking for deadlocks Run on a mounted filesystem
 Test the VFS to exhaustion, specifically looking for deadlocks Run on a mounted filesystem
Package vfsflags implements command line flags to set up a vfs
 Package vfsflags implements command line flags to set up a vfs

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.