filesystem

Documentation

Index

• Variables
• type Filesystem
  • func New(root string, size int64) *Filesystem
  • func (fs *Filesystem) CachedUsage() int64
  • func (fs *Filesystem) Chown(path string) error
  • func (fs *Filesystem) CompressFiles(dir string, paths []string) (os.FileInfo, error)
  • func (fs *Filesystem) Copy(p string) error
  • func (fs *Filesystem) CreateDirectory(name string, p string) error
  • func (fs *Filesystem) DecompressFile(dir string, file string) error
  • func (fs *Filesystem) Delete(p string) error
  • func (fs *Filesystem) DirectorySize(dir string) (int64, error)
  • func (fs *Filesystem) DiskUsage(allowStaleValue bool) (int64, error)
  • func (fs *Filesystem) GetIncludedFiles(dir string, ignored []string) (*backup.IncludedFiles, error)
  • func (fs *Filesystem) HasSpaceAvailable(allowStaleValue bool) bool
  • func (fs *Filesystem) ListDirectory(p string) ([]*Stat, error)
  • func (fs *Filesystem) MaxDisk() int64
  • func (fs *Filesystem) ParallelSafePath(paths []string) ([]string, error)
  • func (fs *Filesystem) Path() string
  • func (fs *Filesystem) Readfile(p string, w io.Writer) error
  • func (fs *Filesystem) Rename(from string, to string) error
  • func (fs *Filesystem) SafePath(p string) (string, error)
  • func (fs *Filesystem) SetDiskLimit(i int64)
  • func (fs *Filesystem) SpaceAvailableForDecompression(dir string, file string) (bool, error)
  • func (fs *Filesystem) Stat(p string) (*Stat, error)
  • func (fs *Filesystem) Writefile(p string, r io.Reader) error
• type SpaceCheckingOpts
• type Stat
  • func (s *Stat) CTime() time.Time
  • func (s *Stat) MarshalJSON() ([]byte, error)

Variables

var ErrBadPathResolution = errors.New("filesystem: invalid path resolution")

var ErrIsDirectory = errors.New("filesystem: is a directory")

var ErrNotEnoughDiskSpace = errors.New("filesystem: not enough disk space")

var ErrUnknownArchiveFormat = errors.New("filesystem: unknown archive format")

Types

type Filesystem

type Filesystem struct {
	// contains filtered or unexported fields
}

func New

func New(root string, size int64) *Filesystem

Creates a new Filesystem instance for a given server.

func (*Filesystem) CachedUsage

func (fs *Filesystem) CachedUsage() int64

Returns the cached value for the amount of disk space used by the filesystem. Do not rely on this function for critical logical checks. It should only be used in areas where the actual disk usage does not need to be perfect, e.g. API responses for server resource usage.

func (*Filesystem) Chown

func (fs *Filesystem) Chown(path string) error

Recursively iterates over a file or directory and sets the permissions on all of the underlying files. Iterate over all of the files and directories. If it is a file just go ahead and perform the chown operation. Otherwise dig deeper into the directory until we've run out of directories to dig into.

func (*Filesystem) CompressFiles

func (fs *Filesystem) CompressFiles(dir string, paths []string) (os.FileInfo, error)

Compresses all of the files matching the given paths in the specified directory. This function also supports passing nested paths to only compress certain files and folders when working in a larger directory. This effectively creates a local backup, but rather than ignoring specific files and folders, it takes an allow-list of files and folders.

All paths are relative to the dir that is passed in as the first argument, and the compressed file will be placed at that location named `archive-{date}.tar.gz`.

func (*Filesystem) Copy

func (fs *Filesystem) Copy(p string) error

Copies a given file to the same location and appends a suffix to the file to indicate that it has been copied.

func (*Filesystem) CreateDirectory

func (fs *Filesystem) CreateDirectory(name string, p string) error

Creates a new directory (name) at a specified path (p) for the server.

func (*Filesystem) DecompressFile

func (fs *Filesystem) DecompressFile(dir string, file string) error

Decompress a file in a given directory by using the archiver tool to infer the file type and go from there. This will walk over all of the files within the given archive and ensure that there is not a zip-slip attack being attempted by validating that the final path is within the server data directory.

func (*Filesystem) Delete

func (fs *Filesystem) Delete(p string) error

Deletes a file or folder from the system. Prevents the user from accidentally (or maliciously) removing their root server data directory.

func (*Filesystem) DirectorySize

func (fs *Filesystem) DirectorySize(dir string) (int64, error)

Determines the directory size of a given location by running parallel tasks to iterate through all of the folders. Returns the size in bytes. This can be a fairly taxing operation on locations with tons of files, so it is recommended that you cache the output.

func (*Filesystem) DiskUsage

func (fs *Filesystem) DiskUsage(allowStaleValue bool) (int64, error)

Internal helper function to allow other parts of the codebase to check the total used disk space as needed without overly taxing the system. This will prioritize the value from the cache to avoid excessive IO usage. We will only walk the filesystem and determine the size of the directory if there is no longer a cached value.

If "allowStaleValue" is set to true, a stale value MAY be returned to the caller if there is an expired cache value AND there is currently another lookup in progress. If there is no cached value but no other lookup is in progress, a fresh disk space response will be returned to the caller.

This is primarily to avoid a bunch of I/O operations from piling up on the server, especially on servers with a large amount of files.

func (*Filesystem) GetIncludedFiles

func (fs *Filesystem) GetIncludedFiles(dir string, ignored []string) (*backup.IncludedFiles, error)

Given a directory, iterate through all of the files and folders within it and determine if they should be included in the output based on an array of ignored matches. This uses standard .gitignore formatting to make that determination.

If no ignored files are passed through you'll get the entire directory listing.

func (*Filesystem) HasSpaceAvailable

func (fs *Filesystem) HasSpaceAvailable(allowStaleValue bool) bool

Determines if the directory a file is trying to be added to has enough space available for the file to be written to.

Because determining the amount of space being used by a server is a taxing operation we will load it all up into a cache and pull from that as long as the key is not expired.

This operation will potentially block unless allowStaleValue is set to true. See the documentation on DiskUsage for how this affects the call.

func (*Filesystem) ListDirectory

func (fs *Filesystem) ListDirectory(p string) ([]*Stat, error)

Lists the contents of a given directory and returns stat information about each file and folder within it.

func (*Filesystem) MaxDisk

func (fs *Filesystem) MaxDisk() int64

Returns the maximum amount of disk space that this Filesystem instance is allowed to use.

func (*Filesystem) ParallelSafePath

func (fs *Filesystem) ParallelSafePath(paths []string) ([]string, error)

Executes the fs.SafePath function in parallel against an array of paths. If any of the calls fails an error will be returned.

func (*Filesystem) Path

func (fs *Filesystem) Path() string

Returns the root path for the Filesystem instance.

func (*Filesystem) Readfile

func (fs *Filesystem) Readfile(p string, w io.Writer) error

Reads a file on the system and returns it as a byte representation in a file reader. This is not the most memory efficient usage since it will be reading the entirety of the file into memory.

func (*Filesystem) Rename

func (fs *Filesystem) Rename(from string, to string) error

Moves (or renames) a file or directory.

func (*Filesystem) SafePath

func (fs *Filesystem) SafePath(p string) (string, error)

Normalizes a directory being passed in to ensure the user is not able to escape from their data directory. After normalization if the directory is still within their home path it is returned. If they managed to "escape" an error will be returned.

This logic is actually copied over from the SFTP server code. Ideally that eventually either gets ported into this application, or is able to make use of this package.

func (*Filesystem) SetDiskLimit

func (fs *Filesystem) SetDiskLimit(i int64)

Sets the disk space limit for this Filesystem instance.

func (*Filesystem) SpaceAvailableForDecompression

func (fs *Filesystem) SpaceAvailableForDecompression(dir string, file string) (bool, error)

Look through a given archive and determine if decompressing it would put the server over its allocated disk space limit.

func (*Filesystem) Stat

func (fs *Filesystem) Stat(p string) (*Stat, error)

Stats a file or folder and returns the base stat object from go along with the MIME data that can be used for editing files.

func (*Filesystem) Writefile

func (fs *Filesystem) Writefile(p string, r io.Reader) error

Writes a file to the system. If the file does not already exist one will be created.

type SpaceCheckingOpts

type SpaceCheckingOpts struct {
	AllowStaleResponse bool
}

type Stat

type Stat struct {
	Info     os.FileInfo
	Mimetype string
}

func (*Stat) CTime

func (s *Stat) CTime() time.Time

Returns the time that the file/folder was created.

func (*Stat) MarshalJSON

func (s *Stat) MarshalJSON() ([]byte, error)