filesystem

package
v1.0.0-alpha.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 24, 2023 License: MIT Imports: 37 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsErrorCode 

func IsErrorCode(err error, code ErrorCode) bool

IsErrorCode checks if "err" is a filesystem Error type. If so, it will then drop in and check that the error code is the same as the provided ErrorCode passed in "code".

func IsFilesystemError 

func IsFilesystemError(err error) bool

IsFilesystemError checks if the given error is one of the Filesystem errors.

func NewBadPathResolution 

func NewBadPathResolution(path string, resolved string) error

NewBadPathResolution returns a new BadPathResolution error.

Types

type Archive 

type Archive struct {
	// BasePath is the absolute path to create the archive from where Files and Ignore are
	// relative to.
	BasePath string

	// Ignore is a gitignore string (most likely read from a file) of files to ignore
	// from the archive.
	Ignore string

	// Files specifies the files to archive, this takes priority over the Ignore option, if
	// unspecified, all files in the BasePath will be archived unless Ignore is set.
	Files []string

	// Progress wraps the writer of the archive to pass through the progress tracker.
	Progress *progress.Progress
}

func (*Archive) Create 

func (a *Archive) Create(ctx context.Context, dst string) error

Create creates an archive at dst with all the files defined in the included Files array.

func (*Archive) Stream 

func (a *Archive) Stream(ctx context.Context, w io.Writer) error

Stream .

type BasicSFTPManager 

type BasicSFTPManager struct {
	// contains filtered or unexported fields
}

BasicSFTPManager is implements SFTPManager and supports basic reconnection on disconnect for SFTPConn returned by NewClient

func NewBasicSFTPManager 

func NewBasicSFTPManager(connString string, config *ssh.ClientConfig) *BasicSFTPManager

NewBasicSFTPManager returns a BasicSFTPManager

func (*BasicSFTPManager) Close 

func (m *BasicSFTPManager) Close() error

Close closes all connections managed by this manager

func (*BasicSFTPManager) GetConnection 

func (m *BasicSFTPManager) GetConnection() (*SFTPConn, error)

GetConnection returns one of the existing connections the manager knows about. If there is no connections, we create a new one instead.

func (*BasicSFTPManager) NewClient 

func (m *BasicSFTPManager) NewClient() (*SFTPConn, error)

NewClient returns an SFTPConn and ensures the underlying connection reconnects on failure

type Error 

type Error struct {
	// contains filtered or unexported fields
}

func (*Error) Code 

func (e *Error) Code() ErrorCode

Code returns the ErrorCode for this specific error instance.

func (*Error) Error 

func (e *Error) Error() string

Returns a human-readable error string to identify the Error by.

func (*Error) Unwrap 

func (e *Error) Unwrap() error

Unwrap returns the underlying cause of this filesystem error. In some causes there may not be a cause present, in which case nil will be returned.

type ErrorCode 

type ErrorCode string
const (
	ErrCodeIsDirectory    ErrorCode = "E_ISDIR"
	ErrCodeDiskSpace      ErrorCode = "E_NODISK"
	ErrCodeUnknownArchive ErrorCode = "E_UNKNFMT"
	ErrCodePathResolution ErrorCode = "E_BADPATH"
	ErrCodeDenylistFile   ErrorCode = "E_DENYLIST"
	ErrCodeUnknownError   ErrorCode = "E_UNKNOWN"
	ErrNotExist           ErrorCode = "E_NOTEXIST"
)

type Filesystem 

type Filesystem struct {
	// contains filtered or unexported fields
}

func New 

func New(root string, size int64, denylist []string, addr string) *Filesystem

New creates a new Filesystem instance for a given server.

func (*Filesystem) CachedUsage 

func (fs *Filesystem) CachedUsage() int64

Returns the cached value for the amount of disk space used by the filesystem. Do not rely on this function for critical logical checks. It should only be used in areas where the actual disk usage does not need to be perfect, e.g. API responses for server resource usage.

func (*Filesystem) Chmod 

func (fs *Filesystem) Chmod(path string, mode os.FileMode) error

func (*Filesystem) Chown 

func (fs *Filesystem) Chown(path string) error

Recursively iterates over a file or directory and sets the permissions on all of the underlying files. Iterate over all of the files and directories. If it is a file just go ahead and perform the chown operation. Otherwise dig deeper into the directory until we've run out of directories to dig into.

func (*Filesystem) Chtimes 

func (fs *Filesystem) Chtimes(path string, atime, mtime time.Time) error

func (*Filesystem) CompressFiles 

func (fs *Filesystem) CompressFiles(dir string, paths []string) (os.FileInfo, error)

CompressFiles compresses all the files matching the given paths in the specified directory. This function also supports passing nested paths to only compress certain files and folders when working in a larger directory. This effectively creates a local backup, but rather than ignoring specific files and folders, it takes an allow-list of files and folders.

All paths are relative to the dir that is passed in as the first argument, and the compressed file will be placed at that location named `archive-{date}.tar.gz`.

func (*Filesystem) Copy 

func (fs *Filesystem) Copy(p string) error

Copies a given file to the same location and appends a suffix to the file to indicate that it has been copied.

func (*Filesystem) CreateDirectory 

func (fs *Filesystem) CreateDirectory(name string, p string) error

Creates a new directory (name) at a specified path (p) for the server.

func (*Filesystem) DecompressFile 

func (fs *Filesystem) DecompressFile(ctx context.Context, dir string, file string) error

DecompressFile will decompress a file in a given directory by using the archiver tool to infer the file type and go from there. This will walk over all the files within the given archive and ensure that there is not a zip-slip attack being attempted by validating that the final path is within the server data directory.

func (*Filesystem) DecompressFileUnsafe 

func (fs *Filesystem) DecompressFileUnsafe(ctx context.Context, dir string, file string) error

DecompressFileUnsafe will decompress any file on the local disk without checking if it is owned by the server. The file will be SAFELY decompressed and extracted into the server's directory.

func (*Filesystem) Delete 

func (fs *Filesystem) Delete(p string) error

Delete removes a file or folder from the system. Prevents the user from accidentally (or maliciously) removing their root server data directory.

func (*Filesystem) DiskUsage 

func (fs *Filesystem) DiskUsage(allowStaleValue bool) (int64, error)

Internal helper function to allow other parts of the codebase to check the total used disk space as needed without overly taxing the system. This will prioritize the value from the cache to avoid excessive IO usage. We will only walk the filesystem and determine the size of the directory if there is no longer a cached value.

If "allowStaleValue" is set to true, a stale value MAY be returned to the caller if there is an expired cache value AND there is currently another lookup in progress. If there is no cached value but no other lookup is in progress, a fresh disk space response will be returned to the caller.

This is primarily to avoid a bunch of I/O operations from piling up on the server, especially on servers with a large amount of files.

func (*Filesystem) File 

func (fs *Filesystem) File(p string) (*sftp.File, os.FileInfo, error)

File returns a reader for a file instance as well as the stat information.

func (*Filesystem) HasSpaceAvailable 

func (fs *Filesystem) HasSpaceAvailable(allowStaleValue bool) bool

Determines if the directory a file is trying to be added to has enough space available for the file to be written to.

Because determining the amount of space being used by a server is a taxing operation we will load it all up into a cache and pull from that as long as the key is not expired.

This operation will potentially block unless allowStaleValue is set to true. See the documentation on DiskUsage for how this affects the call.

func (*Filesystem) HasSpaceErr 

func (fs *Filesystem) HasSpaceErr(allowStaleValue bool) error

The same concept as HasSpaceAvailable however this will return an error if there is no space, rather than a boolean value.

func (*Filesystem) HasSpaceFor 

func (fs *Filesystem) HasSpaceFor(size int64) error

Helper function to determine if a server has space available for a file of a given size. If space is available, no error will be returned, otherwise an ErrNotEnoughSpace error will be raised.

func (*Filesystem) IsIgnored 

func (fs *Filesystem) IsIgnored(paths ...string) error

Checks if the given file or path is in the server's file denylist. If so, an Error is returned, otherwise nil is returned.

func (*Filesystem) ListDirectory 

func (fs *Filesystem) ListDirectory(p string) ([]Stat, error)

ListDirectory lists the contents of a given directory and returns stat information about each file and folder within it.

func (*Filesystem) MaxDisk 

func (fs *Filesystem) MaxDisk() int64

Returns the maximum amount of disk space that this Filesystem instance is allowed to use.

func (*Filesystem) ParallelSafePath 

func (fs *Filesystem) ParallelSafePath(paths []string) ([]string, error)

Executes the fs.SafePath function in parallel against an array of paths. If any of the calls fails an error will be returned.

func (*Filesystem) Path 

func (fs *Filesystem) Path() string

Path returns the root path for the Filesystem instance.

func (*Filesystem) Rename 

func (fs *Filesystem) Rename(from string, to string) error

Rename moves (or renames) a file or directory.

func (*Filesystem) SafePath 

func (fs *Filesystem) SafePath(p string) (string, error)

Normalizes a directory being passed in to ensure the user is not able to escape from their data directory. After normalization if the directory is still within their home path it is returned. If they managed to "escape" an error will be returned.

This logic is actually copied over from the SFTP server code. Ideally that eventually either gets ported into this application, or is able to make use of this package.

func (*Filesystem) SetDiskLimit 

func (fs *Filesystem) SetDiskLimit(i int64)

Sets the disk space limit for this Filesystem instance.

func (*Filesystem) SetManager 

func (fs *Filesystem) SetManager(addr string) error

func (*Filesystem) SpaceAvailableForDecompression 

func (fs *Filesystem) SpaceAvailableForDecompression(ctx context.Context, dir string, file string) error

SpaceAvailableForDecompression looks through a given archive and determines if decompressing it would put the server over its allocated disk space limit.

func (*Filesystem) Stat 

func (fs *Filesystem) Stat(p string) (Stat, error)

Stat stats a file or folder and returns the base stat object from go along with the MIME data that can be used for editing files.

func (*Filesystem) Touch 

func (fs *Filesystem) Touch(p string, flag int) (*sftp.File, error)

Touch acts by creating the given file and path on the disk if it is not present already. If it is present, the file is opened using the defaults which will truncate the contents. The opened file is then returned to the caller.

func (*Filesystem) TruncateRootDirectory 

func (fs *Filesystem) TruncateRootDirectory() error

TruncateRootDirectory removes _all_ files and directories from a server's data directory and resets the used disk space to zero.

func (*Filesystem) Writefile 

func (fs *Filesystem) Writefile(p string, r io.Reader) error

Writefile writes a file to the system. If the file does not already exist one will be created. This will also properly recalculate the disk space used by the server when writing new files or modifying existing ones.

type SFTPConn 

type SFTPConn struct {
	sync.Mutex
	// contains filtered or unexported fields
}

SFTPConn is a wrapped *sftp.Client

func NewSFTPConn 

func NewSFTPConn(client *ssh.Client, sftpClient *sftp.Client) *SFTPConn

NewSFTPConn creates an SFTPConn, mostly used for testing and should not really be used otherwise.

func (*SFTPConn) Close 

func (s *SFTPConn) Close() error

Close closes the underlying connections

func (*SFTPConn) GetClient 

func (s *SFTPConn) GetClient() *sftp.Client

GetClient returns the underlying *sftp.Client

type SFTPManager 

type SFTPManager interface {
	NewClient() (*SFTPConn, error)
	GetConnection() (*SFTPConn, error)
	SetLogger(logger *log.Logger)
	Close() error
}

SFTPManager is an interface for managing SFTP connections

type SFTPWalkerFunc 

type SFTPWalkerFunc func(path string, fileInfo os.FileInfo, err error) error

type SpaceCheckingOpts 

type SpaceCheckingOpts struct {
	AllowStaleResponse bool
}

type Stat 

type Stat struct {
	os.FileInfo
	Mimetype string
}

func (*Stat) CTime 

func (s *Stat) CTime() time.Time

Returns the time that the file/folder was created.

func (*Stat) MarshalJSON 

func (s *Stat) MarshalJSON() ([]byte, error)

type TarProgress 

type TarProgress struct {
	*tar.Writer
	// contains filtered or unexported fields
}

TarProgress .

func NewTarProgress 

func NewTarProgress(w *tar.Writer, p *progress.Progress) *TarProgress

NewTarProgress .

func (*TarProgress) Write 

func (p *TarProgress) Write(v []byte) (int, error)

Write .

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.