fs

Documentation

Index

• Constants
• func GetFilepath(baseDir, path string) string
• func IsDirOrSymlink(de os.DirEntry) bool
• func IsEmptyDir(path string) bool
• func IsPathExist(path string) bool
• func IsScheduledForRemoval(filename string) bool
• func IsTemporaryFileName(fn string) bool
• func MustClose(f *os.File)
• func MustCopyDirectory(srcPath, dstPath string)
• func MustCopyFile(srcPath, dstPath string)
• func MustCreateFlockFile(dir string) *os.File
• func MustFileSize(path string) uint64
• func MustGetFreeSpace(path string) uint64
• func MustHardLinkFiles(srcDir, dstDir string)
• func MustMkdirFailIfExist(path string)
• func MustMkdirIfNotExist(path string)
• func MustReadData(r filestream.ReadCloser, data []byte)
• func MustReadDir(dir string) []os.DirEntry
• func MustRemoveAll(path string)
• func MustRemoveDirAtomic(dir string)
• func MustRemoveTemporaryDirs(dir string)
• func MustStopDirRemover()
• func MustSymlinkRelative(srcPath, dstPath string)
• func MustSyncPath(path string)
• func MustWriteAtomic(path string, data []byte, canOverwrite bool)
• func MustWriteData(w filestream.WriteCloser, data []byte)
• func MustWriteSync(path string, data []byte)
• func ReadFileOrHTTP(path string) ([]byte, error)
• func RemoveDirContents(dir string)
• type MustReadAtCloser
• type ReaderAt
  • func MustOpenReaderAt(path string) *ReaderAt
  • func (r *ReaderAt) MustClose()
  • func (r *ReaderAt) MustFadviseSequentialRead(prefetch bool)
  • func (r *ReaderAt) MustReadAt(p []byte, off int64)
  • func (r *ReaderAt) SetUseLocalStats()

Constants

const FlockFilename = "flock.lock"

FlockFilename is the filename for the file created by MustCreateFlockFile().

Functions

func GetFilepath

func GetFilepath(baseDir, path string) string

GetFilepath returns full path to file for the given baseDir and path.

func IsDirOrSymlink

func IsDirOrSymlink(de os.DirEntry) bool

IsDirOrSymlink returns true if de is directory or symlink.

func IsEmptyDir

func IsEmptyDir(path string) bool

IsEmptyDir returns true if path points to empty directory.

func IsPathExist

func IsPathExist(path string) bool

IsPathExist returns whether the given path exists.

func IsScheduledForRemoval

func IsScheduledForRemoval(filename string) bool

IsScheduledForRemoval returns true if the filename contains .must-remove. substring

func IsTemporaryFileName

func IsTemporaryFileName(fn string) bool

IsTemporaryFileName returns true if fn matches temporary file name pattern from MustWriteAtomic.

func MustClose

func MustClose(f *os.File)

MustClose must close the given file f.

func MustCopyDirectory

func MustCopyDirectory(srcPath, dstPath string)

MustCopyDirectory copies all the files in srcPath to dstPath.

func MustCopyFile

func MustCopyFile(srcPath, dstPath string)

MustCopyFile copies the file from srcPath to dstPath.

func MustCreateFlockFile

func MustCreateFlockFile(dir string) *os.File

MustCreateFlockFile creates FlockFilename file in the directory dir and returns the handler to the file.

func MustFileSize

func MustFileSize(path string) uint64

MustFileSize returns file size for the given path.

func MustGetFreeSpace

func MustGetFreeSpace(path string) uint64

MustGetFreeSpace returns free space for the given directory path.

func MustHardLinkFiles

func MustHardLinkFiles(srcDir, dstDir string)

MustHardLinkFiles makes hard links for all the files from srcDir in dstDir.

func MustMkdirFailIfExist

func MustMkdirFailIfExist(path string)

MustMkdirFailIfExist creates the given path dir if it isn't exist.

If the directory at the given path already exists, then the function logs the error and exits.

func MustMkdirIfNotExist

func MustMkdirIfNotExist(path string)

MustMkdirIfNotExist creates the given path dir if it isn't exist.

func MustReadData

func MustReadData(r filestream.ReadCloser, data []byte)

MustReadData reads len(data) bytes from r.

func MustReadDir

func MustReadDir(dir string) []os.DirEntry

MustReadDir reads directory entries at the given dir.

func MustRemoveAll

func MustRemoveAll(path string)

MustRemoveAll removes path with all the contents.

It properly fsyncs the parent directory after path removal.

It properly handles NFS issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61 .

func MustRemoveDirAtomic

func MustRemoveDirAtomic(dir string)

MustRemoveDirAtomic removes the given dir atomically.

It uses the following algorithm:

1. Atomically rename the "<dir>" to "<dir>.must-remove.<XYZ>", where <XYZ> is an unique number.
2. Remove the "<dir>.must-remove.XYZ" in background.

If the process crashes after the step 1, then the directory must be removed on the next process start by calling MustRemoveTemporaryDirs on the parent directory.

func MustRemoveTemporaryDirs

func MustRemoveTemporaryDirs(dir string)

MustRemoveTemporaryDirs removes all the subdirectories with ".must-remove.<XYZ>" suffix.

Such directories may be left on unclean shutdown during MustRemoveDirAtomic call.

func MustStopDirRemover

func MustStopDirRemover()

MustStopDirRemover must be called in the end of graceful shutdown in order to wait for removing the remaining directories from removeDirConcurrencyCh.

It is expected that nobody calls MustRemoveAll when MustStopDirRemover is called.

func MustSymlinkRelative

func MustSymlinkRelative(srcPath, dstPath string)

MustSymlinkRelative creates relative symlink for srcPath in dstPath.

func MustSyncPath

func MustSyncPath(path string)

MustSyncPath syncs contents of the given path.

func MustWriteAtomic

func MustWriteAtomic(path string, data []byte, canOverwrite bool)

MustWriteAtomic atomically writes data to the given file path.

This function returns only after the file is fully written and synced to the underlying storage.

This function guarantees that the file at path either fully written or not written at all on app crash in the middle of the write.

If the file at path already exists, then the file is overwritten atomically if canOverwrite is true. Otherwise error is returned.

func MustWriteData

func MustWriteData(w filestream.WriteCloser, data []byte)

MustWriteData writes data to w.

func MustWriteSync

func MustWriteSync(path string, data []byte)

MustWriteSync writes data to the file at path and then calls fsync on the created file.

The fsync guarantees that the written data survives hardware reset after successful call.

This function may leave the file at the path in inconsistent state on app crash in the middle of the write. Use MustWriteAtomic if the file at the path must be either written in full or not written at all on app crash in the middle of the write.

func ReadFileOrHTTP

func ReadFileOrHTTP(path string) ([]byte, error)

ReadFileOrHTTP reads path either from local filesystem or from http if path starts with http or https.

func RemoveDirContents

func RemoveDirContents(dir string)

RemoveDirContents removes all the contents of the given dir if it exists.

It doesn't remove the dir itself, so the dir may be mounted to a separate partition.

Types

type MustReadAtCloser

type MustReadAtCloser interface {
	// MustReadAt must read len(p) bytes from offset off to p.
	MustReadAt(p []byte, off int64)

	// MustClose must close the reader.
	MustClose()
}

MustReadAtCloser is rand-access read interface.

type ReaderAt

type ReaderAt struct {
	// contains filtered or unexported fields
}

ReaderAt implements rand-access reader.

func MustOpenReaderAt

func MustOpenReaderAt(path string) *ReaderAt

MustOpenReaderAt opens ReaderAt for reading from filename.

MustClose must be called on the returned ReaderAt when it is no longer needed.

func (*ReaderAt) MustClose

func (r *ReaderAt) MustClose()

MustClose closes r.

func (*ReaderAt) MustFadviseSequentialRead

func (r *ReaderAt) MustFadviseSequentialRead(prefetch bool)

MustFadviseSequentialRead hints the OS that f is read mostly sequentially.

if prefetch is set, then the OS is hinted to prefetch f data.

func (*ReaderAt) MustReadAt

func (r *ReaderAt) MustReadAt(p []byte, off int64)

MustReadAt reads len(p) bytes at off from r.

func (*ReaderAt) SetUseLocalStats

func (r *ReaderAt) SetUseLocalStats()

SetUseLocalStats switches to local stats collection instead of global stats collection.

This function must be called before the first call to MustReadAt().

Collecting local stats may improve performance on systems with big number of CPU cores, since the locally collected stats is pushed to global stats only at MustClose() call instead of pushing it at every MustReadAt call.