fs

Documentation

Index

• func CopyDirectory(srcPath, dstPath string) error
• func CopyFile(srcPath, dstPath string) error
• func CreateFlockFile(dir string) (*os.File, error)
• func GetFilepath(baseDir, path string) string
• func HardLinkFiles(srcDir, dstDir string) error
• func IsDirOrSymlink(de os.DirEntry) bool
• func IsEmptyDir(path string) bool
• func IsPathExist(path string) bool
• func IsScheduledForRemoval(filename string) bool
• func IsTemporaryFileName(fn string) bool
• func MkdirAllFailIfExist(path string) error
• func MkdirAllIfNotExist(path string) error
• func MustClose(f *os.File)
• func MustFileSize(path string) uint64
• func MustGetFreeSpace(path string) uint64
• func MustRemoveAll(path string)
• func MustRemoveDirAtomic(dir string)
• func MustRemoveTemporaryDirs(dir string)
• func MustStopDirRemover()
• func MustSyncPath(path string)
• func MustWriteData(w io.Writer, data []byte)
• func ReadFileOrHTTP(path string) ([]byte, error)
• func ReadFullData(r io.Reader, data []byte) error
• func RemoveDirContents(dir string)
• func SymlinkRelative(srcPath, dstPath string) error
• func WriteFileAndSync(path string, data []byte) error
• func WriteFileAtomically(path string, data []byte, canOverwrite bool) error
• type MustReadAtCloser
• type ReaderAt
  • func MustOpenReaderAt(path string) *ReaderAt
  • func (r *ReaderAt) MustClose()
  • func (r *ReaderAt) MustFadviseSequentialRead(prefetch bool)
  • func (r *ReaderAt) MustReadAt(p []byte, off int64)
  • func (r *ReaderAt) SetUseLocalStats()

Functions

func CopyDirectory

func CopyDirectory(srcPath, dstPath string) error

CopyDirectory copies all the files in srcPath to dstPath.

func CopyFile

func CopyFile(srcPath, dstPath string) error

CopyFile copies the file from srcPath to dstPath.

func CreateFlockFile

func CreateFlockFile(dir string) (*os.File, error)

CreateFlockFile creates flock.lock file in the directory dir and returns the handler to the file.

func GetFilepath

func GetFilepath(baseDir, path string) string

GetFilepath returns full path to file for the given baseDir and path.

func HardLinkFiles

func HardLinkFiles(srcDir, dstDir string) error

HardLinkFiles makes hard links for all the files from srcDir in dstDir.

func IsDirOrSymlink

func IsDirOrSymlink(de os.DirEntry) bool

IsDirOrSymlink returns true if de is directory or symlink.

func IsEmptyDir

func IsEmptyDir(path string) bool

IsEmptyDir returns true if path points to empty directory.

func IsPathExist

func IsPathExist(path string) bool

IsPathExist returns whether the given path exists.

func IsScheduledForRemoval

func IsScheduledForRemoval(filename string) bool

IsScheduledForRemoval returns true if the filename contains .must-remove. substring

func IsTemporaryFileName

func IsTemporaryFileName(fn string) bool

IsTemporaryFileName returns true if fn matches temporary file name pattern from WriteFileAtomically.

func MkdirAllFailIfExist

func MkdirAllFailIfExist(path string) error

MkdirAllFailIfExist creates the given path dir if it isn't exist.

Returns error if path already exists.

func MkdirAllIfNotExist

func MkdirAllIfNotExist(path string) error

MkdirAllIfNotExist creates the given path dir if it isn't exist.

func MustClose

func MustClose(f *os.File)

MustClose must close the given file f.

func MustFileSize

func MustFileSize(path string) uint64

MustFileSize returns file size for the given path.

func MustGetFreeSpace

func MustGetFreeSpace(path string) uint64

MustGetFreeSpace returns free space for the given directory path.

func MustRemoveAll

func MustRemoveAll(path string)

MustRemoveAll removes path with all the contents.

It properly fsyncs the parent directory after path removal.

It properly handles NFS issue https://github.com/VictoriaMetrics/VictoriaMetrics/issues/61 .

func MustRemoveDirAtomic

func MustRemoveDirAtomic(dir string)

MustRemoveDirAtomic removes the given dir atomically.

It uses the following algorithm:

1. Atomically rename the "<dir>" to "<dir>.must-remove.<XYZ>", where <XYZ> is an unique number.
2. Remove the "<dir>.must-remove.XYZ" in background.

If the process crashes after the step 1, then the directory must be removed on the next process start by calling MustRemoveTemporaryDirs on the parent directory.

func MustRemoveTemporaryDirs

func MustRemoveTemporaryDirs(dir string)

MustRemoveTemporaryDirs removes all the subdirectories with ".must-remove.<XYZ>" suffix.

Such directories may be left on unclean shutdown during MustRemoveDirAtomic call.

func MustStopDirRemover

func MustStopDirRemover()

MustStopDirRemover must be called in the end of graceful shutdown in order to wait for removing the remaining directories from removeDirConcurrencyCh.

It is expected that nobody calls MustRemoveAll when MustStopDirRemover is called.

func MustSyncPath

func MustSyncPath(path string)

MustSyncPath syncs contents of the given path.

func MustWriteData

func MustWriteData(w io.Writer, data []byte)

MustWriteData writes data to w.

func ReadFileOrHTTP

func ReadFileOrHTTP(path string) ([]byte, error)

ReadFileOrHTTP reads path either from local filesystem or from http if path starts with http or https.

func ReadFullData

func ReadFullData(r io.Reader, data []byte) error

ReadFullData reads len(data) bytes from r.

func RemoveDirContents

func RemoveDirContents(dir string)

RemoveDirContents removes all the contents of the given dir if it exists.

It doesn't remove the dir itself, so the dir may be mounted to a separate partition.

func SymlinkRelative

func SymlinkRelative(srcPath, dstPath string) error

SymlinkRelative creates relative symlink for srcPath in dstPath.

func WriteFileAndSync

func WriteFileAndSync(path string, data []byte) error

WriteFileAndSync writes data to the file at path and then calls fsync on the created file.

The fsync guarantees that the written data survives hardware reset after successful call.

This function may leave the file at the path in inconsistent state on app crash in the middle of the write. Use WriteFileAtomically if the file at the path must be either written in full or not written at all on app crash in the middle of the write.

func WriteFileAtomically

func WriteFileAtomically(path string, data []byte, canOverwrite bool) error

WriteFileAtomically atomically writes data to the given file path.

This function returns only after the file is fully written and synced to the underlying storage.

This function guarantees that the file at path either fully written or not written at all on app crash in the middle of the write.

If the file at path already exists, then the file is overwritten atomically if canOverwrite is true. Otherwise error is returned.

Types

type MustReadAtCloser

type MustReadAtCloser interface {
	// MustReadAt must read len(p) bytes from offset off to p.
	MustReadAt(p []byte, off int64)

	// MustClose must close the reader.
	MustClose()
}

MustReadAtCloser is rand-access read interface.

type ReaderAt

type ReaderAt struct {
	// contains filtered or unexported fields
}

ReaderAt implements rand-access reader.

func MustOpenReaderAt

func MustOpenReaderAt(path string) *ReaderAt

MustOpenReaderAt opens ReaderAt for reading from filename.

MustClose must be called on the returned ReaderAt when it is no longer needed.

func (*ReaderAt) MustClose

func (r *ReaderAt) MustClose()

MustClose closes r.

func (*ReaderAt) MustFadviseSequentialRead

func (r *ReaderAt) MustFadviseSequentialRead(prefetch bool)

MustFadviseSequentialRead hints the OS that f is read mostly sequentially.

if prefetch is set, then the OS is hinted to prefetch f data.

func (*ReaderAt) MustReadAt

func (r *ReaderAt) MustReadAt(p []byte, off int64)

MustReadAt reads len(p) bytes at off from r.

func (*ReaderAt) SetUseLocalStats

func (r *ReaderAt) SetUseLocalStats()

SetUseLocalStats switches to local stats collection instead of global stats collection.

This function must be called before the first call to MustReadAt().

Collecting local stats may improve performance on systems with big number of CPU cores, since the locally collected stats is pushed to global stats only at MustClose() call instead of pushing it at every MustReadAt call.