fstools

Documentation

Overview

Package fstools provides access to low level syscalls for advanced filesystem functionality.

Index

• Constants
• func FiemapExtentFlagsToStrings(flags uint32) []string
• func FiemapWalk(file *os.File, flags uint32, callback FiemapWalkCallback) error
• func FileDedupeRangeFull(srcFd int, value *unix.FileDedupeRange, progress FileDedupeRangeFullProgress) error
• func FileDedupeRangeStatusToString(status int32) string
• func FileFragDumpExtents(filePath string, syncFirst bool, useBytes bool, faster bool) error
• func IoctlFiemap(fd int, value *Fiemap) error
• type Fiemap
• type FiemapExtent
• type FiemapWalkCallback
• type FileDedupeRangeFullProgress

Constants

const (
	FIEMAP_MAX_OFFSET = math.MaxUint64

	FIEMAP_FLAG_SYNC  = 0x00000001 // sync file data before map
	FIEMAP_FLAG_XATTR = 0x00000002 // map extended attribute tree
	FIEMAP_FLAG_CACHE = 0x00000004 // request caching of the extents

	FIEMAP_FLAGS_COMPAT = (FIEMAP_FLAG_SYNC | FIEMAP_FLAG_XATTR)

	FIEMAP_EXTENT_LAST           = 0x00000001 // Last extent in file.
	FIEMAP_EXTENT_UNKNOWN        = 0x00000002 // Data location unknown.
	FIEMAP_EXTENT_DELALLOC       = 0x00000004 // Location still pending. Sets EXTENT_UNKNOWN.
	FIEMAP_EXTENT_ENCODED        = 0x00000008 // Data can not be read while fs is unmounted
	FIEMAP_EXTENT_DATA_ENCRYPTED = 0x00000080 // Data is encrypted by fs. Sets EXTENT_NO_BYPASS.
	FIEMAP_EXTENT_NOT_ALIGNED    = 0x00000100 // Extent offsets may not be block aligned.
	FIEMAP_EXTENT_DATA_INLINE    = 0x00000200 // Data mixed with metadata. Sets EXTENT_NOT_ALIGNED.
	FIEMAP_EXTENT_DATA_TAIL      = 0x00000400 // Multiple files in block. Sets EXTENT_NOT_ALIGNED.
	FIEMAP_EXTENT_UNWRITTEN      = 0x00000800 // Space allocated, but no data (i.e. zero).
	FIEMAP_EXTENT_MERGED         = 0x00001000 // File does not natively support extents. Result merged for efficiency.
	FIEMAP_EXTENT_SHARED         = 0x00002000 // Space shared with other files.
)

All constacts from uapi/linux/fiemap.h. https://github.com/torvalds/linux/blob/master/include/uapi/linux/fiemap.h

const (
	SizeofRawFiemap       = 32
	SizeofRawFiemapExtent = 56
)

Constants needed to calculate total ioctl request size. The filefrag command uses these to ensure that it only sends a 2048*8 (aligned to 64bit) sized buffer.

const (
	FS_IOC_FIEMAP = 0xC020660B
)

Functions

func FiemapExtentFlagsToStrings

func FiemapExtentFlagsToStrings(flags uint32) []string

FiemapExtentFlagsToStrings converts FIEMAP extent flags to human-readable strings.

func FiemapWalk

func FiemapWalk(file *os.File, flags uint32, callback FiemapWalkCallback) error

FiemapWalk iterates over all extents that back the given file, calling the provided callback for each extent.

The flags value can 0 as the defualt, otherwise, you can set it to the bitwise or of FIEMAP_FLAG_SYNC, FIEMAP_FLAG_XATTR, or FIEMAP_FLAG_CACHE.

func FileDedupeRangeFull

func FileDedupeRangeFull(
	srcFd int,
	value *unix.FileDedupeRange,
	progress FileDedupeRangeFullProgress,
) error

FileDedupeRangeFull is a wrapper around IoctlFileDedupeRange that is able to fulfill deduping full file lengths and is resilient to destination file dedupe failures.

The main IoctlFileDedupeRange directly calls the FIDEDUPERANGE ioctl, which only dedupes up to 1GiB using btrfs and possibly less on other filesystem. This wrapper will iterate and continue to deduplicate the full source length specified in value.Src_length. Given that this can take a very long time for large files, the progress callback is provided to show updates.

func FileDedupeRangeStatusToString

func FileDedupeRangeStatusToString(status int32) string

FileDedupeRangeStatusToString converts a FileDedupeRangeInfo.Status to a human-readable string.

func FileFragDumpExtents

func FileFragDumpExtents(filePath string, syncFirst bool, useBytes bool, faster bool) error

FileFragDumpExtents prints all extents that compose the given filePath. This is very similar to using the "filefrag -v <path>" command.

All options should be false, by default. If syncFirst is enabled, the requested file will be synced to disk before reading the extents. If useBytes is enabled, the units will by in Bytes, which is the default unit received by the FIEMAP ioctl. If faster is enabled, the pretty printing functionality will be disabled.

See https://docs.kernel.org/filesystems/fiemap.html, https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/tree/misc/filefrag.c, and https://github.com/torvalds/linux/blob/master/include/uapi/linux/fiemap.h for more information.

func IoctlFiemap

func IoctlFiemap(fd int, value *Fiemap) error

IoctlFiemap performs an FIEMAP ioctl operation on a given fd.

We choose to use the value.Extents field as purley as the output array to allow reuse on the calller side.

Types

type Fiemap

type Fiemap struct {
	Start          uint64 // in
	Length         uint64 // in
	Flags          uint32 // in/out
	Mapped_extents uint32 // out
	Reserved       uint32
	Extents        []FiemapExtent // out
}

type FiemapExtent

type FiemapExtent struct {
	Logical    uint64
	Physical   uint64
	Length     uint64
	Reserved64 [2]uint64
	Flags      uint32
	Reserved   [3]uint32
}

type FiemapWalkCallback

type FiemapWalkCallback func(index int, extent *FiemapExtent) (finished bool)

FiemapWalkCallback defines the callback signature for FiemapWalk.

type FileDedupeRangeFullProgress

type FileDedupeRangeFullProgress func(bytesDeduped, bytesLength uint64, exit bool)