disklayout

package
v0.0.0-...-522126a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 23, 2019 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package disklayout provides Linux ext file system's disk level structures which can be directly read into from the underlying device. Structs aim to emulate structures `exactly` how they are layed out on disk.

This library aims to be compatible with all ext(2/3/4) systems so it provides a generic interface for all major structures and various implementations (for different versions). The user code is responsible for using appropriate implementations based on the underlying device.

Interfacing all major structures here serves a few purposes:

  • Abstracts away the complexity of the underlying structure from client code. The client only has to figure out versioning on set up and then can use these as black boxes and pass it higher up the stack.
  • Having pointer receivers forces the user to use pointers to these heavy structs. Hence, prevents the client code from unintentionally copying these by value while passing the interface around.
  • Version-based implementation selection is resolved on set up hence avoiding per call overhead of choosing implementation.
  • All interface methods are pretty light weight (do not take in any parameters by design). Passing pointer arguments to interface methods can lead to heap allocation as the compiler won't be able to perform escape analysis on an unknown implementation at compile time.

Notes:

  • All fields in these structs are exported because binary.Read would panic otherwise.
  • All structures on disk are in little-endian order. Only jbd2 (journal) structures are in big-endian order.
  • All OS dependent fields in these structures will be interpretted using the Linux version of that field.
  • The suffix `Lo` in field names stands for lower bits of that field.
  • The suffix `Hi` in field names stands for upper bits of that field.
  • The suffix `Raw` has been added to indicate that the field is not split into Lo and Hi fields and also to resolve name collision with the respective interface.

Index

Constants

View Source 
const (
	// BgInodeUninit indicates that inode table and bitmap are not initialized.
	BgInodeUninit uint16 = 0x1

	// BgBlockUninit indicates that block bitmap is not initialized.
	BgBlockUninit uint16 = 0x2

	// BgInodeZeroed indicates that inode table is zeroed.
	BgInodeZeroed uint16 = 0x4
)

These are the different block group flags.

View Source 
const (
	// MaxFileName is the maximum length of an ext fs file's name.
	MaxFileName = 255

	// DirentSize is the size of ext dirent structures.
	DirentSize = 263
)
View Source 
const (
	// ExtentStructsSize is the size of all the three extent on-disk structs.
	ExtentStructsSize = 12

	// ExtentMagic is the magic number which must be present in the header.
	ExtentMagic = 0xf30a
)
View Source 
const (
	// InSync indicates that all writes to the file must be synchronous.
	InSync = 0x8

	// InImmutable indicates that this file is immutable.
	InImmutable = 0x10

	// InAppend indicates that this file can only be appended to.
	InAppend = 0x20

	// InNoDump indicates that teh dump(1) utility should not dump this file.
	InNoDump = 0x40

	// InNoAccessTime indicates that the access time of this inode must not be
	// updated.
	InNoAccessTime = 0x80

	// InIndex indicates that this directory has hashed indexes.
	InIndex = 0x1000

	// InJournalData indicates that file data must always be written through a
	// journal device.
	InJournalData = 0x4000

	// InDirSync indicates that all the directory entiry data must be written
	// synchronously.
	InDirSync = 0x10000

	// InTopDir indicates that this inode is at the top of the directory hierarchy.
	InTopDir = 0x20000

	// InHugeFile indicates that this is a huge file.
	InHugeFile = 0x40000

	// InExtents indicates that this inode uses extents.
	InExtents = 0x80000

	// InExtendedAttr indicates that this inode stores a large extended attribute
	// value in its data blocks.
	InExtendedAttr = 0x200000

	// InInline indicates that this inode has inline data.
	InInline = 0x10000000

	// InReserved indicates that this inode is reserved for the ext4 library.
	InReserved = 0x80000000
)

Inode flags. This is not comprehensive and flags which were not used in the Linux kernel have been excluded.

View Source 
const (
	InUserReadFlagMask  = 0x4BDFFF
	InUserWriteFlagMask = 0x4B80FF
)

These masks define how users can view/modify inode flags. The rest of the flags are for internal kernel usage only.

View Source 
const (
	// SbDirPrealloc indicates directory preallocation.
	SbDirPrealloc = 0x1

	// SbHasJournal indicates the presence of a journal. jbd2 should only work
	// with this being set.
	SbHasJournal = 0x4

	// SbExtAttr indicates extended attributes support.
	SbExtAttr = 0x8

	// SbResizeInode indicates that the fs has reserved GDT blocks (right after
	// group descriptors) for fs expansion.
	SbResizeInode = 0x10

	// SbDirIndex indicates that the fs has directory indices.
	SbDirIndex = 0x20

	// SbSparseV2 stands for Sparse superblock version 2.
	SbSparseV2 = 0x200
)

Superblock compatible features. This is not exhaustive, unused features are not listed.

View Source 
const (
	// SbDirentFileType indicates that directory entries record the file type.
	// We should use struct DirentNew for dirents then.
	SbDirentFileType = 0x2

	// SbRecovery indicates that the filesystem needs recovery.
	SbRecovery = 0x4

	// SbJournalDev indicates that the filesystem has a separate journal device.
	SbJournalDev = 0x8

	// SbMetaBG indicates that the filesystem is using Meta block groups. Moves
	// the group descriptors from the congested first block group into the first
	// group of each metablock group to increase the maximum block groups limit
	// and hence support much larger filesystems.
	//
	// See https://www.kernel.org/doc/html/latest/filesystems/ext4/overview.html#meta-block-groups.
	SbMetaBG = 0x10

	// SbExtents indicates that the filesystem uses extents. Must be set in ext4
	// filesystems.
	SbExtents = 0x40

	// SbIs64Bit indicates that this filesystem addresses blocks with 64-bits.
	// Hence can support 2^64 data blocks.
	SbIs64Bit = 0x80

	// SbMMP indicates that this filesystem has multiple mount protection.
	//
	// See https://www.kernel.org/doc/html/latest/filesystems/ext4/globals.html#multiple-mount-protection.
	SbMMP = 0x100

	// SbFlexBg indicates that this filesystem has flexible block groups. Several
	// block groups are tied into one logical block group so that all the metadata
	// for the block groups (bitmaps and inode tables) are close together for
	// faster loading. Consequently, large files will be continuous on disk.
	// However, this does not affect the placement of redundant superblocks and
	// group descriptors.
	//
	// See https://www.kernel.org/doc/html/latest/filesystems/ext4/overview.html#flexible-block-groups.
	SbFlexBg = 0x200

	// SbLargeDir shows that large directory enabled. Directory htree can be 3
	// levels deep. Directory htrees are allowed to be 2 levels deep otherwise.
	SbLargeDir = 0x4000

	// SbInlineData allows inline data in inodes for really small files.
	SbInlineData = 0x8000

	// SbEncrypted indicates that this fs contains encrypted inodes.
	SbEncrypted = 0x10000
)

Superblock incompatible features. This is not exhaustive, unused features are not listed.

View Source 
const (
	// SbSparse indicates sparse superblocks. Only groups with number either 0 or
	// a power of 3, 5, or 7 will have redundant copies of the superblock and
	// block descriptors.
	SbSparse = 0x1

	// SbLargeFile indicates that this fs has been used to store a file >= 2GiB.
	SbLargeFile = 0x2

	// SbHugeFile indicates that this fs contains files whose sizes are
	// represented in units of logicals blocks, not 512-byte sectors.
	SbHugeFile = 0x8

	// SbGdtCsum indicates that group descriptors have checksums.
	SbGdtCsum = 0x10

	// SbDirNlink indicates that the new subdirectory limit is 64,999. Ext3 has a
	// 32,000 subdirectory limit.
	SbDirNlink = 0x20

	// SbExtraIsize indicates that large inodes exist on this filesystem.
	SbExtraIsize = 0x40

	// SbHasSnapshot indicates the existence of a snapshot.
	SbHasSnapshot = 0x80

	// SbQuota enables usage tracking for all quota types.
	SbQuota = 0x100

	// SbBigalloc maps to the bigalloc feature. When set, the minimum allocation
	// unit becomes a cluster rather than a data block. Then block bitmaps track
	// clusters, not data blocks.
	//
	// See https://www.kernel.org/doc/html/latest/filesystems/ext4/overview.html#bigalloc.
	SbBigalloc = 0x200

	// SbMetadataCsum indicates that the fs supports metadata checksumming.
	SbMetadataCsum = 0x400

	// SbReadOnly marks this filesystem as readonly. Should refuse to mount in
	// read/write mode.
	SbReadOnly = 0x1000
)

Superblock readonly compatible features. This is not exhaustive, unused features are not listed.

View Source 
const (
	// OldInodeSize is the inode size in ext2/ext3.
	OldInodeSize = 128
)
View Source 
const (
	// RootDirInode is the inode number of the root directory inode.
	RootDirInode = 2
)

Special inodes. See https://www.kernel.org/doc/html/latest/filesystems/ext4/overview.html#special-inodes.

View Source 
const (
	// SbOffset is the absolute offset at which the superblock is placed.
	SbOffset = 1024
)

Variables

This section is empty.

Functions

This section is empty.

Types

type BGFlags 

type BGFlags struct {
	InodeUninit bool
	BlockUninit bool
	InodeZeroed bool
}

BGFlags represents all the different combinations of block group flags.

func BGFlagsFromInt 

func BGFlagsFromInt(flags uint16) BGFlags

BGFlagsFromInt converts the 16-bit flag representation to a BGFlags struct.

func (BGFlags) ToInt 

func (f BGFlags) ToInt() uint16

ToInt converts a BGFlags struct back to its 16-bit representation.

type BlockGroup 

type BlockGroup interface {
	// InodeTable returns the absolute block number of the block containing the
	// inode table. This points to an array of Inode structs. Inode tables are
	// statically allocated at mkfs time. The superblock records the number of
	// inodes per group (length of this table) and the size of each inode struct.
	InodeTable() uint64

	// BlockBitmap returns the absolute block number of the block containing the
	// block bitmap. This bitmap tracks the usage of data blocks within this block
	// group and has its own checksum.
	BlockBitmap() uint64

	// InodeBitmap returns the absolute block number of the block containing the
	// inode bitmap. This bitmap tracks the usage of this group's inode table
	// entries and has its own checksum.
	InodeBitmap() uint64

	// ExclusionBitmap returns the absolute block number of the snapshot exclusion
	// bitmap.
	ExclusionBitmap() uint64

	// FreeBlocksCount returns the number of free blocks in the group.
	FreeBlocksCount() uint32

	// FreeInodesCount returns the number of free inodes in the group.
	FreeInodesCount() uint32

	// DirectoryCount returns the number of inodes that represent directories
	// under this block group.
	DirectoryCount() uint32

	// UnusedInodeCount returns the number of unused inodes beyond the last used
	// inode in this group's inode table. As a result, we needn’t scan past the
	// (InodesPerGroup - UnusedInodeCount())th entry in the inode table.
	UnusedInodeCount() uint32

	// BlockBitmapChecksum returns the block bitmap checksum. This is calculated
	// using crc32c(FS UUID + group number + entire bitmap).
	BlockBitmapChecksum() uint32

	// InodeBitmapChecksum returns the inode bitmap checksum. This is calculated
	// using crc32c(FS UUID + group number + entire bitmap).
	InodeBitmapChecksum() uint32

	// Checksum returns this block group's checksum.
	//
	// If SbMetadataCsum feature is set:
	//     - checksum is crc32c(FS UUID + group number + group descriptor
	//       structure) & 0xFFFF.
	//
	// If SbGdtCsum feature is set:
	//     - checksum is crc16(FS UUID + group number + group descriptor
	//       structure).
	//
	// SbMetadataCsum and SbGdtCsum should not be both set.
	// If they are, Linux warns and asks to run fsck.
	Checksum() uint16

	// Flags returns BGFlags which represents the block group flags.
	Flags() BGFlags
}

BlockGroup represents a Linux ext block group descriptor. An ext file system is split into a series of block groups. This provides an access layer to information needed to access and use a block group.

Location:

  • The block group descriptor table is always placed in the blocks immediately after the block containing the superblock.
  • The 1st block group descriptor in the original table is in the (sb.FirstDataBlock() + 1)th block.
  • See SuperBlock docs to see where the block group descriptor table is replicated.
  • sb.BgDescSize() must be used as the block group descriptor entry size while reading the table from disk.

See https://www.kernel.org/doc/html/latest/filesystems/ext4/globals.html#block-group-descriptors.

type BlockGroup32Bit 

type BlockGroup32Bit struct {
	BlockBitmapLo         uint32
	InodeBitmapLo         uint32
	InodeTableLo          uint32
	FreeBlocksCountLo     uint16
	FreeInodesCountLo     uint16
	UsedDirsCountLo       uint16
	FlagsRaw              uint16
	ExcludeBitmapLo       uint32
	BlockBitmapChecksumLo uint16
	InodeBitmapChecksumLo uint16
	ItableUnusedLo        uint16
	ChecksumRaw           uint16
}

BlockGroup32Bit emulates the first half of struct ext4_group_desc in fs/ext4/ext4.h. It is the block group descriptor struct for ext2, ext3 and 32-bit ext4 filesystems. It implements BlockGroup interface.

func (*BlockGroup32Bit) BlockBitmap 

func (bg *BlockGroup32Bit) BlockBitmap() uint64

BlockBitmap implements BlockGroup.BlockBitmap.

func (*BlockGroup32Bit) BlockBitmapChecksum 

func (bg *BlockGroup32Bit) BlockBitmapChecksum() uint32

BlockBitmapChecksum implements BlockGroup.BlockBitmapChecksum.

func (*BlockGroup32Bit) Checksum 

func (bg *BlockGroup32Bit) Checksum() uint16

Checksum implements BlockGroup.Checksum.

func (*BlockGroup32Bit) DirectoryCount 

func (bg *BlockGroup32Bit) DirectoryCount() uint32

DirectoryCount implements BlockGroup.DirectoryCount.

func (*BlockGroup32Bit) ExclusionBitmap 

func (bg *BlockGroup32Bit) ExclusionBitmap() uint64

ExclusionBitmap implements BlockGroup.ExclusionBitmap.

func (*BlockGroup32Bit) Flags 

func (bg *BlockGroup32Bit) Flags() BGFlags

Flags implements BlockGroup.Flags.

func (*BlockGroup32Bit) FreeBlocksCount 

func (bg *BlockGroup32Bit) FreeBlocksCount() uint32

FreeBlocksCount implements BlockGroup.FreeBlocksCount.

func (*BlockGroup32Bit) FreeInodesCount 

func (bg *BlockGroup32Bit) FreeInodesCount() uint32

FreeInodesCount implements BlockGroup.FreeInodesCount.

func (*BlockGroup32Bit) InodeBitmap 

func (bg *BlockGroup32Bit) InodeBitmap() uint64

InodeBitmap implements BlockGroup.InodeBitmap.

func (*BlockGroup32Bit) InodeBitmapChecksum 

func (bg *BlockGroup32Bit) InodeBitmapChecksum() uint32

InodeBitmapChecksum implements BlockGroup.InodeBitmapChecksum.

func (*BlockGroup32Bit) InodeTable 

func (bg *BlockGroup32Bit) InodeTable() uint64

InodeTable implements BlockGroup.InodeTable.

func (*BlockGroup32Bit) UnusedInodeCount 

func (bg *BlockGroup32Bit) UnusedInodeCount() uint32

UnusedInodeCount implements BlockGroup.UnusedInodeCount.

type BlockGroup64Bit 

type BlockGroup64Bit struct {
	// We embed the 32-bit struct here because 64-bit version is just an extension
	// of the 32-bit version.
	BlockGroup32Bit

	// 64-bit specific fields.
	BlockBitmapHi         uint32
	InodeBitmapHi         uint32
	InodeTableHi          uint32
	FreeBlocksCountHi     uint16
	FreeInodesCountHi     uint16
	UsedDirsCountHi       uint16
	ItableUnusedHi        uint16
	ExcludeBitmapHi       uint32
	BlockBitmapChecksumHi uint16
	InodeBitmapChecksumHi uint16
	// contains filtered or unexported fields
}

BlockGroup64Bit emulates struct ext4_group_desc in fs/ext4/ext4.h. It is the block group descriptor struct for 64-bit ext4 filesystems. It implements BlockGroup interface. It is an extension of the 32-bit version of BlockGroup.

func (*BlockGroup64Bit) BlockBitmap 

func (bg *BlockGroup64Bit) BlockBitmap() uint64

BlockBitmap implements BlockGroup.BlockBitmap.

func (*BlockGroup64Bit) BlockBitmapChecksum 

func (bg *BlockGroup64Bit) BlockBitmapChecksum() uint32

BlockBitmapChecksum implements BlockGroup.BlockBitmapChecksum.

func (*BlockGroup64Bit) DirectoryCount 

func (bg *BlockGroup64Bit) DirectoryCount() uint32

DirectoryCount implements BlockGroup.DirectoryCount.

func (*BlockGroup64Bit) ExclusionBitmap 

func (bg *BlockGroup64Bit) ExclusionBitmap() uint64

ExclusionBitmap implements BlockGroup.ExclusionBitmap.

func (*BlockGroup64Bit) FreeBlocksCount 

func (bg *BlockGroup64Bit) FreeBlocksCount() uint32

FreeBlocksCount implements BlockGroup.FreeBlocksCount.

func (*BlockGroup64Bit) FreeInodesCount 

func (bg *BlockGroup64Bit) FreeInodesCount() uint32

FreeInodesCount implements BlockGroup.FreeInodesCount.

func (*BlockGroup64Bit) InodeBitmap 

func (bg *BlockGroup64Bit) InodeBitmap() uint64

InodeBitmap implements BlockGroup.InodeBitmap.

func (*BlockGroup64Bit) InodeBitmapChecksum 

func (bg *BlockGroup64Bit) InodeBitmapChecksum() uint32

InodeBitmapChecksum implements BlockGroup.InodeBitmapChecksum.

func (*BlockGroup64Bit) InodeTable 

func (bg *BlockGroup64Bit) InodeTable() uint64

InodeTable implements BlockGroup.InodeTable.

func (*BlockGroup64Bit) UnusedInodeCount 

func (bg *BlockGroup64Bit) UnusedInodeCount() uint32

UnusedInodeCount implements BlockGroup.UnusedInodeCount.

type CompatFeatures 

type CompatFeatures struct {
	DirPrealloc bool
	HasJournal  bool
	ExtAttr     bool
	ResizeInode bool
	DirIndex    bool
	SparseV2    bool
}

CompatFeatures represents a superblock's compatible feature set. If the kernel does not understand any of these feature, it can still read/write to this fs.

func CompatFeaturesFromInt 

func CompatFeaturesFromInt(f uint32) CompatFeatures

CompatFeaturesFromInt converts the integer representation of superblock compatible features to CompatFeatures struct.

func (CompatFeatures) ToInt 

func (f CompatFeatures) ToInt() uint32

ToInt converts superblock compatible features back to its 32-bit rep.

type Dirent 

type Dirent interface {
	// Inode returns the absolute inode number of the underlying inode.
	// Inode number 0 signifies an unused dirent.
	Inode() uint32

	// RecordSize returns the record length of this dirent on disk. The next
	// dirent in the dirent list should be read after these many bytes from
	// the current dirent. Must be a multiple of 4.
	RecordSize() uint16

	// FileName returns the name of the file. Can be at most 255 is length.
	FileName() string

	// FileType returns the inode type of the underlying inode. This is a
	// performance hack so that we do not have to read the underlying inode struct
	// to know the type of inode. This will only work when the SbDirentFileType
	// feature is set. If not, the second returned value will be false indicating
	// that user code has to use the inode mode to extract the file type.
	FileType() (fs.InodeType, bool)
}

The Dirent interface should be implemented by structs representing ext directory entries. These are for the linear classical directories which just store a list of dirent structs. A directory is a series of data blocks where is each data block contains a linear array of dirents. The last entry of the block has a record size that takes it to the end of the block. The end of the directory is when you read dirInode.Size() bytes from the blocks.

See https://www.kernel.org/doc/html/latest/filesystems/ext4/dynamic.html#linear-classic-directories.

type DirentNew 

type DirentNew struct {
	InodeNumber  uint32
	RecordLength uint16
	NameLength   uint8
	FileTypeRaw  uint8
	FileNameRaw  [MaxFileName]byte
}

DirentNew represents the ext4 directory entry struct. This emulates Linux's ext4_dir_entry_2 struct. The FileName can not be more than 255 bytes so we only need 8 bits to store the NameLength. As a result, NameLength has been shortened and the other 8 bits are used to encode the file type. Use the FileTypeRaw field only if the SbDirentFileType feature is set.

Note: This struct can be of variable size on disk. The one described below is of maximum size and the FileName beyond NameLength bytes might contain garbage.

func (*DirentNew) FileName 

func (d *DirentNew) FileName() string

FileName implements Dirent.FileName.

func (*DirentNew) FileType 

func (d *DirentNew) FileType() (fs.InodeType, bool)

FileType implements Dirent.FileType.

func (*DirentNew) Inode 

func (d *DirentNew) Inode() uint32

Inode implements Dirent.Inode.

func (*DirentNew) RecordSize 

func (d *DirentNew) RecordSize() uint16

RecordSize implements Dirent.RecordSize.

type DirentOld 

type DirentOld struct {
	InodeNumber  uint32
	RecordLength uint16
	NameLength   uint16
	FileNameRaw  [MaxFileName]byte
}

DirentOld represents the old directory entry struct which does not contain the file type. This emulates Linux's ext4_dir_entry struct.

Note: This struct can be of variable size on disk. The one described below is of maximum size and the FileName beyond NameLength bytes might contain garbage.

func (*DirentOld) FileName 

func (d *DirentOld) FileName() string

FileName implements Dirent.FileName.

func (*DirentOld) FileType 

func (d *DirentOld) FileType() (fs.InodeType, bool)

FileType implements Dirent.FileType.

func (*DirentOld) Inode 

func (d *DirentOld) Inode() uint32

Inode implements Dirent.Inode.

func (*DirentOld) RecordSize 

func (d *DirentOld) RecordSize() uint16

RecordSize implements Dirent.RecordSize.

type Extent 

type Extent struct {
	FirstFileBlock uint32
	Length         uint16
	StartBlockHi   uint16
	StartBlockLo   uint32
}

Extent represents the ext4_extent struct in ext4. Only present in leaf nodes. Sorted in ascending order based on FirstFileBlock since Linux does a binary search on this. This points to an array of data blocks containing the file data. It covers `Length` data blocks starting from `StartBlock`.

func (*Extent) FileBlock 

func (e *Extent) FileBlock() uint32

FileBlock implements ExtentEntry.FileBlock.

func (*Extent) PhysicalBlock 

func (e *Extent) PhysicalBlock() uint64

PhysicalBlock implements ExtentEntry.PhysicalBlock. It returns the physical block number of the first data block this extent covers.

type ExtentEntry 

type ExtentEntry interface {
	// FileBlock returns the first file block number covered by this entry.
	FileBlock() uint32

	// PhysicalBlock returns the child physical block that this entry points to.
	PhysicalBlock() uint64
}

ExtentEntry reprsents an extent tree node entry. The entry can either be an ExtentIdx or Extent itself. This exists to simplify navigation logic.

type ExtentEntryPair 

type ExtentEntryPair struct {
	// Entry points to the child node on disk.
	Entry ExtentEntry
	// Node points to child node in memory. Is nil if the current node is a leaf.
	Node *ExtentNode
}

ExtentEntryPair couples an in-memory ExtendNode with the ExtentEntry that points to it. We want to cache these structs in memory to avoid repeated disk reads.

Note: This struct itself does not represent an on-disk struct.

type ExtentHeader 

type ExtentHeader struct {
	// Magic in the extent magic number, must be 0xf30a.
	Magic uint16

	// NumEntries indicates the number of valid entries following the header.
	NumEntries uint16

	// MaxEntries that could follow the header. Used while adding entries.
	MaxEntries uint16

	// Height represents the distance of this node from the farthest leaf. Please
	// note that Linux incorrectly calls this `Depth` (which means the distance
	// of the node from the root).
	Height uint16
	// contains filtered or unexported fields
}

ExtentHeader emulates the ext4_extent_header struct in ext4. Each extent tree node begins with this and is followed by `NumEntries` number of:

  • Extent if `Depth` == 0
  • ExtentIdx otherwise

type ExtentIdx 

type ExtentIdx struct {
	FirstFileBlock uint32
	ChildBlockLo   uint32
	ChildBlockHi   uint16
	// contains filtered or unexported fields
}

ExtentIdx emulates the ext4_extent_idx struct in ext4. Only present in internal nodes. Sorted in ascending order based on FirstFileBlock since Linux does a binary search on this. This points to a block containing the child node.

func (*ExtentIdx) FileBlock 

func (ei *ExtentIdx) FileBlock() uint32

FileBlock implements ExtentEntry.FileBlock.

func (*ExtentIdx) PhysicalBlock 

func (ei *ExtentIdx) PhysicalBlock() uint64

PhysicalBlock implements ExtentEntry.PhysicalBlock. It returns the physical block number of the child block.

type ExtentNode 

type ExtentNode struct {
	Header  ExtentHeader
	Entries []ExtentEntryPair
}

ExtentNode represents an extent tree node. For internal nodes, all Entries will be ExtendIdxs. For leaf nodes, they will all be Extents.

Note: This struct itself does not represent an on-disk struct.

type IncompatFeatures 

type IncompatFeatures struct {
	DirentFileType bool
	Recovery       bool
	JournalDev     bool
	MetaBG         bool
	Extents        bool
	Is64Bit        bool
	MMP            bool
	FlexBg         bool
	LargeDir       bool
	InlineData     bool
	Encrypted      bool
}

IncompatFeatures represents a superblock's incompatible feature set. If the kernel does not understand any of these feature, it should refuse to mount.

func IncompatFeaturesFromInt 

func IncompatFeaturesFromInt(f uint32) IncompatFeatures

IncompatFeaturesFromInt converts the integer representation of superblock incompatible features to IncompatFeatures struct.

func (IncompatFeatures) ToInt 

func (f IncompatFeatures) ToInt() uint32

ToInt converts superblock incompatible features back to its 32-bit rep.

type Inode 

type Inode interface {
	// Mode returns the linux file mode which is majorly used to extract
	// information like:
	// - File permissions (read/write/execute by user/group/others).
	// - Sticky, set UID and GID bits.
	// - File type.
	//
	// Masks to extract this information are provided in pkg/abi/linux/file.go.
	Mode() linux.FileMode

	// UID returns the owner UID.
	UID() auth.KUID

	// GID returns the owner GID.
	GID() auth.KGID

	// Size returns the size of the file in bytes.
	Size() uint64

	// InodeSize returns the size of this inode struct in bytes.
	// In ext2 and ext3, the inode struct and inode disk record size was fixed at
	// 128 bytes. Ext4 makes it possible for the inode struct to be bigger.
	// However, accessing any field beyond the 128 bytes marker must be verified
	// using this method.
	InodeSize() uint16

	// AccessTime returns the last access time. Shows when the file was last read.
	//
	// If InExtendedAttr is set, then this should NOT be used because the
	// underlying field is used to store the extended attribute value checksum.
	AccessTime() time.Time

	// ChangeTime returns the last change time. Shows when the file meta data
	// (like permissions) was last changed.
	//
	// If InExtendedAttr is set, then this should NOT be used because the
	// underlying field is used to store the lower 32 bits of the attribute
	// value’s reference count.
	ChangeTime() time.Time

	// ModificationTime returns the last modification time. Shows when the file
	// content was last modified.
	//
	// If InExtendedAttr is set, then this should NOT be used because
	// the underlying field contains the number of the inode that owns the
	// extended attribute.
	ModificationTime() time.Time

	// DeletionTime returns the deletion time. Inodes are marked as deleted by
	// writing to the underlying field. FS tools can restore files until they are
	// actually overwritten.
	DeletionTime() time.Time

	// LinksCount returns the number of hard links to this inode.
	//
	// Normally there is an upper limit on the number of hard links:
	//   - ext2/ext3 = 32,000
	//   - ext4      = 65,000
	//
	// This implies that an ext4 directory cannot have more than 64,998
	// subdirectories because each subdirectory will have a hard link to the
	// directory via the `..` entry. The directory has hard link via the `.` entry
	// of its own. And finally the inode is initiated with 1 hard link (itself).
	//
	// The underlying value is reset to 1 if all the following hold:
	//     - Inode is a directory.
	//     - SbDirNlink is enabled.
	//     - Number of hard links is incremented past 64,999.
	// Hard link value of 1 for a directory would indicate that the number of hard
	// links is unknown because a directory can have minimum 2 hard links (itself
	// and `.` entry).
	LinksCount() uint16

	// Flags returns InodeFlags which represents the inode flags.
	Flags() InodeFlags

	// Data returns the underlying inode.i_block array as a slice so it's
	// modifiable. This field is special and is used to store various kinds of
	// things depending on the filesystem version and inode type. The underlying
	// field name in Linux is a little misleading.
	//   - In ext2/ext3, it contains the block map.
	//   - In ext4, it contains the extent tree root node.
	//   - For inline files, it contains the file contents.
	//   - For symlinks, it contains the link path (if it fits here).
	//
	// See https://www.kernel.org/doc/html/latest/filesystems/ext4/dynamic.html#the-contents-of-inode-i-block.
	Data() []byte
}

The Inode interface must be implemented by structs representing ext inodes. The inode stores all the metadata pertaining to the file (except for the file name which is held by the directory entry). It does NOT expose all fields and should be extended if need be.

Some file systems (e.g. FAT) use the directory entry to store all this information. Ext file systems do not so that they can support hard links. However, ext4 cheats a little bit and duplicates the file type in the directory entry for performance gains.

See https://www.kernel.org/doc/html/latest/filesystems/ext4/dynamic.html#index-nodes.

type InodeFlags 

type InodeFlags struct {
	Sync         bool
	Immutable    bool
	Append       bool
	NoDump       bool
	NoAccessTime bool
	Index        bool
	JournalData  bool
	DirSync      bool
	TopDir       bool
	HugeFile     bool
	Extents      bool
	ExtendedAttr bool
	Inline       bool
	Reserved     bool
}

InodeFlags represents all possible combinations of inode flags. It aims to cover the bit masks and provide a more user-friendly interface.

func InodeFlagsFromInt 

func InodeFlagsFromInt(f uint32) InodeFlags

InodeFlagsFromInt converts the integer representation of inode flags to a InodeFlags struct.

func (InodeFlags) ToInt 

func (f InodeFlags) ToInt() uint32

ToInt converts inode flags back to its 32-bit rep.

type InodeNew 

type InodeNew struct {
	InodeOld

	ExtraInodeSize        uint16
	ChecksumHi            uint16
	ChangeTimeExtra       uint32
	ModificationTimeExtra uint32
	AccessTimeExtra       uint32
	CreationTime          uint32
	CreationTimeExtra     uint32
	VersionHi             uint32
	ProjectID             uint32
}

InodeNew represents ext4 inode structure which can be bigger than OldInodeSize. The actual size of this struct should be determined using inode.ExtraInodeSize. Accessing any field here should be verified with the actual size. The extra space between the end of the inode struct and end of the inode record can be used to store extended attr.

If the TimeExtra fields are in scope, the lower 2 bits of those are used to extend their counter part to be 34 bits wide; the rest (upper) 30 bits are used to provide nanoscond precision. Hence, these timestamps will now overflow in May 2446. See https://www.kernel.org/doc/html/latest/filesystems/ext4/dynamic.html#inode-timestamps.

func (*InodeNew) AccessTime 

func (in *InodeNew) AccessTime() time.Time

AccessTime implements Inode.AccessTime.

func (*InodeNew) ChangeTime 

func (in *InodeNew) ChangeTime() time.Time

ChangeTime implements Inode.ChangeTime.

func (*InodeNew) InodeSize 

func (in *InodeNew) InodeSize() uint16

InodeSize implements Inode.InodeSize.

func (*InodeNew) ModificationTime 

func (in *InodeNew) ModificationTime() time.Time

ModificationTime implements Inode.ModificationTime.

func (*InodeNew) Size 

func (in *InodeNew) Size() uint64

Size implements Inode.Size.

type InodeOld 

type InodeOld struct {
	ModeRaw uint16
	UIDLo   uint16
	SizeLo  uint32

	// The time fields are signed integers because they could be negative to
	// represent time before the epoch.
	AccessTimeRaw       int32
	ChangeTimeRaw       int32
	ModificationTimeRaw int32
	DeletionTimeRaw     int32

	GIDLo         uint16
	LinksCountRaw uint16
	BlocksCountLo uint32
	FlagsRaw      uint32
	VersionLo     uint32 // This is OS dependent.
	DataRaw       [60]byte
	Generation    uint32
	FileACLLo     uint32
	SizeHi        uint32
	ObsoFaddr     uint32

	// OS dependent fields have been inlined here.
	BlocksCountHi uint16
	FileACLHi     uint16
	UIDHi         uint16
	GIDHi         uint16
	ChecksumLo    uint16
	// contains filtered or unexported fields
}

InodeOld implements Inode interface. It emulates ext2/ext3 inode struct. Inode struct size and record size are both 128 bytes for this.

All fields representing time are in seconds since the epoch. Which means that they will overflow in January 2038.

func (*InodeOld) AccessTime 

func (in *InodeOld) AccessTime() time.Time

AccessTime implements Inode.AccessTime.

func (*InodeOld) ChangeTime 

func (in *InodeOld) ChangeTime() time.Time

ChangeTime implements Inode.ChangeTime.

func (*InodeOld) Data 

func (in *InodeOld) Data() []byte

Data implements Inode.Data.

func (*InodeOld) DeletionTime 

func (in *InodeOld) DeletionTime() time.Time

DeletionTime implements Inode.DeletionTime.

func (*InodeOld) Flags 

func (in *InodeOld) Flags() InodeFlags

Flags implements Inode.Flags.

func (*InodeOld) GID 

func (in *InodeOld) GID() auth.KGID

GID implements Inode.GID.

func (*InodeOld) InodeSize 

func (in *InodeOld) InodeSize() uint16

InodeSize implements Inode.InodeSize.

func (*InodeOld) LinksCount 

func (in *InodeOld) LinksCount() uint16

LinksCount implements Inode.LinksCount.

func (*InodeOld) Mode 

func (in *InodeOld) Mode() linux.FileMode

Mode implements Inode.Mode.

func (*InodeOld) ModificationTime 

func (in *InodeOld) ModificationTime() time.Time

ModificationTime implements Inode.ModificationTime.

func (*InodeOld) Size 

func (in *InodeOld) Size() uint64

Size implements Inode.Size.

func (*InodeOld) UID 

func (in *InodeOld) UID() auth.KUID

UID implements Inode.UID.

type RoCompatFeatures 

type RoCompatFeatures struct {
	Sparse       bool
	LargeFile    bool
	HugeFile     bool
	GdtCsum      bool
	DirNlink     bool
	ExtraIsize   bool
	HasSnapshot  bool
	Quota        bool
	Bigalloc     bool
	MetadataCsum bool
	ReadOnly     bool
}

RoCompatFeatures represents a superblock's readonly compatible feature set. If the kernel does not understand any of these feature, it can still mount readonly. But if the user wants to mount read/write, the kernel should refuse to mount.

func RoCompatFeaturesFromInt 

func RoCompatFeaturesFromInt(f uint32) RoCompatFeatures

RoCompatFeaturesFromInt converts the integer representation of superblock readonly compatible features to RoCompatFeatures struct.

func (RoCompatFeatures) ToInt 

func (f RoCompatFeatures) ToInt() uint32

ToInt converts superblock readonly compatible features to its 32-bit rep.

type SbRevision 

type SbRevision uint32

SbRevision is the type for superblock revisions. 

const (
	// OldRev is the good old (original) format.
	OldRev SbRevision = 0

	// DynamicRev is v2 format w/ dynamic inode sizes.
	DynamicRev SbRevision = 1
)

Super block revisions.

type SuperBlock 

type SuperBlock interface {
	// InodesCount returns the total number of inodes in this filesystem.
	InodesCount() uint32

	// BlocksCount returns the total number of data blocks in this filesystem.
	BlocksCount() uint64

	// FreeBlocksCount returns the number of free blocks in this filesystem.
	FreeBlocksCount() uint64

	// FreeInodesCount returns the number of free inodes in this filesystem.
	FreeInodesCount() uint32

	// MountCount returns the number of mounts since the last fsck.
	MountCount() uint16

	// MaxMountCount returns the number of mounts allowed beyond which a fsck is
	// needed.
	MaxMountCount() uint16

	// FirstDataBlock returns the absolute block number of the first data block,
	// which contains the super block itself.
	//
	// If the filesystem has 1kb data blocks then this should return 1. For all
	// other configurations, this typically returns 0.
	FirstDataBlock() uint32

	// BlockSize returns the size of one data block in this filesystem.
	// This can be calculated by 2^(10 + sb.s_log_block_size). This ensures that
	// the smallest block size is 1kb.
	BlockSize() uint64

	// BlocksPerGroup returns the number of data blocks in a block group.
	BlocksPerGroup() uint32

	// ClusterSize returns block cluster size (set during mkfs time by admin).
	// This can be calculated by 2^(10 + sb.s_log_cluster_size). This ensures that
	// the smallest cluster size is 1kb.
	//
	// sb.s_log_cluster_size must equal sb.s_log_block_size if bigalloc feature
	// is NOT set and consequently BlockSize() = ClusterSize() in that case.
	ClusterSize() uint64

	// ClustersPerGroup returns:
	//     - number of clusters per group        if bigalloc is enabled.
	//     - BlocksPerGroup()                    otherwise.
	ClustersPerGroup() uint32

	// InodeSize returns the size of the inode disk record size in bytes. Use this
	// to iterate over inode arrays on disk.
	//
	// In ext2 and ext3:
	//     - Each inode had a disk record of 128 bytes.
	//     - The inode struct size was fixed at 128 bytes.
	//
	// In ext4 its possible to allocate larger on-disk inodes:
	//     - Inode disk record size = sb.s_inode_size (function return value).
	//                              = 256 (default)
	//     - Inode struct size = 128 + inode.i_extra_isize.
	//                         = 128 + 32 = 160 (default)
	InodeSize() uint16

	// InodesPerGroup returns the number of inodes in a block group.
	InodesPerGroup() uint32

	// BgDescSize returns the size of the block group descriptor struct.
	//
	// In ext2, ext3, ext4 (without 64-bit feature), the block group descriptor
	// is only 32 bytes long.
	// In ext4 with 64-bit feature, the block group descriptor expands to AT LEAST
	// 64 bytes. It might be bigger than that.
	BgDescSize() uint16

	// CompatibleFeatures returns the CompatFeatures struct which holds all the
	// compatible features this fs supports.
	CompatibleFeatures() CompatFeatures

	// IncompatibleFeatures returns the CompatFeatures struct which holds all the
	// incompatible features this fs supports.
	IncompatibleFeatures() IncompatFeatures

	// ReadOnlyCompatibleFeatures returns the CompatFeatures struct which holds all the
	// readonly compatible features this fs supports.
	ReadOnlyCompatibleFeatures() RoCompatFeatures

	// Magic() returns the magic signature which must be 0xef53.
	Magic() uint16

	// Revision returns the superblock revision. Superblock struct fields from
	// offset 0x54 till 0x150 should only be used if superblock has DynamicRev.
	Revision() SbRevision
}

SuperBlock should be implemented by structs representing the ext superblock. The superblock holds a lot of information about the enclosing filesystem. This interface aims to provide access methods to important information held by the superblock. It does NOT expose all fields of the superblock, only the ones necessary. This can be expanded when need be.

Location and replication:

  • The superblock is located at offset 1024 in block group 0.
  • Redundant copies of the superblock and group descriptors are kept in all groups if SbSparse feature flag is NOT set. If it is set, the replicas only exist in groups whose group number is either 0 or a power of 3, 5, or 7.
  • There is also a sparse superblock feature v2 in which there are just two replicas saved in the block groups pointed by sb.s_backup_bgs.

Replicas should eventually be updated if the superblock is updated.

See https://www.kernel.org/doc/html/latest/filesystems/ext4/globals.html#super-block.

type SuperBlock32Bit 

type SuperBlock32Bit struct {
	// We embed the old superblock struct here because the 32-bit version is just
	// an extension of the old version.
	SuperBlockOld

	FirstInode         uint32
	InodeSizeRaw       uint16
	BlockGroupNumber   uint16
	FeatureCompat      uint32
	FeatureIncompat    uint32
	FeatureRoCompat    uint32
	UUID               [16]byte
	VolumeName         [16]byte
	LastMounted        [64]byte
	AlgoUsageBitmap    uint32
	PreallocBlocks     uint8
	PreallocDirBlocks  uint8
	ReservedGdtBlocks  uint16
	JournalUUID        [16]byte
	JournalInum        uint32
	JournalDev         uint32
	LastOrphan         uint32
	HashSeed           [4]uint32
	DefaultHashVersion uint8
	JnlBackupType      uint8
	BgDescSizeRaw      uint16
	DefaultMountOpts   uint32
	FirstMetaBg        uint32
	MkfsTime           uint32
	JnlBlocks          [17]uint32
}

SuperBlock32Bit implements SuperBlock and represents the 32-bit version of the ext4_super_block struct in fs/ext4/ext4.h. Should be used only if RevLevel = DynamicRev and 64-bit feature is disabled.

func (*SuperBlock32Bit) CompatibleFeatures 

func (sb *SuperBlock32Bit) CompatibleFeatures() CompatFeatures

CompatibleFeatures implements SuperBlock.CompatibleFeatures.

func (*SuperBlock32Bit) IncompatibleFeatures 

func (sb *SuperBlock32Bit) IncompatibleFeatures() IncompatFeatures

IncompatibleFeatures implements SuperBlock.IncompatibleFeatures.

func (*SuperBlock32Bit) InodeSize 

func (sb *SuperBlock32Bit) InodeSize() uint16

InodeSize implements SuperBlock.InodeSize.

func (*SuperBlock32Bit) ReadOnlyCompatibleFeatures 

func (sb *SuperBlock32Bit) ReadOnlyCompatibleFeatures() RoCompatFeatures

ReadOnlyCompatibleFeatures implements SuperBlock.ReadOnlyCompatibleFeatures.

type SuperBlock64Bit 

type SuperBlock64Bit struct {
	// We embed the 32-bit struct here because 64-bit version is just an extension
	// of the 32-bit version.
	SuperBlock32Bit

	BlocksCountHi         uint32
	ReservedBlocksCountHi uint32
	FreeBlocksCountHi     uint32
	MinInodeSize          uint16
	WantInodeSize         uint16
	Flags                 uint32
	RaidStride            uint16
	MmpInterval           uint16
	MmpBlock              uint64
	RaidStripeWidth       uint32
	LogGroupsPerFlex      uint8
	ChecksumType          uint8

	KbytesWritten           uint64
	SnapshotInum            uint32
	SnapshotID              uint32
	SnapshotRsrvBlocksCount uint64
	SnapshotList            uint32
	ErrorCount              uint32
	FirstErrorTime          uint32
	FirstErrorInode         uint32
	FirstErrorBlock         uint64
	FirstErrorFunction      [32]byte
	FirstErrorLine          uint32
	LastErrorTime           uint32
	LastErrorInode          uint32
	LastErrorLine           uint32
	LastErrorBlock          uint64
	LastErrorFunction       [32]byte
	MountOpts               [64]byte
	UserQuotaInum           uint32
	GroupQuotaInum          uint32
	OverheadBlocks          uint32
	BackupBgs               [2]uint32
	EncryptAlgos            [4]uint8
	EncryptPwSalt           [16]uint8
	LostFoundInode          uint32
	ProjectQuotaInode       uint32
	ChecksumSeed            uint32
	WtimeHi                 uint8
	MtimeHi                 uint8
	MkfsTimeHi              uint8
	LastCheckHi             uint8
	FirstErrorTimeHi        uint8
	LastErrorTimeHi         uint8

	Encoding      uint16
	EncodingFlags uint16

	Checksum uint32
	// contains filtered or unexported fields
}

SuperBlock64Bit implements SuperBlock and represents the 64-bit version of the ext4_super_block struct in fs/ext4/ext4.h. This sums up to be exactly 1024 bytes (smallest possible block size) and hence the superblock always fits in no more than one data block. Should only be used when the 64-bit feature is set.

func (*SuperBlock64Bit) BgDescSize 

func (sb *SuperBlock64Bit) BgDescSize() uint16

BgDescSize implements SuperBlock.BgDescSize.

func (*SuperBlock64Bit) BlocksCount 

func (sb *SuperBlock64Bit) BlocksCount() uint64

BlocksCount implements SuperBlock.BlocksCount.

func (*SuperBlock64Bit) FreeBlocksCount 

func (sb *SuperBlock64Bit) FreeBlocksCount() uint64

FreeBlocksCount implements SuperBlock.FreeBlocksCount.

type SuperBlockOld 

type SuperBlockOld struct {
	InodesCountRaw      uint32
	BlocksCountLo       uint32
	ReservedBlocksCount uint32
	FreeBlocksCountLo   uint32
	FreeInodesCountRaw  uint32
	FirstDataBlockRaw   uint32
	LogBlockSize        uint32
	LogClusterSize      uint32
	BlocksPerGroupRaw   uint32
	ClustersPerGroupRaw uint32
	InodesPerGroupRaw   uint32
	Mtime               uint32
	Wtime               uint32
	MountCountRaw       uint16
	MaxMountCountRaw    uint16
	MagicRaw            uint16
	State               uint16
	Errors              uint16
	MinorRevLevel       uint16
	LastCheck           uint32
	CheckInterval       uint32
	CreatorOS           uint32
	RevLevel            uint32
	DefResUID           uint16
	DefResGID           uint16
}

SuperBlockOld implements SuperBlock and represents the old version of the superblock struct. Should be used only if RevLevel = OldRev.

func (*SuperBlockOld) BgDescSize 

func (sb *SuperBlockOld) BgDescSize() uint16

BgDescSize implements SuperBlock.BgDescSize.

func (*SuperBlockOld) BlockSize 

func (sb *SuperBlockOld) BlockSize() uint64

BlockSize implements SuperBlock.BlockSize.

func (*SuperBlockOld) BlocksCount 

func (sb *SuperBlockOld) BlocksCount() uint64

BlocksCount implements SuperBlock.BlocksCount.

func (*SuperBlockOld) BlocksPerGroup 

func (sb *SuperBlockOld) BlocksPerGroup() uint32

BlocksPerGroup implements SuperBlock.BlocksPerGroup.

func (*SuperBlockOld) ClusterSize 

func (sb *SuperBlockOld) ClusterSize() uint64

ClusterSize implements SuperBlock.ClusterSize.

func (*SuperBlockOld) ClustersPerGroup 

func (sb *SuperBlockOld) ClustersPerGroup() uint32

ClustersPerGroup implements SuperBlock.ClustersPerGroup.

func (*SuperBlockOld) CompatibleFeatures 

func (sb *SuperBlockOld) CompatibleFeatures() CompatFeatures

CompatibleFeatures implements SuperBlock.CompatibleFeatures.

func (*SuperBlockOld) FirstDataBlock 

func (sb *SuperBlockOld) FirstDataBlock() uint32

FirstDataBlock implements SuperBlock.FirstDataBlock.

func (*SuperBlockOld) FreeBlocksCount 

func (sb *SuperBlockOld) FreeBlocksCount() uint64

FreeBlocksCount implements SuperBlock.FreeBlocksCount.

func (*SuperBlockOld) FreeInodesCount 

func (sb *SuperBlockOld) FreeInodesCount() uint32

FreeInodesCount implements SuperBlock.FreeInodesCount.

func (*SuperBlockOld) IncompatibleFeatures 

func (sb *SuperBlockOld) IncompatibleFeatures() IncompatFeatures

IncompatibleFeatures implements SuperBlock.IncompatibleFeatures.

func (*SuperBlockOld) InodeSize 

func (sb *SuperBlockOld) InodeSize() uint16

InodeSize implements SuperBlock.InodeSize.

func (*SuperBlockOld) InodesCount 

func (sb *SuperBlockOld) InodesCount() uint32

InodesCount implements SuperBlock.InodesCount.

func (*SuperBlockOld) InodesPerGroup 

func (sb *SuperBlockOld) InodesPerGroup() uint32

InodesPerGroup implements SuperBlock.InodesPerGroup.

func (*SuperBlockOld) Magic 

func (sb *SuperBlockOld) Magic() uint16

Magic implements SuperBlock.Magic.

func (*SuperBlockOld) MaxMountCount 

func (sb *SuperBlockOld) MaxMountCount() uint16

MaxMountCount implements SuperBlock.MaxMountCount.

func (*SuperBlockOld) MountCount 

func (sb *SuperBlockOld) MountCount() uint16

MountCount implements SuperBlock.MountCount.

func (*SuperBlockOld) ReadOnlyCompatibleFeatures 

func (sb *SuperBlockOld) ReadOnlyCompatibleFeatures() RoCompatFeatures

ReadOnlyCompatibleFeatures implements SuperBlock.ReadOnlyCompatibleFeatures.

func (*SuperBlockOld) Revision 

func (sb *SuperBlockOld) Revision() SbRevision

Revision implements SuperBlock.Revision.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.