squashfs

README

squashfs

A PURE Go library to read squashfs. There is currently no plans to add archive creation support as it will almost always be better to just call mksquashfs. I could see some possible use cases, but probably won't spend time on it unless it's requested (open a discussion if you want this feature).

The library has two parts with this github.com/CalebQ42/squashfs being easy to use as it implements io/fs interfaces and doesn't expose unnecessary information. 95% this is the library you want. If you need lower level access to the information, use github.com/CalebQ42/squashfs/low where far more information is exposed.

Currently has support for reading squashfs files and extracting files and folders.

Special thanks to https://dr-emann.github.io/squashfs/ for some VERY important information in an easy to understand format. Thanks also to distri's squashfs library as I referenced it to figure some things out (and double check others).

FUSE

As of v1.0, FUSE capabilities has been moved to a separate library.

Limitations

• No Xattr parsing.
• Socket files are not extracted.
  • From my research, it seems like a socket file would be useless if it could be created.
• Fifo files are ignored on darwin

Issues

• Significantly slower then unsquashfs when nested images
  • This seems to be related to above along with the general optimization of unsquashfs and it's compression libraries.
    • Not to mention it's written in C
  • Times seem to be largely dependent on file tree size and compression type.
    • My main testing image (~100MB) using Zstd takes about 5x longer.
    • An Arch Linux airootfs image (~780MB) using XZ compression with LZMA filters takes about 30x longer.
    • A Tensorflow docker image (~3.3GB) using Zstd takes about 12x longer.

Note: These numbers are using FastOptions(). DefaultOptions() takes about 2x longer.

Recommendations on Usage

Due to the above performance consideration, this library should only be used to access files within the archive without extraction, or to mount it via Fuse.

• Neither of these use cases are largely effected by the issue above.

Documentation

Index

• type ExtractionOptions
  • func DefaultOptions() *ExtractionOptions
  • func FastOptions() *ExtractionOptions
• type FS
  • func (f *FS) Extract(folder string) error
  • func (f *FS) ExtractWithOptions(folder string, op *ExtractionOptions) error
  • func (f *FS) File() *File
  • func (f *FS) Glob(pattern string) (out []string, err error)
  • func (f *FS) Open(name string) (fs.File, error)
  • func (f *FS) ReadDir(name string) ([]fs.DirEntry, error)
  • func (f *FS) ReadFile(name string) (out []byte, err error)
  • func (f *FS) Stat(name string) (fs.FileInfo, error)
  • func (f *FS) Sub(dir string) (fs.FS, error)
• type File
  • func (f *File) Close() error
  • func (f *File) Extract(folder string) error
  • func (f *File) ExtractWithOptions(path string, op *ExtractionOptions) error
  • func (f *File) FS() (*FS, error)
  • func (f *File) GetSymlinkFile() fs.File
  • func (f *File) IsDir() bool
  • func (f *File) IsRegular() bool
  • func (f *File) IsSymlink() bool
  • func (f *File) Mode() fs.FileMode
  • func (f *File) Read(b []byte) (int, error)
  • func (f *File) ReadDir(n int) ([]fs.DirEntry, error)
  • func (f *File) Stat() (fs.FileInfo, error)
  • func (f *File) SymlinkPath() string
  • func (f *File) WriteTo(w io.Writer) (int64, error)
• type Reader
  • func NewReader(r io.ReaderAt) (*Reader, error)
  • func NewReaderAtOffset(r io.ReaderAt, offset int64) (*Reader, error)
  • func (r *Reader) FSFromDirectory(d squashfslow.Directory, parent *FS) *FS
  • func (r *Reader) FileFromBase(b squashfslow.FileBase, parent *FS) *File
  • func (r *Reader) ModTime() time.Time

Types

type ExtractionOptions

type ExtractionOptions struct {
	LogOutput          io.Writer   //Where the verbose log should write.
	DereferenceSymlink bool        //Replace symlinks with the target file.
	UnbreakSymlink     bool        //Try to make sure symlinks remain unbroken when extracted, without changing the symlink.
	Verbose            bool        //Prints extra info to log on an error.
	IgnorePerm         bool        //Ignore file's permissions and instead use Perm.
	Perm               fs.FileMode //Permission to use when IgnorePerm. Defaults to 0777.
	SimultaneousFiles  uint16      //Number of files to process in parallel. Default set based on runtime.NumCPU().
	ExtractionRoutines uint16      //Number of goroutines to use for each file's extraction. Only applies to regular files. Default set based on runtime.NumCPU().
	// contains filtered or unexported fields
}

func DefaultOptions

func DefaultOptions() *ExtractionOptions

The default extraction options.

func FastOptions

func FastOptions() *ExtractionOptions

Less limited default options. Can run up 2x faster than DefaultOptions. Tends to use all available CPU resources.

type FS

type FS struct {
	// contains filtered or unexported fields
}

FS is a fs.FS representation of a squashfs directory. Implements fs.GlobFS, fs.ReadDirFS, fs.ReadFileFS, fs.StatFS, and fs.SubFS

func (*FS) Extract

func (f *FS) Extract(folder string) error

Extract the FS to the given folder. If the file is a folder, the folder's contents will be extracted to the folder. Uses default extraction options.

func (*FS) ExtractWithOptions

func (f *FS) ExtractWithOptions(folder string, op *ExtractionOptions) error

Extract the FS to the given folder. If the file is a folder, the folder's contents will be extracted to the folder. Allows setting various extraction options via ExtractionOptions.

func (*FS) File

func (f *FS) File() *File

Returns the FS as a *File

func (*FS) Glob

func (f *FS) Glob(pattern string) (out []string, err error)

Glob returns the name of the files at the given pattern. All paths are relative to the FS. Uses filepath.Match to compare names.

func (*FS) Open

func (f *FS) Open(name string) (fs.File, error)

Opens the file at name. Returns a *File as an fs.File.

func (*FS) ReadDir

func (f *FS) ReadDir(name string) ([]fs.DirEntry, error)

Returns all DirEntry's for the directory at name. If name is not a directory, returns an error.

func (*FS) ReadFile

func (f *FS) ReadFile(name string) (out []byte, err error)

Returns the contents of the file at name.

func (*FS) Stat

func (f *FS) Stat(name string) (fs.FileInfo, error)

Returns the fs.FileInfo for the file at name.

func (*FS) Sub

func (f *FS) Sub(dir string) (fs.FS, error)

Returns the FS at dir

type File

type File struct {
	// contains filtered or unexported fields
}

File represents a file inside a squashfs archive.

func (*File) Close

func (f *File) Close() error

Closes the underlying readers. Further calls to Read and WriteTo will re-create the readers. Never returns an error.

func (*File) Extract

func (f *File) Extract(folder string) error

Extract the file to the given folder. If the file is a folder, the folder's contents will be extracted to the folder. Uses default extraction options.

func (*File) ExtractWithOptions

func (f *File) ExtractWithOptions(path string, op *ExtractionOptions) error

Extract the file to the given folder. If the file is a folder, the folder's contents will be extracted to the folder. Allows setting various extraction options via ExtractionOptions.

func (*File) FS

func (f *File) FS() (*FS, error)

func (*File) GetSymlinkFile

func (f *File) GetSymlinkFile() fs.File

Returns the file the symlink points to. If the file isn't a symlink, or points to a file outside the archive, returns nil.

func (*File) IsDir

func (f *File) IsDir() bool

Returns whether the file is a directory.

func (*File) IsRegular

func (f *File) IsRegular() bool

Returns whether the file is a regular file.

func (*File) IsSymlink

func (f *File) IsSymlink() bool

Returns whether the file is a symlink.

func (*File) Mode

func (f *File) Mode() fs.FileMode

func (*File) Read

func (f *File) Read(b []byte) (int, error)

Read reads the data from the file. Only works if file is a normal file.

func (*File) ReadDir

func (f *File) ReadDir(n int) ([]fs.DirEntry, error)

ReadDir returns n fs.DirEntry's that's contained in the File (if it's a directory). If n <= 0 all fs.DirEntry's are returned.

func (*File) Stat

func (f *File) Stat() (fs.FileInfo, error)

Returns the file's fs.FileInfo

func (*File) SymlinkPath

func (f *File) SymlinkPath() string

SymlinkPath returns the symlink's target path. Is the File isn't a symlink, returns an empty string.

func (*File) WriteTo

func (f *File) WriteTo(w io.Writer) (int64, error)

Writes all data from the file to the given writer in a multi-threaded manner. The underlying reader is separate

type Reader

type Reader struct {
	*FS
	Low squashfslow.Reader
}

func NewReader

func NewReader(r io.ReaderAt) (*Reader, error)

func NewReaderAtOffset

func NewReaderAtOffset(r io.ReaderAt, offset int64) (*Reader, error)

func (*Reader) FSFromDirectory

func (r *Reader) FSFromDirectory(d squashfslow.Directory, parent *FS) *FS

Creates a new *FS from the given squashfs.directory

func (*Reader) FileFromBase

func (r *Reader) FileFromBase(b squashfslow.FileBase, parent *FS) *File

Creates a new *File from the given *squashfs.Base

func (*Reader) ModTime

func (r *Reader) ModTime() time.Time