squashfs

README

squashfs (WIP)

A PURE Go library to read and write squashfs.

Currently has support for reading squashfs files and extracting files and folders. Supports all compression types except LZO, but additional compression options are hit or miss.

The only major thing missing from squashfs reading is Xattr parsing.

Special thanks to https://dr-emann.github.io/squashfs/ for some VERY important information in an easy to understand format. Thanks also to distri's squashfs library as I referenced it to figure some things out (and double check others).

Performane

This library, decompressing the firefox AppImage and using go tests, takes about twice as long as unsquashfs on my quad core laptop. (~1 second with the libarary and about half a second with unsquashfs)

TODO

Documentation

Index

• Constants
• Variables
• type File
  • func (f *File) ExtractSymlink(path string) []error
  • func (f *File) ExtractTo(path string) []error
  • func (f *File) ExtractWithOptions(path string, dereferenceSymlink, unbreakSymlink bool, folderPerm os.FileMode, ...) (errs []error)
  • func (f *File) GetChildren() (children []*File, err error)
  • func (f *File) GetChildrenRecursively() (children []*File, err error)
  • func (f *File) GetFileAtPath(dirPath string) *File
  • func (f *File) GetSymlinkFile() *File
  • func (f *File) GetSymlinkFileRecursive() *File
  • func (f *File) IsDir() bool
  • func (f *File) IsFile() bool
  • func (f *File) IsSymlink() bool
  • func (f *File) ModTime() time.Time
  • func (f *File) Mode() os.FileMode
  • func (f *File) Name() string
  • func (f *File) Path() string
  • func (f *File) Read(p []byte) (int, error)
  • func (f *File) Size() int64
  • func (f *File) SymlinkPath() string
  • func (f *File) Sys() interface{}
• type Reader
  • func NewSquashfsReader(r io.ReaderAt) (*Reader, error)
  • func (r *Reader) ExtractTo(path string) []error
  • func (r *Reader) FindAll(query func(*File) bool) (all []*File)
  • func (r *Reader) FindFile(query func(*File) bool) *File
  • func (r *Reader) GetAllFiles() (fils []*File, err error)
  • func (r *Reader) GetFileAtPath(filepath string) *File
  • func (r *Reader) GetRootFolder() (*File, error)
  • func (r *Reader) ModTime() time.Time
• type Writer
  • func NewWriter() (*Writer, error)
  • func NewWriterWithOptions(compressionType int, allowErrors bool) (*Writer, error)
  • func (w *Writer) AddFile(file *os.File) error
  • func (w *Writer) AddFileTo(filepath string, file *os.File) error
  • func (w *Writer) AddFileToFolder(folder string, file *os.File) error
  • func (w *Writer) AddReaderTo(filepath string, reader io.Reader) error
  • func (w *Writer) Contains(filepath string) bool
  • func (w *Writer) FixSymlinks() (errs []error, problems bool)
  • func (w *Writer) Remove(filepath string) bool
  • func (w *Writer) WriteTo(write io.Writer) (int64, error)
  • func (w *Writer) WriteToFilename(filepath string) error

Constants

const (
	GzipCompression = 1 + iota
	LzmaCompression
	LzoCompression
	XzCompression
	Lz4Compression
	ZstdCompression
)

The types of compression supported by squashfs.

Variables

var (

	//ErrBrokenSymlink is returned when using ExtractWithOptions with the unbreakSymlink set to true, but the symlink's file cannot be extracted.
	ErrBrokenSymlink = errors.New("Extracted symlink is probably broken")
)

var (

	//ErrOptions is returned when compression options that I haven't tested is set. When this is returned, the Reader is also returned.
	ErrOptions = errors.New("Possibly incompatible compressor options")
)

Types

type File

type File struct {
	Parent *File
	// contains filtered or unexported fields
}

File is the main way to interact with files within squashfs, or when putting files into a squashfs. File can be either a file or folder. When reading from a squashfs, it reads from the datablocks. When writing, this holds the information on WHERE the file will be placed inside the archive.

If copying data from a squashfs, the returned reader from io.Sys() implements io.WriterTo which will be significantly faster then calling Read directly. Ex: use io.Sys().(io.Reader) for io.Copy instead of using the File directly.

Implements os.FileInfo and io.Reader

func (*File) ExtractSymlink

func (f *File) ExtractSymlink(path string) []error

ExtractSymlink is similar to ExtractTo, but when it extracts a symlink, it instead extracts the file associated with the symlink in it's place. This is the same as ExtractWithOptions(path, true, false, os.ModePerm, false)

func (*File) ExtractTo

func (f *File) ExtractTo(path string) []error

ExtractTo extracts the file to the given path. This is the same as ExtractWithOptions(path, false, false, os.ModePerm, false). Will NOT try to keep symlinks valid, folders extracted will have the permissions set by the squashfs, but the folder to make path will have full permissions (777).

Will try it's best to extract all files, and if any errors come up, they will be appended to the error slice that's returned.

func (*File) ExtractWithOptions

func (f *File) ExtractWithOptions(path string, dereferenceSymlink, unbreakSymlink bool, folderPerm os.FileMode, verbose bool) (errs []error)

ExtractWithOptions will extract the file to the given path, while allowing customization on how it works. ExtractTo is the "default" options. Will try it's best to extract all files, and if any errors come up, they will be appended to the error slice that's returned. Should only return multiple errors if extracting a folder.

If dereferenceSymlink is set, instead of extracting a symlink, it will extract the file the symlink is pointed to in it's place. If both dereferenceSymlink and unbreakSymlink is set, dereferenceSymlink takes precendence.

If unbreakSymlink is set, it will also try to extract the symlink's associated file. WARNING: the symlink's file may have to go up the directory to work. If unbreakSymlink is set and the file cannot be extracted, a ErrBrokenSymlink will be appended to the returned error slice.

folderPerm only applies to the folders created to get to path. Folders from the archive are given the correct permissions defined by the archive.

func (*File) GetChildren

func (f *File) GetChildren() (children []*File, err error)

GetChildren returns a *squashfs.File slice of every direct child of the directory. If the File is not a directory, will return ErrNotDirectory

func (*File) GetChildrenRecursively

func (f *File) GetChildrenRecursively() (children []*File, err error)

GetChildrenRecursively returns ALL children. Goes down ALL folder paths.

func (*File) GetFileAtPath

func (f *File) GetFileAtPath(dirPath string) *File

GetFileAtPath tries to return the File at the given path, relative to the file. Returns nil if called on something other then a folder, OR if the path goes oustide the archive. Allows wildcards supported by path.Match (namely * and ?) and will return the FIRST file that matches.

func (*File) GetSymlinkFile

func (f *File) GetSymlinkFile() *File

GetSymlinkFile tries to return the squashfs.File associated with the symlink. If the file isn't a symlink or the symlink points to a location outside the archive, nil is returned.

func (*File) GetSymlinkFileRecursive

func (f *File) GetSymlinkFileRecursive() *File

GetSymlinkFileRecursive tries to return the squasfs.File associated with the symlink. It will recursively try to get the symlink's file. This will return either a non-symlink File, or nil.

func (*File) IsDir

func (f *File) IsDir() bool

IsDir returns if the file is a directory.

func (*File) IsFile

func (f *File) IsFile() bool

IsFile returns if the file is a file.

func (*File) IsSymlink

func (f *File) IsSymlink() bool

IsSymlink returns if the file is a symlink.

func (*File) ModTime

func (f *File) ModTime() time.Time

ModTime is the time of last modification.

func (*File) Mode

func (f *File) Mode() os.FileMode

Mode returns the os.FileMode of the File. Sets mode bits for directories and symlinks.

func (*File) Name

func (f *File) Name() string

Name is the file's name

func (*File) Path

func (f *File) Path() string

Path returns the path of the file within the archive.

func (*File) Read

func (f *File) Read(p []byte) (int, error)

Read from the file. Doesn't do anything fancy, just pases it to the underlying io.Reader. If a directory, return io.EOF.

func (*File) Size

func (f *File) Size() int64

Size is the complete size of the file. Zero if it's not a file.

func (*File) SymlinkPath

func (f *File) SymlinkPath() string

SymlinkPath returns the path the symlink is pointing to. If the file ISN'T a symlink, will return an empty string. If a path begins with "/" then the symlink is pointing to an absolute path (starting from root, and not a file inside the archive)

func (*File) Sys

func (f *File) Sys() interface{}

Sys returns the underlying reader. If the reader isn't initialized, it will initialize it. If called on something other then a file, returns nil.

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader processes and reads a squashfs archive.

func NewSquashfsReader

func NewSquashfsReader(r io.ReaderAt) (*Reader, error)

NewSquashfsReader returns a new squashfs.Reader from an io.ReaderAt

func (*Reader) ExtractTo

func (r *Reader) ExtractTo(path string) []error

ExtractTo tries to extract ALL files to the given path. This is the same as getting the root folder and extracting that.

func (*Reader) FindAll

func (r *Reader) FindAll(query func(*File) bool) (all []*File)

FindAll returns all files where the given function returns true.

func (*Reader) FindFile

func (r *Reader) FindFile(query func(*File) bool) *File

FindFile returns the first file (in the same order as Reader.GetAllFiles) that the given function returns true for. Returns nil if nothing is found.

func (*Reader) GetAllFiles

func (r *Reader) GetAllFiles() (fils []*File, err error)

GetAllFiles returns a slice of ALL files and folders contained in the squashfs.

func (*Reader) GetFileAtPath

func (r *Reader) GetFileAtPath(filepath string) *File

GetFileAtPath will return the file at the given path. If the file cannot be found, will return nil.

func (*Reader) GetRootFolder

func (r *Reader) GetRootFolder() (*File, error)

GetRootFolder returns a squashfs.File that references the root directory of the squashfs archive.

func (*Reader) ModTime

func (r *Reader) ModTime() time.Time

ModTime is the last time the file was modified/created.

type Writer

type Writer struct {
	Flags superblockFlags
	// contains filtered or unexported fields
}

Writer is used to creaste squashfs archives. Currently unusable TODO: Make usable

func NewWriter

func NewWriter() (*Writer, error)

NewWriter creates a new with the default options (Gzip compression and allow errors)

func NewWriterWithOptions

func NewWriterWithOptions(compressionType int, allowErrors bool) (*Writer, error)

NewWriterWithOptions creates a new squashfs.Writer with the given options. compressionType can be of any types, except LZO (which this library doesn't have support for yet) allowErrors determines if, when adding folders, it allows errors encountered with it's sub-directories and instead logs the errors.

func (*Writer) AddFile

func (w *Writer) AddFile(file *os.File) error

AddFile attempts to add an os.File to the archive at it's root.

func (*Writer) AddFileTo

func (w *Writer) AddFileTo(filepath string, file *os.File) error

AddFileTo adds the given file to the squashfs archive at the given filepath.

func (*Writer) AddFileToFolder

func (w *Writer) AddFileToFolder(folder string, file *os.File) error

AddFileToFolder adds the given file to the squashfs archive, placing it inside the given folder.

func (*Writer) AddReaderTo

func (w *Writer) AddReaderTo(filepath string, reader io.Reader) error

AddReaderTo adds the data from the given reader to the archive as a file located at the given filepath. Data from the reader is not read until the squashfs archive is writen. If the given reader implements io.Closer, it will be closed after it is fully read.

func (*Writer) Contains

func (w *Writer) Contains(filepath string) bool

Contains returns whether a file is present at the given filepath

func (*Writer) FixSymlinks

func (w *Writer) FixSymlinks() (errs []error, problems bool)

FixSymlinks will scan through the squashfs archive and try to find broken symlinks and fix them. This done by replacing the symlink with the target file and then pointing other symlinks to that file. If all symlinks can be resolved, the error slice will be nil, and the bool false, otherwise all errors occured will be in the slice.

func (*Writer) Remove

func (w *Writer) Remove(filepath string) bool

Remove tries to remove the file(s) at the given filepath. If wildcards are used, it will remove all files that match. Returns true if one or more files are removed.

func (*Writer) WriteTo

func (w *Writer) WriteTo(write io.Writer) (int64, error)

WriteTo attempts to write the archive to the given io.Writer.

func (*Writer) WriteToFilename

func (w *Writer) WriteToFilename(filepath string) error

WriteToFilename creates the squashfs archive with the given filepath.