modzip

Documentation

Overview

Package modzip provides functions for creating and extracting module zip files.

WARNING: THIS PACKAGE IS EXPERIMENTAL. ITS API MAY CHANGE AT ANY TIME.

Module zip files have several restrictions listed below. These are necessary to ensure that module zip files can be extracted consistently on supported platforms and file systems.

• All file paths within a zip file must be valid (see cuelang.org/go/mod/module.CheckFilePath).

• No two file paths may be equal under Unicode case-folding (see strings.EqualFold).

• A cue.mod/module.cue file must appear in the top-level directory. If present, it must be named exactly that, not any other case. Directories or files named "cue.mod" are not allowed in any other directory.

• The total size in bytes of a module zip file may be at most MaxZipFile bytes (500 MiB). The total uncompressed size of the files within the zip may also be at most MaxZipFile bytes.

• Each file's uncompressed size must match its declared 64-bit uncompressed size in the zip file header.

• If the zip contains files named "cue.mod/module.cue" or "LICENSE", their sizes in bytes may be at most MaxCUEMod or MaxLICENSE, respectively (both are 16 MiB).

• Empty directories are ignored. File permissions and timestamps are also ignored.

• Symbolic links and other irregular files are not allowed.

Note that this package does not provide hashing functionality. See golang.org/x/mod/sumdb/dirhash.

Index

• Constants
• func Create[F any](w io.Writer, m module.Version, files []F, fio FileIO[F]) (err error)
• func CreateFromDir(w io.Writer, m module.Version, dir string) (err error)
• func Unzip(dir string, m module.Version, zipFile string) (err error)
• type CheckedFiles
  • func CheckDir(dir string) (CheckedFiles, error)
  • func CheckFiles[F any](files []F, fio FileIO[F]) (CheckedFiles, error)
  • func CheckZip(m module.Version, r io.ReaderAt, zipSize int64) (*zip.Reader, *zip.File, CheckedFiles, error)
  • func CheckZipFile(m module.Version, zipFile string) (CheckedFiles, error)
  • func (cf CheckedFiles) Err() error
• type FileError
  • func (e FileError) Error() string
  • func (e FileError) Unwrap() error
• type FileErrorList
  • func (el FileErrorList) Error() string
• type FileIO

Constants

const (
	// MaxZipFile is the maximum size in bytes of a module zip file. The
	// cue command will report an error if either the zip file or its extracted
	// content is larger than this.
	MaxZipFile = 500 << 20

	// MaxCUEMod is the maximum size in bytes of a cue.mod/module.cue file within a
	// module zip file.
	MaxCUEMod = 16 << 20

	// MaxLICENSE is the maximum size in bytes of a LICENSE file within a
	// module zip file.
	MaxLICENSE = 16 << 20
)

Functions

func Create

func Create[F any](w io.Writer, m module.Version, files []F, fio FileIO[F]) (err error)

Create builds a zip archive for module m from an abstract list of files and writes it to w, after first sorting the slice of files in a path-aware lexical fashion (files first, then directories, both sorted lexically).

Note that m.Version is checked for validity but only the major version is used for checking correctness of the cue.mod/module.cue file.

Create verifies the restrictions described in the package documentation and should not produce an archive that Unzip cannot extract. Create does not include files in the output archive if they don't belong in the module zip. In particular, Create will not include files in modules found in subdirectories, most files in vendor directories, or irregular files (such as symbolic links) in the output archive.

Deprecated: this will be removed in a future API iteration that reduces dependence on zip archives.

func CreateFromDir

func CreateFromDir(w io.Writer, m module.Version, dir string) (err error)

CreateFromDir creates a module zip file for module m from the contents of a directory, dir. The zip content is written to w.

CreateFromDir verifies the restrictions described in the package documentation and should not produce an archive that Unzip cannot extract. CreateFromDir does not include files in the output archive if they don't belong in the module zip. In particular, CreateFromDir will not include files in modules found in subdirectories, most files in vendor directories, or irregular files (such as symbolic links) in the output archive. Additionally, unlike Create, CreateFromDir will not include directories named ".bzr", ".git", ".hg", or ".svn".

func Unzip

func Unzip(dir string, m module.Version, zipFile string) (err error)

Unzip extracts the contents of a module zip file to a directory.

Unzip checks all restrictions listed in the package documentation and returns an error if the zip archive is not valid. In some cases, files may be written to dir before an error is returned (for example, if a file's uncompressed size does not match its declared size).

dir may or may not exist: Unzip will create it and any missing parent directories if it doesn't exist. If dir exists, it must be empty.

Types

type CheckedFiles

type CheckedFiles struct {
	// Valid is a list of file paths that should be included in a zip file.
	Valid []string

	// Omitted is a list of files that are ignored when creating a module zip
	// file, along with the reason each file is ignored.
	Omitted []FileError

	// Invalid is a list of files that should not be included in a module zip
	// file, along with the reason each file is invalid.
	Invalid []FileError

	// SizeError is non-nil if the total uncompressed size of the valid files
	// exceeds the module zip size limit or if the zip file itself exceeds the
	// limit.
	SizeError error

	// NoModError is non-nil if there was no module.cue file present.
	NoModError error
}

CheckedFiles reports whether a set of files satisfy the name and size constraints required by module zip files. The constraints are listed in the package documentation.

Functions that produce this report may include slightly different sets of files. See documentation for CheckFiles, CheckDir, and CheckZip for details.

func CheckDir

func CheckDir(dir string) (CheckedFiles, error)

CheckDir reports whether the files in dir satisfy the name and size constraints listed in the package documentation. The returned CheckedFiles record contains lists of valid, invalid, and omitted files. If a directory is omitted (for example, a nested module or vendor directory), it will appear in the omitted list, but its files won't be listed.

CheckDir returns an error if it encounters an I/O error or if the returned CheckedFiles does not describe a valid module zip file (according to CheckedFiles.Err). The returned CheckedFiles is still populated when such an error is returned.

Note that CheckDir will not open any files, so CreateFromDir may still fail when CheckDir is successful due to I/O errors.

Deprecated: this will be removed in a future API iteration that reduces dependence on zip archives.

func CheckFiles

func CheckFiles[F any](files []F, fio FileIO[F]) (CheckedFiles, error)

CheckFiles reports whether a list of files satisfy the name and size constraints listed in the package documentation. The returned CheckedFiles record contains lists of valid, invalid, and omitted files. Every file in the given list will be included in exactly one of those lists.

CheckFiles returns an error if the returned CheckedFiles does not describe a valid module zip file (according to CheckedFiles.Err). The returned CheckedFiles is still populated when an error is returned.

Note that CheckFiles will not open any files, so Create may still fail when CheckFiles is successful due to I/O errors, reported size differences or an invalid module.cue file.

Deprecated: this will be removed in a future API iteration that reduces dependence on zip archives.

func CheckZip

func CheckZip(m module.Version, r io.ReaderAt, zipSize int64) (*zip.Reader, *zip.File, CheckedFiles, error)

CheckZip reports whether the files contained in a zip file satisfy the name and size constraints listed in the package documentation.

CheckZip returns an error if the returned CheckedFiles does not describe a valid module zip file (according to CheckedFiles.Err). The returned CheckedFiles is still populated when an error is returned. CheckZip will also return an error if the module path or version is malformed or if it encounters an error reading the zip file.

It also returns the file entry for the module.cue file.

Note that checkZip does not read individual files, so zip.Unzip may still fail when checkZip is successful due to I/O errors.

func CheckZipFile

func CheckZipFile(m module.Version, zipFile string) (CheckedFiles, error)

CheckZipFile calls CheckZip with the given zip file.

func (CheckedFiles) Err

func (cf CheckedFiles) Err() error

Err returns an error if CheckedFiles does not describe a valid module zip file. SizeError is returned if that field is set. A FileErrorList is returned if there are one or more invalid files. Other errors may be returned in the future.

type FileError

type FileError struct {
	Path string
	Err  error
}

func (FileError) Error

func (e FileError) Error() string

func (FileError) Unwrap

func (e FileError) Unwrap() error

type FileErrorList

type FileErrorList []FileError

func (FileErrorList) Error

func (el FileErrorList) Error() string

type FileIO

type FileIO[F any] interface {
	// Path returns a clean slash-separated relative path from the module root
	// directory to the file.
	Path(f F) string

	// Lstat returns information about the file. If the file is a symbolic link,
	// Lstat returns information about the link itself, not the file it points to.
	Lstat(f F) (os.FileInfo, error)

	// Open provides access to the data within a regular file. Open may return
	// an error if called on a directory or symbolic link.
	Open(f F) (io.ReadCloser, error)
}

File provides an abstraction for a file in a directory, zip, or anything else that looks like a file - it knows how to open files represented as a particular type without being a file itself.

Deprecated: this will be removed in a future API iteration that reduces dependence on zip archives.