archive

package
v27.0.1+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 21, 2024 License: Apache-2.0 Imports: 31 Imported by: 5,086

Documentation

Overview

Package archive provides helper functions for dealing with archive files.

Index

Constants

View Source 
const (
	ChangeModify = 0 // ChangeModify represents the modify operation.
	ChangeAdd    = 1 // ChangeAdd represents the add operation.
	ChangeDelete = 2 // ChangeDelete represents the delete operation.
)
View Source 
const ImpliedDirectoryMode = 0o755

ImpliedDirectoryMode represents the mode (Unix permissions) applied to directories that are implied by files in a tar, but that do not have their own header entry.

The permissions mask is stored in a constant instead of locally to ensure that magic numbers do not proliferate in the codebase. The default value 0755 has been selected based on the default umask of 0022, and a convention of mkdir(1) calling mkdir(2) with permissions of 0777, resulting in a final value of 0755.

This value is currently implementation-defined, and not captured in any cross-runtime specification. Thus, it is subject to change in Moby at any time -- image authors who require consistent or known directory permissions should explicitly control them by ensuring that header entries exist for any applicable path.

View Source 
const WhiteoutLinkDir = WhiteoutMetaPrefix + "plnk"

WhiteoutLinkDir is a directory AUFS uses for storing hardlink links to other layers. Normally these should not go into exported archives and all changed hardlinks should be copied to the top layer.

View Source 
const WhiteoutMetaPrefix = WhiteoutPrefix + WhiteoutPrefix

WhiteoutMetaPrefix prefix means whiteout has a special meaning and is not for removing an actual file. Normally these files are excluded from exported archives.

View Source 
const WhiteoutOpaqueDir = WhiteoutMetaPrefix + ".opq"

WhiteoutOpaqueDir file means directory has been made opaque - meaning readdir calls to this directory do not follow to lower layers.

View Source 
const WhiteoutPrefix = ".wh."

WhiteoutPrefix prefix means file is a whiteout. If this is followed by a filename this means that file has been removed from the base layer.

Variables

View Source 
var (
	ErrNotDirectory      = errors.New("not a directory")
	ErrDirNotExists      = errors.New("no such directory")
	ErrCannotCopyDir     = errors.New("cannot copy directory")
	ErrInvalidCopySource = errors.New("invalid copy source content")
)

Errors used or returned by this file.

Functions

func ApplyLayer 

func ApplyLayer(dest string, layer io.Reader) (int64, error)

ApplyLayer parses a diff in the standard layer format from `layer`, and applies it to the directory `dest`. The stream `layer` can be compressed or uncompressed. Returns the size in bytes of the contents of the layer.

func ApplyUncompressedLayer added in v1.8.0 

func ApplyUncompressedLayer(dest string, layer io.Reader, options *TarOptions) (int64, error)

ApplyUncompressedLayer parses a diff in the standard layer format from `layer`, and applies it to the directory `dest`. The stream `layer` can only be uncompressed. Returns the size in bytes of the contents of the layer.

func CanonicalTarNameForPath deprecated added in v1.6.0 

func CanonicalTarNameForPath(relativePath string) string

CanonicalTarNameForPath canonicalizes relativePath to a POSIX-style path using forward slashes. It is an alias for filepath.ToSlash, which is a no-op on Linux and Unix.

Deprecated: use filepath.ToSlash. This function will be removed in the next release.

func ChangesSize 

func ChangesSize(newDir string, changes []Change) int64

ChangesSize calculates the size in bytes of the provided changes, based on newDir.

func CheckSystemDriveAndRemoveDriveLetter 

func CheckSystemDriveAndRemoveDriveLetter(path string) (string, error)

CheckSystemDriveAndRemoveDriveLetter verifies that a path, if it includes a drive letter, is the system drive. On Linux: this is a no-op. On Windows: this does the following> CheckSystemDriveAndRemoveDriveLetter verifies and manipulates a Windows path. This is used, for example, when validating a user provided path in docker cp. If a drive letter is supplied, it must be the system drive. The drive letter is always removed. Also, it translates it to OS semantics (IOW / to \). We need the path in this syntax so that it can ultimately be concatenated with a Windows long-path which doesn't support drive-letters. Examples: C: --> Fail C:\ --> \ a --> a /a --> \a d:\ --> Fail

func CompressStream 

func CompressStream(dest io.Writer, compression Compression) (io.WriteCloser, error)

CompressStream compresses the dest with specified compression algorithm.

func CopyResource added in v1.8.0 

func CopyResource(srcPath, dstPath string, followLink bool) error

CopyResource performs an archive copy from the given source path to the given destination path. The source path MUST exist and the destination path's parent directory must exist.

func CopyTo added in v1.8.0 

func CopyTo(content io.Reader, srcInfo CopyInfo, dstPath string) error

CopyTo handles extracting the given content whose entries should be sourced from srcInfo to dstPath.

func DecompressStream 

func DecompressStream(archive io.Reader) (io.ReadCloser, error)

DecompressStream decompresses the archive and returns a ReaderCloser with the decompressed archive.

func ExportChanges 

func ExportChanges(dir string, changes []Change, idMap idtools.IdentityMapping) (io.ReadCloser, error)

ExportChanges produces an Archive from the provided changes, relative to dir.

func FileInfoHeader 

func FileInfoHeader(name string, fi os.FileInfo, link string) (*tar.Header, error)

FileInfoHeader creates a populated Header from fi.

Compared to the archive/tar package, this function fills in less information but is safe to call from a chrooted process. The AccessTime and ChangeTime fields are not set in the returned header, ModTime is truncated to one-second precision, and the Uname and Gname fields are only set when fi is a FileInfo value returned from tar.Header.FileInfo().

func FileInfoHeaderNoLookups 

func FileInfoHeaderNoLookups(fi os.FileInfo, link string) (*tar.Header, error)

FileInfoHeaderNoLookups creates a partially-populated tar.Header from fi.

Compared to the archive/tar.FileInfoHeader function, this function is safe to call from a chrooted process as it does not populate fields which would require operating system lookups. It behaves identically to tar.FileInfoHeader when fi is a FileInfo value returned from tar.Header.FileInfo().

When fi is a FileInfo for a native file, such as returned from os.Stat() and os.Lstat(), the returned Header value differs from one returned from tar.FileInfoHeader in the following ways. The Uname and Gname fields are not set as OS lookups would be required to populate them. The AccessTime and ChangeTime fields are not currently set (not yet implemented) although that is subject to change. Callers which require the AccessTime or ChangeTime fields to be zeroed should explicitly zero them out in the returned Header value to avoid any compatibility issues in the future.

func Generate 

func Generate(input ...string) (io.Reader, error)

Generate generates a new archive from the content provided as input.

`files` is a sequence of path/content pairs. A new file is added to the archive for each pair. If the last pair is incomplete, the file is created with an empty content. For example:

Generate("foo.txt", "hello world", "emptyfile")

The above call will return an archive with 2 files:

  • ./foo.txt with content "hello world"
  • ./empty with empty content

FIXME: stream content instead of buffering FIXME: specify permissions and other archive metadata

func GetRebaseName added in v1.10.0 

func GetRebaseName(path, resolvedPath string) (string, string)

GetRebaseName normalizes and compares path and resolvedPath, return completed resolved path and rebased file name

func IsArchivePath added in v1.10.0 

func IsArchivePath(path string) bool

IsArchivePath checks if the (possibly compressed) file at the given path starts with a tar file header.

func IsEmpty 

func IsEmpty(rd io.Reader) (bool, error)

IsEmpty checks if the tar archive is empty (doesn't contain any entries).

func PrepareArchiveCopy added in v1.8.0 

func PrepareArchiveCopy(srcContent io.Reader, srcInfo, dstInfo CopyInfo) (dstDir string, content io.ReadCloser, err error)

PrepareArchiveCopy prepares the given srcContent archive, which should contain the archived resource described by srcInfo, to the destination described by dstInfo. Returns the possibly modified content archive along with the path to the destination directory which it should be extracted to.

func PreserveTrailingDotOrSeparator added in v1.8.0 

func PreserveTrailingDotOrSeparator(cleanedPath string, originalPath string) string

PreserveTrailingDotOrSeparator returns the given cleaned path (after processing using any utility functions from the path or filepath stdlib packages) and appends a trailing `/.` or `/` if its corresponding original path (from before being processed by utility functions from the path or filepath stdlib packages) ends with a trailing `/.` or `/`. If the cleaned path already ends in a `.` path segment, then another is not added. If the clean path already ends in a path separator, then another is not added.

func ReadSecurityXattrToTarHeader 

func ReadSecurityXattrToTarHeader(path string, hdr *tar.Header) error

ReadSecurityXattrToTarHeader reads security.capability xattr from filesystem to a tar header

func RebaseArchiveEntries added in v1.10.0 

func RebaseArchiveEntries(srcContent io.Reader, oldBase, newBase string) io.ReadCloser

RebaseArchiveEntries rewrites the given srcContent archive replacing an occurrence of oldBase with newBase at the beginning of entry names.

func ReplaceFileTarWrapper 

func ReplaceFileTarWrapper(inputTarStream io.ReadCloser, mods map[string]TarModifierFunc) io.ReadCloser

ReplaceFileTarWrapper converts inputTarStream to a new tar stream. Files in the tar stream are modified if they match any of the keys in mods.

func ResolveHostSourcePath added in v1.10.0 

func ResolveHostSourcePath(path string, followLink bool) (resolvedPath, rebaseName string, err error)

ResolveHostSourcePath decides real path need to be copied with parameters such as whether to follow symbol link or not, if followLink is true, resolvedPath will return link target of any symbol link file, else it will only resolve symlink of directory but return symbol link file itself without resolving.

func SplitPathDirEntry added in v1.8.0 

func SplitPathDirEntry(path string) (dir, base string)

SplitPathDirEntry splits the given path between its directory name and its basename by first cleaning the path but preserves a trailing "." if the original path specified the current directory.

func Tar 

func Tar(path string, compression Compression) (io.ReadCloser, error)

Tar creates an archive from the directory at `path`, and returns it as a stream of bytes.

func TarResource added in v1.8.0 

func TarResource(sourceInfo CopyInfo) (content io.ReadCloser, err error)

TarResource archives the resource described by the given CopyInfo to a Tar archive. A non-nil error is returned if sourcePath does not exist or is asserted to be a directory but exists as another type of file.

This function acts as a convenient wrapper around TarWithOptions, which requires a directory as the source path. TarResource accepts either a directory or a file path and correctly sets the Tar options.

func TarResourceRebase added in v1.8.0 

func TarResourceRebase(sourcePath, rebaseName string) (content io.ReadCloser, err error)

TarResourceRebase is like TarResource but renames the first path element of items in the resulting tar archive to match the given rebaseName if not "".

func TarWithOptions 

func TarWithOptions(srcPath string, options *TarOptions) (io.ReadCloser, error)

TarWithOptions creates an archive from the directory at `path`, only including files whose relative paths are included in `options.IncludeFiles` (if non-nil) or not in `options.ExcludePatterns`.

func Unpack added in v1.3.3 

func Unpack(decompressedArchive io.Reader, dest string, options *TarOptions) error

Unpack unpacks the decompressedArchive to dest with options.

func UnpackLayer added in v1.3.3 

func UnpackLayer(dest string, layer io.Reader, options *TarOptions) (size int64, err error)

UnpackLayer unpack `layer` to a `dest`. The stream `layer` can be compressed or uncompressed. Returns the size in bytes of the contents of the layer.

func Untar 

func Untar(tarArchive io.Reader, dest string, options *TarOptions) error

Untar reads a stream of bytes from `archive`, parses it as a tar archive, and unpacks it into the directory at `dest`. The archive may be compressed with one of the following algorithms: identity (uncompressed), gzip, bzip2, xz.

FIXME: specify behavior when target path exists vs. doesn't exist.

func UntarUncompressed added in v1.8.0 

func UntarUncompressed(tarArchive io.Reader, dest string, options *TarOptions) error

UntarUncompressed reads a stream of bytes from `archive`, parses it as a tar archive, and unpacks it into the directory at `dest`. The archive must be an uncompressed stream.

Types

type Archiver added in v1.3.2 

type Archiver struct {
	Untar     func(io.Reader, string, *TarOptions) error
	IDMapping idtools.IdentityMapping
}

Archiver implements the Archiver interface and allows the reuse of most utility functions of this package with a pluggable Untar function. Also, to facilitate the passing of specific id mappings for untar, an Archiver can be created with maps which will then be passed to Untar operations.

func NewDefaultArchiver 

func NewDefaultArchiver() *Archiver

NewDefaultArchiver returns a new Archiver without any IdentityMapping

func (*Archiver) CopyFileWithTar added in v1.3.2 

func (archiver *Archiver) CopyFileWithTar(src, dst string) (err error)

CopyFileWithTar emulates the behavior of the 'cp' command-line for a single file. It copies a regular file from path `src` to path `dst`, and preserves all its metadata.

func (*Archiver) CopyWithTar added in v1.3.2 

func (archiver *Archiver) CopyWithTar(src, dst string) error

CopyWithTar creates a tar archive of filesystem path `src`, and unpacks it at filesystem path `dst`. The archive is streamed directly with fixed buffering and no intermediary disk IO.

func (*Archiver) IdentityMapping 

func (archiver *Archiver) IdentityMapping() idtools.IdentityMapping

IdentityMapping returns the IdentityMapping of the archiver.

func (*Archiver) TarUntar added in v1.3.2 

func (archiver *Archiver) TarUntar(src, dst string) error

TarUntar is a convenience function which calls Tar and Untar, with the output of one piped into the other. If either Tar or Untar fails, TarUntar aborts and returns the error.

func (*Archiver) UntarPath added in v1.3.2 

func (archiver *Archiver) UntarPath(src, dst string) error

UntarPath untar a file from path to a destination, src is the source tar file path.

type Change 

type Change struct {
	Path string
	Kind ChangeType
}

Change represents a change, it wraps the change type and path. It describes changes of the files in the path respect to the parent layers. The change could be modify, add, delete. This is used for layer diff.

func Changes 

func Changes(layers []string, rw string) ([]Change, error)

Changes walks the path rw and determines changes for the files in the path, with respect to the parent layers

func ChangesDirs 

func ChangesDirs(newDir, oldDir string) ([]Change, error)

ChangesDirs compares two directories and generates an array of Change objects describing the changes. If oldDir is "", then all files in newDir will be Add-Changes.

func (*Change) String 

func (change *Change) String() string

type ChangeType 

type ChangeType int

ChangeType represents the change type.

func (ChangeType) String added in v1.10.0 

func (c ChangeType) String() string

type Compression 

type Compression int

Compression is the state represents if compressed or not. 

const (
	Uncompressed Compression = 0 // Uncompressed represents the uncompressed.
	Bzip2        Compression = 1 // Bzip2 is bzip2 compression algorithm.
	Gzip         Compression = 2 // Gzip is gzip compression algorithm.
	Xz           Compression = 3 // Xz is xz compression algorithm.
	Zstd         Compression = 4 // Zstd is zstd compression algorithm.
)

func DetectCompression 

func DetectCompression(source []byte) Compression

DetectCompression detects the compression algorithm of the source.

func (*Compression) Extension 

func (compression *Compression) Extension() string

Extension returns the extension of a file that uses the specified compression algorithm.

type CopyInfo added in v1.8.0 

type CopyInfo struct {
	Path       string
	Exists     bool
	IsDir      bool
	RebaseName string
}

CopyInfo holds basic info about the source or destination path of a copy operation.

func CopyInfoDestinationPath added in v1.8.0 

func CopyInfoDestinationPath(path string) (info CopyInfo, err error)

CopyInfoDestinationPath stats the given path to create a CopyInfo struct representing that resource for the destination of an archive copy operation. The given path should be an absolute local path.

func CopyInfoSourcePath added in v1.8.0 

func CopyInfoSourcePath(path string, followLink bool) (CopyInfo, error)

CopyInfoSourcePath stats the given path to create a CopyInfo struct representing that resource for the source of an archive copy operation. The given path should be an absolute local path. A source path has all symlinks evaluated that appear before the last path separator ("/" on Unix). As it is to be a copy source, the path must exist.

type FileInfo 

type FileInfo struct {
	// contains filtered or unexported fields
}

FileInfo describes the information of a file.

func (*FileInfo) Changes 

func (info *FileInfo) Changes(oldInfo *FileInfo) []Change

Changes add changes to file information.

func (*FileInfo) LookUp 

func (info *FileInfo) LookUp(path string) *FileInfo

LookUp looks up the file information of a file.

type TarModifierFunc 

type TarModifierFunc func(path string, header *tar.Header, content io.Reader) (*tar.Header, []byte, error)

TarModifierFunc is a function that can be passed to ReplaceFileTarWrapper to modify the contents or header of an entry in the archive. If the file already exists in the archive the TarModifierFunc will be called with the Header and a reader which will return the files content. If the file does not exist both header and content will be nil.

type TarOptions 

type TarOptions struct {
	IncludeFiles     []string
	ExcludePatterns  []string
	Compression      Compression
	NoLchown         bool
	IDMap            idtools.IdentityMapping
	ChownOpts        *idtools.Identity
	IncludeSourceDir bool
	// WhiteoutFormat is the expected on disk format for whiteout files.
	// This format will be converted to the standard format on pack
	// and from the standard format on unpack.
	WhiteoutFormat WhiteoutFormat
	// When unpacking, specifies whether overwriting a directory with a
	// non-directory is allowed and vice versa.
	NoOverwriteDirNonDir bool
	// For each include when creating an archive, the included name will be
	// replaced with the matching name from this map.
	RebaseNames map[string]string
	InUserNS    bool
	// Allow unpacking to succeed in spite of failures to set extended
	// attributes on the unpacked files due to the destination filesystem
	// not supporting them or a lack of permissions. Extended attributes
	// were probably in the archive for a reason, so set this option at
	// your own peril.
	BestEffortXattrs bool
}

TarOptions wraps the tar options.

func TarResourceRebaseOpts 

func TarResourceRebaseOpts(sourceBase string, rebaseName string) *TarOptions

TarResourceRebaseOpts does not preform the Tar, but instead just creates the rebase parameters to be sent to TarWithOptions (the TarOptions struct)

type Tarballer 

type Tarballer struct {
	// contains filtered or unexported fields
}

Tarballer is a lower-level interface to TarWithOptions which gives the caller control over which goroutine the archiving operation executes on.

func NewTarballer 

func NewTarballer(srcPath string, options *TarOptions) (*Tarballer, error)

NewTarballer constructs a new tarballer. The arguments are the same as for TarWithOptions.

func (*Tarballer) Do 

func (t *Tarballer) Do()

Do performs the archiving operation in the background. The resulting archive can be read from t.Reader(). Do should only be called once on each Tarballer instance.

func (*Tarballer) Reader 

func (t *Tarballer) Reader() io.ReadCloser

Reader returns the reader for the created archive.

type TempArchive deprecated 

type TempArchive struct {
	*os.File
	Size int64 // Pre-computed from Stat().Size() as a convenience
	// contains filtered or unexported fields
}

TempArchive is a temporary archive. The archive can only be read once - as soon as reading completes, the file will be deleted.

Deprecated: TempArchive is only used in tests and will be removed in the next release.

func NewTempArchive deprecated 

func NewTempArchive(src io.Reader, dir string) (*TempArchive, error)

NewTempArchive reads the content of src into a temporary file, and returns the contents of that file as an archive. The archive can only be read once - as soon as reading completes, the file will be deleted.

Deprecated: NewTempArchive is only used in tests and will be removed in the next release.

func (*TempArchive) Close added in v1.4.0 

func (archive *TempArchive) Close() error

Close closes the underlying file if it's still open, or does a no-op to allow callers to try to close the TempArchive multiple times safely.

func (*TempArchive) Read 

func (archive *TempArchive) Read(data []byte) (int, error)

type WhiteoutFormat added in v1.12.0 

type WhiteoutFormat int

WhiteoutFormat is the format of whiteouts unpacked 

const (
	AUFSWhiteoutFormat    WhiteoutFormat = 0 // AUFSWhiteoutFormat is the default format for whiteouts
	OverlayWhiteoutFormat WhiteoutFormat = 1 // OverlayWhiteoutFormat formats whiteout according to the overlay standard.
)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.