archive

package module
v0.0.0-...-7149510 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 12, 2020 License: MIT Imports: 29 Imported by: 0

README

go-archive

go-archive are a set of non-production test bindings to work with a Debian archive.

Documentation

Overview

This module is experimental and incomplete. Please be careful.

The archive module provides a few `pault.ag/go/debian` compatable bindings to read and write Debian apt archives.

Index

Constants

View Source 
const DebianArchiveKeyring = "/usr/share/keyrings/debian-archive-keyring.gpg"

DebianArchiveKeyring is the full path to the GPG keyring containing the public keys used for signing the Debian archive.

Variables

View Source 
var DefaultDownloader = &Downloader{
	Parallel:            10,
	MaxTransientRetries: 3,
	Mirror:              "https://deb.debian.org/debian",
}

DefaultDownloader is a ready-to-use Downloader, used by convenience wrappers such as CachedRelease and, by extension, TempFile.

Functions

func CachedRelease 

func CachedRelease(suite string) (*Release, *ReleaseDownloader, error)

CachedRelease returns DefaultDownloader.Release(suite), caching releases for the duration of the process.

func TempFile 

func TempFile(path string) (*os.File, error)

TempFile expects a path starting with dists/<suite>, calls Release(suite), looks up the remaining path within the release and calls TempFile on the corresponding ReleaseDownloader.

Types

type Archive 

type Archive struct {
	Store blobstore.Store

	Pool Pool
	// contains filtered or unexported fields
}

Core Archive abstrcation. This contains helpers to write out package files, as well as handles creating underlying abstractions (such as Suites).

func New 

func New(path string, signer *openpgp.Entity) (*Archive, error)

Create a new Archive at the given `root` on the filesystem, with the openpgp.Entity `signer` (an Entity which contains an OpenPGP Private Key).

This interface is intended to *write* Archives, not *read* them. Extra steps must be taken to load an Archive over the network, and attention must be paid when handling the Cryptographic chain of trust.

func (Archive) Engross 

func (a Archive) Engross(suite Suite) (ArchiveState, error)

Engross a Suite for signing and final commit into the blobstore. This will return handle(s) to the signed and ready Objects, fit for passage to Link.

This will contain all the related Packages and Release files.

func (Archive) GC 

func (a Archive) GC() error

Use the default backend to remove any unlinked files from the Blob store.

If files you care about are not linked onto the stage, they will be removed by the garbage collector. GC only when you're sure the stage has been set. 

func (a Archive) Link(blobs ArchiveState) error

Given a list of objects, link them to the keyed paths.

func (Archive) Path 

func (a Archive) Path() string

func (Archive) Suite 

func (a Archive) Suite(name string) (*Suite, error)

Get a handle to write a given Suite from an Archive. The suite will be entirely blank, and attributes will not be read from the existing files, if any.

type ArchiveState 

type ArchiveState map[string]blobstore.Object

This is a set of file changes ready to be passed to `Link` to link in. Basically, this maps file paths to blobstore objects, which will be swapped in all at once. This allows errors to avoid mutating state in the archive.

type Component 

type Component struct {
	// contains filtered or unexported fields
}

Small wrapper to represent a Component of a Suite, which, at its core is simply a set of Indexes to be written to.

This contains no state read off disk, and is purely for writing to.

func (*Component) AddPackage 

func (c *Component) AddPackage(pkg Package) error

Add a given Package to a Package List. Under the hood, this will get or create a IndexWriter, and invoke the .Add method on the Package Writer.

type Downloader 

type Downloader struct {
	// Parallel limits the maximum number of concurrent archive accesses.
	Parallel int

	// MaxTransientRetries caps retries of transient errors.
	// The default value of 0 means retry forever.
	MaxTransientRetries int

	// Mirror is the HTTP URL of a Debian mirror, e.g. "https://deb.debian.org/debian".
	// Mirror supports TLS and HTTP/2.
	Mirror string

	// LocalMirror overrides Mirror with a local file system path.
	// E.g. /srv/mirrors/debian on DSA-maintained machines.
	LocalMirror string

	// TempDir is passed as dir argument to ioutil.TempFile.
	// The default value of empty string uses the default directory, see os.TempDir.
	TempDir string

	// Keyring is used for validating archive GPG signatures. If nil, the
	// keyring is loaded from DebianArchiveKeyring.
	Keyring openpgp.EntityList
	// contains filtered or unexported fields
}

Downloader makes files from the Debian archive available.

func (*Downloader) Release 

func (g *Downloader) Release(suite string) (*Release, *ReleaseDownloader, error)

Release returns a release and a corresponding ReleaseDownloader from the archive.

If cryptographic verification using DebianArchiveKeyring fails, an error will be returned.

func (*Downloader) TempFile 

func (g *Downloader) TempFile(fh control.FileHash) (*os.File, error)

TempFile calls ioutil.TempFile, then downloads fh from the archive and returns it.

If hash checksum verification fails, the temporary file will be deleted and an error will be returned.

If err is nil, the caller must remove the file when no longer needed: 

f, err := r.GetTempFile(fh)
if err != nil {
    return nil, err
}
defer f.Close() // avoid leaking resources
defer os.Remove(f.Name()) // remove from file system
return parseSources(f)

Remember that files must be closed before they can be read by external processes: 

f, err := r.GetTempFile(fh)
if err != nil {
    return err
}
if err := f.Close(); err != nil {
    return err
}
defer os.Remove(f.Name()) // remove from file system
return exec.Command("tar", "xf", f.Name()).Run()

type IndexWriter 

type IndexWriter struct {
	// contains filtered or unexported fields
}

This writer represents a Package list - which is to say, a list of binary .deb files, for a particular Architecture, in a particular Component in a particular Suite, in a particular Archive.

This is not an encapsulation to store the entire Index in memory, rather, it's a wrapper to help write Package entries into the Index.

func (IndexWriter) Add 

func (p IndexWriter) Add(data interface{}) error

Write a Package entry into the Packages index.

type Package 

type Package struct {
	control.Paragraph

	Package       string `required:"true"`
	Source        SourceName
	Version       version.Version `required:"true"`
	Section       string
	Priority      string
	Architecture  dependency.Arch `required:"true"`
	Essential     string
	InstalledSize int    `control:"Installed-Size"`
	Maintainer    string `required:"true"`
	Description   string `required:"true"`
	Homepage      string

	Filename       string `required:"true"`
	Size           int    `required:"true"`
	MD5sum         string
	SHA1           string
	SHA256         string
	SHA512         string
	DescriptionMD5 string `control:"Description-md5"`

	Depends    dependency.Dependency
	Suggests   dependency.Dependency
	BuiltUsing dependency.Dependency `control:"Built-Using"`
	Breaks     dependency.Dependency
	PreDepends dependency.Dependency `control:"Pre-Depends"`
}

Binary .deb Package entry, as it exists in the Packages file, which contains the .deb Control information, as well as information on where the file lives, the file size, and some hashes.

func PackageFromDeb 

func PackageFromDeb(debFile deb.Deb) (*Package, error)

Create a Package entry from a deb.Deb file. This will copy the binary .deb Control file into the Package entry, and set information as to the location of the file, the size of the file, and hash the file.

func SortPackages 

func SortPackages(packages []Package) []Package

type PackageMap 

type PackageMap map[string][]Package

func LoadPackageMap 

func LoadPackageMap(binaries Packages) (*PackageMap, error)

type Packages 

type Packages struct {
	// contains filtered or unexported fields
}

Iterator to access the entries contained in the Packages entry in an apt repo. This contians information about the binary Debian packages.

func LoadPackages 

func LoadPackages(in io.Reader) (*Packages, error)

Given an io.Reader, create a Packages iterator. Note that the Packages file is not OpenPGP signed, so one will need to verify the integrety of this file from the InRelease file before trusting any output.

func LoadPackagesFile 

func LoadPackagesFile(path string) (*Packages, error)

Given a path, create a Packages iterator. Note that the Packages file is not OpenPGP signed, so one will need to verify the integrety of this file from the InRelease file before trusting any output.

func (*Packages) Map 

func (p *Packages) Map(q func(*Package) bool) ([]Package, error)

Get any packages that match the criteria

func (*Packages) Next 

func (p *Packages) Next() (*Package, error)

Get the next Package entry in the Packages list. This will return an io.EOF at the last entry.

type Pool 

type Pool struct {
	Store blobstore.Store
}

func (Pool) Copy 

func (p Pool) Copy(path string) (*blobstore.Object, error)

func (Pool) IncludeDeb 

func (p Pool) IncludeDeb(debFile *deb.Deb) (string, *blobstore.Object, error)

func (Pool) IncludeSources 

func (p Pool) IncludeSources(dsc *control.DSC) (string, map[string]blobstore.Object, error)

type Release 

type Release struct {
	control.Paragraph

	Description string

	// Optional field indicating the origin of the repository, a single line
	// of free form text.
	Origin string

	// Optional field including some kind of label, a single line of free form
	// text.
	//
	// Typically used extensively in repositories split over multiple media
	// such as repositories stored on CDs.
	Label string

	// The Version field, if specified, shall be the version of the release.
	// This is usually a sequence of integers separated by the character
	// "." (full stop).
	//
	// Example:
	//
	//   Version: 6.0
	Version string

	// The Suite field may describe the suite. A suite is a single word. In
	// Debian, this shall be one of oldstable, stable, testing, unstable,
	// or experimental; with optional suffixes such as -updates.
	//
	// Example:
	// //   Suite: stable
	Suite string

	// The Codename field shall describe the codename of the release. A
	// codename is a single word. Debian releases are codenamed after Toy
	// Story Characters, and the unstable suite has the codename sid, the
	// experimental suite has the codename experimental.
	//
	// Example:
	//
	//   Codename: squeeze
	Codename string

	// A whitespace separated list of areas.
	//
	// Example:
	//
	//   Components: main contrib non-free
	//
	// May also include be prefixed by parts of the path following the
	// directory beneath dists, if the Release file is not in a directory
	// directly beneath dists/. As an example, security updates are specified
	// in APT as:
	//
	// deb http://security.debian.org/debian-security stable-security main)
	//
	// The Release file would be located at
	// http://security.debian.org/dists/stable-security/Release and look like:
	//
	//   Suite: stable-security
	//   Components: main contrib non-free
	Components []string `delim:" "`

	// Whitespace separated unique single words identifying Debian machine
	// architectures as described in Architecture specification strings,
	// Section 11.1. Clients should ignore Architectures they do not know
	// about.
	Architectures []dependency.Arch

	// The Date field shall specify the time at which the Release file was
	// created. Clients updating a local on-disk cache should ignore a Release
	// file with an earlier date than the date in the already stored Release
	// file.
	//
	// The Valid-Until field may specify at which time the Release file should
	// be considered expired by the client. Client behaviour on expired Release
	// files is unspecified.
	//
	// The format of the dates is the same as for the Date field in .changes
	// files; and as used in debian/changelog files, and documented in Policy
	// 4.4 ( Debian changelog: debian/changelog)
	Date       string
	ValidUntil string `control:"Valid-Until"`

	// note the upper-case S in MD5Sum (unlike in Packages and Sources files)
	//
	// These fields are used for two purposes:
	//
	// describe what package index files are present when release signature is
	// available it certifies that listed index files and files referenced by
	// those index files are genuine Those fields shall be multi-line fields
	// containing multiple lines of whitespace separated data. Each line shall
	// contain
	//
	// The checksum of the file in the format corresponding to the field The
	// size of the file (integer >= 0) The filename relative to the directory
	// of the Release file Each datum must be separated by one or more
	// whitespace characters.
	//
	// Server requirements:
	//
	// The checksum and sizes shall match the actual existing files. If indexes
	// are compressed, checksum data must be provided for uncompressed files as
	// well, even if not present on the server.  Client behaviour:
	//
	// Any file should be checked at least once, either in compressed or
	// uncompressed form, depending on which data is available. If a file has
	// no associated data, the client shall inform the user about this under
	// possibly dangerous situations (such as installing a package from that
	// repository). If a file does not match the data specified in the release
	// file, the client shall not use any information from that file, inform
	// the user, and might use old information (such as the previous locally
	// kept information) instead.
	MD5Sum []control.MD5FileHash    `delim:"\n" strip:" \t\n\r" multiline:"true"`
	SHA1   []control.SHA1FileHash   `delim:"\n" strip:" \t\n\r" multiline:"true"`
	SHA256 []control.SHA256FileHash `delim:"\n" strip:" \t\n\r" multiline:"true"`
	SHA512 []control.SHA512FileHash `delim:"\n" strip:" \t\n\r" multiline:"true"`

	// The NotAutomatic and ButAutomaticUpgrades fields are optional boolean
	// fields instructing the package manager. They may contain the values
	// "yes" and "no". If one the fields is not specified, this has the same
	// meaning as a value of "no".
	//
	// If a value of "yes" is specified for the NotAutomatic field, a package
	// manager should not install packages (or upgrade to newer versions) from
	// this repository without explicit user consent (APT assigns priority 1 to
	// this) If the field ButAutomaticUpgrades is specified as well and has the
	// value "yes", the package manager should automatically install package
	// upgrades from this repository, if the installed version of the package
	// is higher than the version of the package in other sources (APT assigns
	// priority 100).
	//
	// Specifying "yes" for ButAutomaticUpgrades without specifying "yes" for
	// NotAutomatic is invalid.
	NotAutomatic         string
	ButAutomaticUpgrades string

	AcquireByHash bool `control:"Acquire-By-Hash"`
}

The file "dists/$DIST/InRelease" shall contain meta-information about the distribution and checksums for the indices, possibly signed with a GPG clearsign signature (for example created by "gpg -a -s --clearsign"). For older clients there can also be a "dists/$DIST/Release" file without any signature and the file "dists/$DIST/Release.gpg" with a detached GPG signature of the "Release" file, compatible with the format used by the GPG options "-a -b -s".

func LoadInRelease 

func LoadInRelease(in io.Reader, keyring *openpgp.EntityList) (*Release, error)

Given an InRelease io.Reader, and the OpenPGP keyring to validate against, return the parsed InRelease file.

func LoadInReleaseFile 

func LoadInReleaseFile(path string, keyring *openpgp.EntityList) (*Release, error)

Given a path to the InRelease file on the filesystem, and the OpenPGP keyring to validate against, return the parsed InRelease file.

func (*Release) AddHash 

func (r *Release) AddHash(h control.FileHash) error

func (*Release) Indices 

func (r *Release) Indices() map[string]control.FileHashes

Given a file declared in the Release file, get the FileHash entries for that file (SHA256, SHA512). These can be used to ensure the integrety of files in the archive.

type ReleaseDownloader 

type ReleaseDownloader struct {
	// LastModified contains the last modification timestamp of the release
	// metadata file.
	LastModified time.Time
	// contains filtered or unexported fields
}

ReleaseDownloader is like Downloader, but for a specific release (e.g. unstable).

func (*ReleaseDownloader) TempFile 

func (r *ReleaseDownloader) TempFile(fh control.FileHash) (*os.File, error)

GetTempFile is like Downloader.GetTempFile, but for fhs of the release.

type Source 

type Source struct {
	control.Paragraph

	Package string

	Directory string `required:"true"`
	Priority  string
	Section   string

	Format           string
	Binaries         []string          `control:"Binary" delim:","`
	Architectures    []dependency.Arch `control:"Architecture"`
	Version          version.Version
	Origin           string
	Maintainer       string
	Uploaders        []string
	Homepage         string
	StandardsVersion string   `control:"Standards-Version"`
	PackageList      []string `control:"Package-List" delim:"\n" strip:" \t\n\r" multiline:"true"`

	ChecksumsSha1   []control.SHA1FileHash   `control:"Checksums-Sha1" delim:"\n" strip:" \t\n\r" multiline:"true"`
	ChecksumsSha256 []control.SHA256FileHash `control:"Checksums-Sha256" delim:"\n" strip:" \t\n\r" multiline:"true"`
	Files           []control.MD5FileHash    `delim:"\n" strip:" \t\n\r" multiline:"true"`
}

The files dists/$DIST/$COMP/source/Sources are called Sources indices. They consist of multiple paragraphs, where each paragraph has the format defined in Policy 5.5 (5.4 Debian source control files -- .dsc), with the following changes and additional fields. The changes are:

  • The "Source" field is renamed to "Package"
  • A new mandatory field "Directory"
  • A new optional field "Priority"
  • A new optional field "Section"
  • (Note that any fields present in .dsc files can end here as well, even if
  • they are not documented by Debian policy, or not yet documented yet).

Each paragraph shall begin with a "Package" field. Clients may also accept files where this is not the case.

func SortSources 

func SortSources(sources []Source) []Source

func SourceFromDsc 

func SourceFromDsc(dsc *control.DSC, directory string) (*Source, error)

func (Source) BuildDepends 

func (s Source) BuildDepends() (*dependency.Dependency, error)

type SourceMap 

type SourceMap map[string][]Source

func LoadSourceMap 

func LoadSourceMap(sources Sources) (*SourceMap, error)

func (SourceMap) Matches 

func (s SourceMap) Matches(possi dependency.Possibility) (int, error)

type SourceName 

type SourceName struct {
	Name    string
	Version version.Version
}

func (*SourceName) MarshalControl 

func (sn *SourceName) MarshalControl() (string, error)

func (*SourceName) UnmarshalControl 

func (sn *SourceName) UnmarshalControl(data string) error

type Sources 

type Sources struct {
	// contains filtered or unexported fields
}

func LoadSources 

func LoadSources(in io.Reader) (*Sources, error)

Given an io.Reader, create a Sources iterator. Note that the Sources file is not OpenPGP signed, so one will need to verify the integrety of this file from the InRelease file before trusting any output.

func LoadSourcesFile 

func LoadSourcesFile(path string) (*Sources, error)

Given a path, create a Sources iterator. Note that the Sources file is not OpenPGP signed, so one will need to verify the integrety of this file from the InRelease file before trusting any output.

func (*Sources) Next 

func (p *Sources) Next() (*Source, error)

Get the next Source entry in the Sources list. This will return an io.EOF at the last entry.

type Suite 

type Suite struct {
	control.Paragraph

	Name        string `control:"Suite"`
	Description string
	Origin      string
	Label       string
	Version     string
	// contains filtered or unexported fields
}

Abstraction to handle writing data into a Suite. This is a write-only target, and is not intended to read a Release file.

This contains no state read off disk, and is purely for writing to.

func (Suite) Component 

func (s Suite) Component(name string) (*Component, error)

Get or create a Component for a given Suite. If no such Component has been created so far, this will create a new object, otherwise it will return the existing entry.

This contains no state read off disk, and is purely for writing to.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.