pm

README

pm: a simple, cross-platform system package manager

pm exists amid a set of trade-offs in distributing software. The ideas behind pm were born at a time when:

• There was no overlap in the Venn diagram of system package managers that offered both strong security promises (signed packages) and permissive licensing (most are GPL).
• There was reason to suspect that Unix systems might be shipped without scripting languages; software like brew would cease to work and engineers would be left without a way to fetch and install software.
• Engineers wanted to deploy software to a variety of Unix-like environments using a single system.
• Engineers wanted a simple-to-reason-about system that used familiar Unix primitives as building blocks to distribute their software.

Simplicity is a principal design goal of this project. When offered an opportunity to chose between two designs the design that requires less mental scaffolding to describe or implement should be used. As a concrete example: transitive dependency calculations are implemented, but supporting compatible version ranges are not.

The project is currently in early design phases, and this document describes the high-level approach of the project.

Components

There are two main components to this project.

1. pm is the name of the client-side cli command. This is the tool used to fetch, install, verify, create, upload, etc. packages.
2. pmd is the name of the server-side component. It hosts packages (over http for now), available package metadata, and cryptographic public key information to clients.

Securely installing the pm command is important. Be sure to verify its contents before use.

Package Format

The intention is to be able to create and open package files with commonly used Unix utilities. The package file is an uncompressed tar file contaning the following files:

1. meta.yaml -- contains information about the package's contents, and is transmitted to clients during for which available packages a remote can serve, e.g.:

name: foo
version: 2.3.29
description: Foo is the world's simplest frobnicator
deps: [baz, bar@0.9.2]

1. root.tar.bz2 -- A compressed tarball that will eventually be expanded starting at $PM_ROOT
2. bom.sha256 -- checksum file containing sha256 checksums of the expected contents of root.tar.bz2
3. manifest.sha256 -- checksum file of the expected contents of the .pkg file.
4. manifest.sha256.asc -- OpenPGP detached signature for the manifest.sha256 file. Its validity communicates that the contents have not been tampered with.
5. bin/{pre,post}-{install,ugrade,remove} (optional) -- a collection of executables that are run at the relevant stages.

As a minimum package authors are required to author the root.tar.bz2 and the meta.yaml files, and the pm pkg create will generate the rest of the files, using the key information associated with the PM_PGP_EMAIL environment variable. If you can make a tar file and write a yaml file, you can create a pmpackage!

Remote Repositories

The notion of remote is borrowed from git; a pm client can be configured to pull packages from multiple remote repositories. It is intended to be trivial to deploy pmd, and equally trivial to configure clients to fetch from multiple remotes.

The example remote url:

https://pm.mcquay.me/darwin/amd64/testing

encodes a remote that is served over https on the host pm.mcquay.me and informs the client to pull packages from the /darwin/amd64/testing namespace, specified by the Path. pm pull will collect available package information from configured remote and will populate its local database with the contents of the response. pm can then list available packages, and the user can then request that they be installed.

As a practical example a client can be configured to pull from two remotes as:

$ pm add remote https://pm.mcquay.me/darwin/amd64/stable
$ pm add remote https://pm.example.com/generic/testing
$ pm pull
$ pm available
foo     0.1.2      https://pm.mcquay.me/darwin/amd64/stable
bar     3.2.3      https://pm.mcquay.me/generic/testing

Here each remote advertises one package each. After pulling metadata from the remote server the client database is populated, and the user listed all installable packages. In the case of collisions the first configured remote offering a colliding packages will be the used.

Previous versions of pm use to implicitly formulate namespace values based on host information (os and arch), but allowing package maintainers and end users to specify this value explicitly allows for greater flexibility.

Documentation

Index

• func ParseCS(f io.Reader) (map[string]string, error)
• type Available
  • func (a Available) Add(m Meta) error
  • func (a Available) Get(n Name, v Version) (Meta, error)
  • func (a Available) Installable(in []string) (Metas, error)
  • func (a Available) SetRemote(u url.URL)
  • func (a Available) Traverse() <-chan Meta
  • func (a Available) Update(o Available) error
• type Installed
  • func (i Installed) Removable(names []string) (Metas, error)
  • func (i Installed) Traverse() <-chan Meta
• type Meta
  • func (m Meta) Pkg() string
  • func (m Meta) String() string
  • func (m Meta) URL() string
  • func (m Meta) Valid() (bool, error)
• type Metas
• type Name
• type Names
  • func (n Names) Len() int
  • func (n Names) Less(a, b int) bool
  • func (n Names) Swap(a, b int)
• type Version
• type Versions
  • func (v Versions) Len() int
  • func (v Versions) Less(a, b int) bool
  • func (v Versions) Swap(a, b int)

Functions

func ParseCS

func ParseCS(f io.Reader) (map[string]string, error)

ParseCS returns a parsed checksum file.

Types

type Available

type Available map[Name]map[Version]Meta

Available is the structure used to represent the collection of all packages that can be installed.

func (Available) Add

func (a Available) Add(m Meta) error

Add inserts m into a.

func (Available) Get

func (a Available) Get(n Name, v Version) (Meta, error)

Get returns the meta stored at a[n][v] or an error explaining why it could not be Get.

func (Available) Installable

func (a Available) Installable(in []string) (Metas, error)

Installable calculates if the packages requested in "in" can be installed.

func (Available) SetRemote

func (a Available) SetRemote(u url.URL)

SetRemote adds the information in the url to the database.

func (Available) Traverse

func (a Available) Traverse() <-chan Meta

Traverse returns a chan of Meta that will be sanely sorted.

func (Available) Update

func (a Available) Update(o Available) error

Update inserts all data from o into a.

type Installed

type Installed map[Name]Meta

Installed tracks installed packages.

func (Installed) Removable

func (i Installed) Removable(names []string) (Metas, error)

Removable calculates if the packages requested in "in" can all be removed.

func (Installed) Traverse

func (i Installed) Traverse() <-chan Meta

Traverse returns a chan of Meta that will be sanely sorted.

type Meta

type Meta struct {
	Name        Name    `json:"name"`
	Version     Version `json:"version"`
	Description string  `json:"description"`

	Remote url.URL `json:"remote"`
}

Meta tracks metadata for a package

func (Meta) Pkg

func (m Meta) Pkg() string

Pkg returns the string name the .pkg should have on disk.

func (Meta) String

func (m Meta) String() string

func (Meta) URL

func (m Meta) URL() string

URL returns the http location of this package.

func (Meta) Valid

func (m Meta) Valid() (bool, error)

Valid validates the contents of a Meta for requires fields.

type Metas

type Metas []Meta

Metas is a slice of Meta

type Name

type Name string

Name exists to document the keys in Available

type Names

type Names []Name

Names is a slice of names ... with sorting!

func (Names) Len

func (n Names) Len() int

func (Names) Less

func (n Names) Less(a, b int) bool

func (Names) Swap

func (n Names) Swap(a, b int)

type Version

type Version string

Version exists to document the keys in Available

type Versions

type Versions []Version

Versions is a slice of Version ... with sorting!

func (Versions) Len

func (v Versions) Len() int

TODO (sm): make this semver sort?

func (Versions) Less

func (v Versions) Less(a, b int) bool

func (Versions) Swap

func (v Versions) Swap(a, b int)