caryatid

README

Caryatid

An Atlas is "a support sculpted in the form of a man"; a Caryatid is such a support in the form of a woman.

Caryatid is a packer post-processor that can generate or update Vagrant catalogs on local storage. Vagrant will read versioning information from these catalogs and detect when there is a new version of the box available, which is not possible when just doing vagrant box add.

In the future, it will support remote catalogs, like scp, as well.

Prerequisites

• Go
• Packer
• Disk space to keep (large) Vagrant box files

Caryatid is intended to work on any platform that Packer supports, but gets somewhat less testing on Windows. If you find something that's broken, please open an issue.

Installing

• Build with go build
• Copy to ~/.packer.d/plugins or %APPDATA%\packer.d\plugins
• See also the official plugins documentation
• To build a release, run go build buildrelease.go; this will result in a release directory that contains executables for every supported architecture

Using

In your packerfile, you must add it as a post-processor in a series, and coming after a vagrant post-processor (because caryatid requires Vagrant boxes to come in as artifacts).

There are five configuration parameters:

• name (required): The name of the box.
• description (required): A longer description for the box
• version (required): The version of the box
  • Sometimes, it makes sense to set this based on the date; setting the version to "1.0.{{isotime \"20060102150405\"}}" will result in a version number of 1.0.YYYYMMDDhhmmss
  • This can be especially useful during development, so that you don't have to pass an ever-incrementing version number variable to packer build
  • See the isotime global function in the packer documentation for configuration templates for more information
• catalog_root_url (required): A file:// URL for the directory containing the catalog
  • If there is a catalog at /srv/vagrant/boxname.json, this should be set to file:///srv/vagrant
  • Caryatid assumes the catalog name is always just <box name>.json
  • This is designed so that you might have many catalogs that share the same catalog_root_url; as long as the box names are different, they won't step on each other
  • See the "Output and directory structure" section for more information
• keep_input_artifact (optional): Keep a copy of the Vagrant box at whatever location the Vagrant post-processor stored its output
  • By default, input artifacts are deleted; this suppresses that behavior, and will result in two copies of the Vagrant box on your filesystem - one where the Vagrant post-processor was configured to store its output, and one where Caryatid will copy it

That might look like this:

"variables": {
  "boxname": "wintriallab-win10-32",
  "version": "1.0.{{isotime \"20060102150405\"}}",
  "description": "Windows Trial Lab: Windows 10 x86",
  "catalog_root_url": "file://{{env `HOME`}}"
},
...<snip>...
"post-processors": [
  [
    { "type": "vagrant", },
    {
      "type": "caryatid",
      "name": "{{user `boxname`}}",
      "version": "{{user `version`}}",
      "description": "{{user `description`}}",
      "catalog_root_url": "{{user `catalog_root_url`}}"
    }
  ]
]

Note: post-processor series

See the double open square brackets ([) after "post-processors":? The first square bracket indicates the start of the post-processors section; the second indicates the start of a post-processor sequence, where artifacts from the previous post-processor are fed as input into the next.

If you don't define a sequence using that extra set of square brackets, but instead just place the vagrant and caryatid entries in the post-processors section directly, the vagrant post-processor will run with inputs from the builder, and then caryatid post-processor will run afterwards also with inputs from the builder, rather than with inputs from the vagrant post-processor.

See the official post-processor documentation for more details on sequences.

Note: Filesystem permissions

On Unix, files created by Caryatid honor the user's umask. If you intend to share these boxes with other users on your system, make sure to set a umask that lets those users read your files.

TODO: Investigate default permissions on Windows.

Permissions of existing files that Caryatid updates (like an existing catalog) are not changed.

Output and directory structure

Using a destination of /srv/vagrant, a box name of testbox, and trying to add a Virtualbox edition of that box at version 1.0.0 would result in a directory structure like this:

/srv/vagrant
    /testbox.json: the JSON catalog
    /testbox
        /testbox_1.0.0_virtualbox.box: the large VM box file itself

And the testbox.json catalog will look like this:

{
    "name": "testbox",
    "description": "a box for testing",
    "versions": [{
        "version": "1.0.0",
        "providers": [{
            "name": "virtualbox",
            "url": "file:///srv/vagrant/testbox/testbox_1.0.0.box",
            "checksum_type": "sha1",
            "checksum": "d3597dccfdc6953d0a6eff4a9e1903f44f72ab94"
        }]
    }]
}

This can be consumed in a Vagrant file by using the JSON catalog as the box URL in a Vagrantfile:

config.vm.box_url = "file:///srv/vagrant/testbox.json"

Roadmap / wishlist

• Add scp support. Vagrant is supposed to support scp, but apparently doesn't bundle a properly-built curl yet. This means you may need to build your own curl that supports scp, and possibly even replace your system-supplied curl with that one, in order to use catalogs hosted on scp with Vagrant. (Note that Caryatid will not rely on curl, so even if your curl is old, we will still be able to push to scp backends; the only concern is whether your system's Vagrant can pull from them by default or not.)
• Add support for S3 storage. Unfortunately, there isn't a way to authenticate to S3 through Vagrant as far as I can tell, so boxes stored on S3 would have to be public.
• Some sort of webserver mode would be nice, and is in line with the no server-side logic goal. Probably require an scp url for doing uploads in addition to an http url for vagrant to fetch the boxes? It does look like Vagrant supports HTTP basic authentication, so downloads for HTTP/HTTPS catalogs could be protected.
• WebDAV is a possibility, but I'm not sure whether it would be truly valuable or not - I don't see a lot of WebDAV servers out in the wild. On the other hand, it would be conceptually simpler than a webserver mode that requires an scp upload mechanism to supplement it, and it would support HTTP basic auth as well.