sti

README

docker-source-to-images (sti)

Source-to-images (sti) is a tool for building reproducable Docker images. sti produces ready-to-run images by injecting a user source into a docker image and assembling a new Docker image which incorporates the base image and built source, and is ready to use with docker run. sti supports incremental builds which re-use previously downloaded dependencies, previously built artifacts, etc

Interested in learning more? Read on!

Want to just get started now? Check out the instructions.

Philosophy

1. Simplify the process of application source + base image -> usable image for most use cases (the 80%)
2. Define and implement a workflow for incremental build that eventually uses only docker primitives
3. Develop tooling that can assist in verifying that two different base images result in the same "docker run" outcome for the same input
4. Use native docker primitives to accomplish this - map out useful improvements to docker that benefit all image builders

Anatomy of a source image

Building source images is easy. sti expects you to supply the following scripts to use with an image:

1. assemble : This script builds and/or deploys the source
2. run: This script runs the deployed source
3. save-artifacts (optional): This script saves the build context for an incremental build

Build methodologies

sti builds images with docker run. The workflow is:

1. sti creates a container based on the build image. with:
   1. The application source bind-mounted to /tmp/src
   2. The build artifacts bind-mounted to /tmp/artifacts (if applicable - see incremental builds)
   3. Runs the build image's assemble script
2. sti starts the container and waits for it to finish running
3. sti commits the container, setting the CMD for the output image to be the run script and tagging the image with the name provided.

Basic (--clean) builds

sti accepts the following inputs to do a build:

1. Application source: this can be source code, zipped source, a binary, etc
2. Build image: the basis for the new image to build
3. Application image tag: the tag to use for the newly created application image

The basic build process is as follows:

1. sti pulls the build image if it is not already present on the system
2. sti builds the new image from the supplied build image and source, tagging the output image with the supplied tag

Incremental builds

sti automatically detects:

1. Whether a source image is compatible with incremental building
2. Whether an incremental build can be formed when an image is compatible

If a save-artifacts script exists, a prior build already exists, and the --clean option is not used, the workflow is as follows:

1. sti creates a new docker container from the prior build image, with a volume in /tmp/artifacts
2. sti runs save-artifacts in this container - this script is responsible for copying the build artifacts into /tmp/artifacts.
3. sti builds the new output image using the selected build methodology:
   1. The artifacts from the previous build will be in /tmp/artifacts during the build
   2. The build image's assemble script is responsible for detecting and using the build artifacts

Note the invocation of the save-artifacts script; this script is responsible for moving build dependencies to /tmp/artifacts

Dependencies

1. Docker
2. Go
3. Mercurial
4. Rpms: libselinux libselinux-devel libselinux-utils

Installation

Assuming docker are installed and configured, execute the following commands:

go get github.com/openshift/geard
src/github.com/openshift/geard/contrib/build -s

Getting Started

You can start using sti right away with the following test sources and publicly available images:

sti build git://github.com/pmorie/simple-ruby openshift/ruby-19-centos test-ruby-app
docker run -rm -i -p :9292 -t test-ruby-app

sti build git://github.com/pmorie/simple-ruby openshift/buildpack-ubuntu test-foreman-app \
-e 'BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-ruby.git'
docker run -rm -i -p :5000 -t test-foreman-app

sti build git://github.com/pmorie/simple-html pmorie/fedora-mock test-html-app
docker run -rm -i -p :8080 -t sti_app

Building a Deployable Image

sti build SOURCE BUILD_IMAGE APP_IMAGE_TAG [flags]

Available Flags:
     --callbackUrl="": Specify a URL to invoke via HTTP POST upon build completion
     --clean=false: Perform a clean build
 -e, --env="": Specify an environment var NAME=VALUE,NAME2=VALUE2,...
 -r, --ref="": Specify a ref to check-out
     --rm=false: Remove the previous image during incremental builds
     --savetempdir=false: Save the temporary directory used by STI instead of deleting it
 -s, --scripts="": Specify a URL for the assemble and run scripts
 -U, --url="unix:///var/run/docker.sock": Set the url of the docker socket to use
     --verbose=false: Enable verbose output

The most basic sti build uses a single build image:

sti build SOURCE BUILD_IMAGE_TAG APP_IMAGE_TAG

If the build is successful, the built image will be tagged with APP_IMAGE_TAG.

If the build image is compatible with incremental builds, sti build will look for an image tagged with APP_IMAGE_TAG. If an image is present with that tag, and a save-artifacts script is present, sti build will save the build artifacts from that image and add them to the build container at /tmp/artifacts so the assemble script can restore them before building the source.

When using an image that supports incremental builds, you can do a clean build with --clean:

sti build SOURCE BUILD_IMAGE_TAG APP_IMAGE_TAG --clean

Using scripts from a URL

You don't need a special image to use with sti. You can use any set of assemble/run/ save-artifacts scripts with any image you want by specifying a URL:

sti build SOURCE BUILD_IMAGE_TAG APP_IMAGE_TAG -s <url>

If you're creating an image and you want to supply a default set of scripts to use with sti, you can specify the STI_SCRIPTS_URL environment variable in your Dockerfile:

ENV STI_SCRIPTS_URL <url>

Using scripts from an application

You can also supply assemble/run/save-artifacts scripts in your application source. The scripts must be located under .sti/bin within the root of your source directory.

Script precedence

STI selects which location to use for a given script (assemble, run, and save-artifacts) based on the following ordering:

1. A script found at the --scripts URL
2. A script found in the application source .sti/bin directory
3. A script found at the default image URL (STI_SCRIPTS_URL)

Build from a git ref

When the source is a git repo, sti can check out a git ref before doing the build:

sti build SOURCE BUILD_IMAGE_TAG APP_IMAGE_TAG -r <ref>

If the ref is invalid or not present in the source repo, the build will fail.

Build callbacks

Upon completion (or failure) of a build, sti can HTTP POST to a URL with information about the build:

sti build SOURCE BUILD_IMAGE_TAG APP_IMAGE_TAG --callbackUrl=<url>

The data posted will be of the form:

{
    "payload": "A string containing all build messages",
    "success": true
}

Documentation

Overview

Provides functionality to build docker images from user source and a base image.

Index

• Constants
• func FileExistsInContainer(dockerClient *docker.Client, cId string, path string) bool
• func Usage(req *STIRequest) error
• type STIRequest
• type STIResult
  • func Build(req *STIRequest) (result *STIResult, err error)
• type SchemeReader
• type StiError
  • func (s StiError) Error() string

Constants

const (
	SVirtSandboxFileLabel = "system_u:object_r:svirt_sandbox_file_t:s0"
	ContainerInitDirName  = ".sti.init"
	ContainerInitDirPath  = "/" + ContainerInitDirName
)

Functions

func FileExistsInContainer

func FileExistsInContainer(dockerClient *docker.Client, cId string, path string) bool

Determine whether a file exists in a container.

func Usage

func Usage(req *STIRequest) error

Usage processes a build request by starting the container and executing the assemble script with a "-h" argument to print usage information for the script.

Types

type STIRequest

type STIRequest struct {
	BaseImage           string
	DockerSocket        string
	DockerTimeout       int
	Verbose             bool
	PreserveWorkingDir  bool
	Source              string
	Ref                 string
	Tag                 string
	Clean               bool
	RemovePreviousImage bool
	Environment         map[string]string
	Writer              io.Writer
	CallbackUrl         string
	ScriptsUrl          string
	// contains filtered or unexported fields
}

STIRequest contains essential fields for any request: a Configuration, a base image, and an optional runtime image.

type STIResult

type STIResult struct {
	Success    bool
	Messages   []string
	WorkingDir string
	ImageID    string
}

func Build

func Build(req *STIRequest) (result *STIResult, err error)

Build processes a Request and returns a *Result and an error. An error represents a failure performing the build rather than a failure of the build itself. Callers should check the Success field of the result to determine whether a build succeeded or not.

type SchemeReader

type SchemeReader func(*url.URL) (io.ReadCloser, error)

SchemeReaders create an io.Reader from the given url.

type StiError

type StiError int

const (
	ErrDockerConnectionFailed StiError = iota
	ErrNoSuchBaseImage
	ErrNoSuchRuntimeImage
	ErrPullImageFailed
	ErrScriptsDownloadFailed
	ErrSaveArtifactsFailed
	ErrCreateDockerfileFailed
	ErrCreateContainerFailed
	ErrInvalidBuildMethod
	ErrBuildFailed
	ErrCommitContainerFailed
	ErrInvalidRef
)

func (StiError) Error

func (s StiError) Error() string