sourcetoimage

package module
v1.0.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 7, 2016 License: Apache-2.0 Imports: 0 Imported by: 0

README

source-to-image (s2i)

Go Report Card GoDoc Travis License

Source-to-image (s2i) is a tool for building reproducible Docker images. s2i produces ready-to-run images by injecting source code into a Docker image and assembling a new Docker image which incorporates the builder image and built source. The result is then ready to use with docker run. s2i supports incremental builds which re-use previously downloaded dependencies, previously built artifacts, etc.

Interested in learning more? Read on!

Want to just get started now? Check out the instructions.

Philosophy

  1. Simplify the process of application source + builder image -> usable image for most use cases (the 80%)
  2. Define and implement a workflow for incremental builds that eventually uses only Docker primitives
  3. Develop tooling that can assist in verifying that two different builder images result in the same docker run outcome for the same input
  4. Use native Docker primitives to accomplish this - map out useful improvements to Docker that benefit all image builders

Anatomy of a builder image

See a practical tutorial on how to create a builder image here

Creating builder images is easy. s2i looks for you to supply the following scripts to use with an image:

  1. assemble - builds and/or deploys the source
  2. run- runs the assembled artifacts
  3. save-artifacts (optional) - captures the artifacts from a previous build into the next incremental build
  4. usage (optional) - displays builder image usage information

Additionally for the best user experience and optimized s2i operation we suggest images to have /bin/sh and tar commands available.

Filtering the contents of the source tree is possible if the user supplies a .s2iignore file in the root directory of the source repository, where .s2iignore contains regular expressions that capture the set of files and directories you want filtered from the image s2i produces.

Specifically:

  1. Specify one rule per line, with each line terminating in \n.
  2. Filepaths are appended to the absolute path of the root of the source tree (either the local directory supplied, or the target destination of the clone of the remote source repository s2i creates).
  3. Wildcards and globbing (file name expansion) leverage Go's filepath.Match and filepath.Glob functions.
  4. Search is not recursive. Subdirectory paths must be specified (though wildcards and regular expressions can be used in the subdirectory specifications).
  5. If the first character is the # character, the line is treated as a comment.
  6. If the first character is the !, the rule is an exception rule, and can undo candidates selected for filtering by prior rules (but only prior rules).

Here are some examples to help illustrate:

With specifying subdirectories, the */temp* rule prevents the filtering of any files starting with temp that are in any subdirectory that is immediately (or one level) below the root directory. And the */*/temp* rule prevents the filtering of any files starting with temp that are in any subdirectory that is two levels below the root directory.

Next, to illustrate exception rules, first consider the following example snippet of a .s2iignore file:

*.md
!README.md

With this exception rule example, README.md will not be filtered, and remain in the image s2i produces. However, with this snippet:

!README.md
*.md

README.md, if filtered by any prior rules, but then put back in by !README.md, would be filtered, and not part of the resulting image s2i produces. Since *.md follows !README.md, *.md takes precedence.

Users can also set extra environment variables in the application source code. They are passed to the build, and the assemble script consumes them. All environment variables are also present in the output application image. These variables are defined in the .s2i/environment file inside the application sources. The format of this file is a simple key-value, for example:

FOO=bar

In this case, the value of FOO environment variable will be set to bar.

See here for a detailed description of the requirements and scripts along with examples of builder images.

Build workflow

The s2i build workflow is:

  1. s2i creates a container based on the build image and passes it a tar file that contains:
    1. The application source in src, excluding any files selected by .s2iignore
    2. The build artifacts in artifacts (if applicable - see incremental builds)
  2. s2i sets the environment variables from .s2i/environment (optional)
  3. s2i starts the container and runs its assemble script
  4. s2i waits for the container to finish
  5. s2i commits the container, setting the CMD for the output image to be the run script and tagging the image with the name provided.

Using ONBUILD images

In case you want to use one of the official Docker language stack images for your build you don't have do anything extra. S2I is capable of recognizing the Docker image with ONBUILD instructions and choosing the OnBuild strategy. This strategy will trigger all ONBUILD instructions and execute the assemble script (if it exists) as the last instruction.

Since the ONBUILD images usually don't provide any entrypoint, in order to use this build strategy you will have to provide one. You can either include the 'run', 'start' or 'execute' script in your application source root folder or you can specify a valid S2I script URL and the 'run' script will be fetched and set as an entrypoint in that case.

Incremental builds

s2i automatically detects:

  • Whether a builder image is compatible with incremental building
  • Whether a previous image exists, with the same name as the output name for this build

If a save-artifacts script exists, a prior image already exists, and the --incremental=true option is used, the workflow is as follows:

  1. s2i creates a new Docker container from the prior build image
  2. s2i runs save-artifacts in this container - this script is responsible for streaming out a tar of the artifacts to stdout
  3. s2i builds the new output image:
    1. The artifacts from the previous build will be in the artifacts directory of the tar passed to the build
    2. The build image's assemble script is responsible for detecting and using the build artifacts

NOTE: The save-artifacts script is responsible for streaming out dependencies in a tar file.

Dependencies

  1. Docker >= 1.6
  2. Go >= 1.4
  3. (optional) Git

Installation

Assuming Go and Docker are installed and configured, execute the following commands:

$ go get github.com/openshift/source-to-image
$ cd ${GOPATH}/src/github.com/openshift/source-to-image
$ export PATH=$PATH:${GOPATH}/src/github.com/openshift/source-to-image/_output/local/bin/linux/amd64/
$ hack/build-go.sh

Security

Since the s2i command uses the Docker client library, it has to run in the same security context as the docker command. For some systems, it is enough to add yourself into the 'docker' group to be able to work with Docker as 'non-root'. In the latest versions of Fedora/RHEL, it is recommended to use the sudo command as this way is more auditable and secure.

If you are using the sudo docker command already, then you will have to also use sudo s2i to give S2I permission to work with Docker directly.

Be aware that being a member of the 'docker' group effectively grants root access, as described here.

Getting Started

You can start using s2i right away (see releases) with the following test sources and publicly available images:

$ s2i build git://github.com/pmorie/simple-ruby openshift/ruby-20-centos7 test-ruby-app
$ docker run --rm -i -p :8080 -t test-ruby-app
$ s2i build git://github.com/bparees/openshift-jee-sample openshift/wildfly-8-centos test-jee-app
$ docker run --rm -i -p :8080 -t test-jee-app

Interested in more advanced s2i usage? See here for detailed descriptions of the different CLI commands with examples.

Running into some issues and need some advice debugging? Peruse here for some tips.

Documentation

Overview

Source-to-image (`sti`) is a tool for building reproducible Docker images. `sti` produces ready-to-run images by injecting source code into a docker image and <i>assembling</i> a new Docker image which incorporates the base image and built source, and is ready to use with `docker run`.

Directories

Path Synopsis
Godeps
_workspace/src/github.com/docker/docker/builder/command
This package contains the set of Dockerfile commands.
This package contains the set of Dockerfile commands.
_workspace/src/github.com/docker/docker/builder/parser
This package implements a parser and parse tree dumper for Dockerfiles.
This package implements a parser and parse tree dumper for Dockerfiles.
_workspace/src/github.com/docker/docker/pkg/pools
Package pools provides a collection of pools which provide various data types with buffers.
Package pools provides a collection of pools which provide various data types with buffers.
_workspace/src/github.com/fsouza/go-dockerclient
Package docker provides a client for the Docker remote API.
Package docker provides a client for the Docker remote API.
_workspace/src/github.com/fsouza/go-dockerclient/external/github.com/Sirupsen/logrus
Package logrus is a structured logger for Go, completely API compatible with the standard library logger.
Package logrus is a structured logger for Go, completely API compatible with the standard library logger.
_workspace/src/github.com/fsouza/go-dockerclient/external/github.com/docker/docker/pkg/pools
Package pools provides a collection of pools which provide various data types with buffers.
Package pools provides a collection of pools which provide various data types with buffers.
_workspace/src/github.com/fsouza/go-dockerclient/external/github.com/docker/go-units
Package units provides helper function to parse and print size and time units in human-readable format.
Package units provides helper function to parse and print size and time units in human-readable format.
_workspace/src/github.com/fsouza/go-dockerclient/external/github.com/gorilla/context
Package context stores values shared during a request lifetime.
Package context stores values shared during a request lifetime.
_workspace/src/github.com/fsouza/go-dockerclient/external/github.com/gorilla/mux
Package gorilla/mux implements a request router and dispatcher.
Package gorilla/mux implements a request router and dispatcher.
_workspace/src/github.com/fsouza/go-dockerclient/external/golang.org/x/net/context
Package context defines the Context type, which carries deadlines, cancelation signals, and other request-scoped values across API boundaries and between processes.
Package context defines the Context type, which carries deadlines, cancelation signals, and other request-scoped values across API boundaries and between processes.
_workspace/src/github.com/fsouza/go-dockerclient/external/golang.org/x/sys/unix
Package unix contains an interface to the low-level operating system primitives.
Package unix contains an interface to the low-level operating system primitives.
_workspace/src/github.com/fsouza/go-dockerclient/testing
Package testing provides a fake implementation of the Docker API, useful for testing purpose.
Package testing provides a fake implementation of the Docker API, useful for testing purpose.
_workspace/src/github.com/golang/glog
Package glog implements logging analogous to the Google-internal C++ INFO/ERROR/V setup.
Package glog implements logging analogous to the Google-internal C++ INFO/ERROR/V setup.
_workspace/src/github.com/russross/blackfriday
Blackfriday markdown processor.
Blackfriday markdown processor.
_workspace/src/github.com/shurcooL/sanitized_anchor_name
Package sanitized_anchor_name provides a func to create sanitized anchor names.
Package sanitized_anchor_name provides a func to create sanitized anchor names.
_workspace/src/github.com/spf13/cobra
Package cobra is a commander providing a simple interface to create powerful modern CLI interfaces.
Package cobra is a commander providing a simple interface to create powerful modern CLI interfaces.
_workspace/src/github.com/spf13/pflag
Package pflag is a drop-in replacement for Go's flag package, implementing POSIX/GNU-style --flags.
Package pflag is a drop-in replacement for Go's flag package, implementing POSIX/GNU-style --flags.
cmd
s2i
pkg
api
Provides types used for processing sti builds.
Provides types used for processing sti builds.
cmd
run
Package run supports running images produced by S2I. It is used by the --run=true command line option.
Package run supports running images produced by S2I. It is used by the --run=true command line option.
scm
tar
test

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL