envbuilder

README

envbuilder

Build development environments from a Dockerfile on Docker, Kubernetes, and OpenShift. Allow developers to modify their environment in a tight feedback loop.

• Supports devcontainer.json and Dockerfile
• Cache image layers with registries for speedy builds
• Runs on Kubernetes, Docker, and OpenShift

Quickstart

The easiest way to get started is to run the envbuilder Docker container that clones a repository, builds the image from a Dockerfile, and runs the $INIT_SCRIPT in the freshly built container.

/tmp/envbuilder is used to persist data between commands for the purpose of this demo. You can change it to any directory you want.

docker run -it --rm \
    -v /tmp/envbuilder:/workspaces \
    -e GIT_URL=https://github.com/coder/envbuilder-starter-devcontainer \
    -e INIT_SCRIPT=bash \
    ghcr.io/coder/envbuilder

Edit .devcontainer/Dockerfile to add htop:

$ vim .devcontainer/Dockerfile

- RUN apt-get install vim sudo -y
+ RUN apt-get install vim sudo htop -y

Exit the container, and re-run the docker run command... after the build completes, htop should exist in the container! 🥳

Container Registry Authentication

envbuilder uses Kaniko to build containers. You should follow their instructions to create an authentication configuration.

After you have a configuration that resembles the following:

{
  "auths": {
    "https://index.docker.io/v1/": {
      "auth": "base64-encoded-username-and-password"
    }
  }
}

base64 encode the JSON and provide it to envbuilder as the DOCKER_CONFIG_BASE64 environment variable.

Git Authentication

GIT_USERNAME and GIT_PASSWORD are environment variables to provide Git authentication for private repositories.

For access token-based authentication, follow the following schema (if empty, there's no need to provide the field):

Provider	GIT_USERNAME	GIT_PASSWORD
GitHub	[access-token]
GitLab	oauth2	[access-token]
BitBucket	x-token-auth	[access-token]
Azure DevOps	[access-token]

If using envbuilder inside of Coder, you can use the coder_git_auth Terraform resource to automatically provide this token on workspace creation:

resource "coder_git_auth" "github" {
    id = "github"
}

resource "docker_container" "dev" {
    env = [
        GIT_USERNAME = coder_git_auth.github.access_token,
    ]
}

Layer Caching

Cache layers in a container registry to speed up builds. To enable caching, authenticate with your registry and set the CACHE_REPO environment variable.

CACHE_REPO=ghcr.io/coder/repo-cache

Each layer is stored in the registry as a separate image. The image tag is the hash of the layer's contents. The image digest is the hash of the image tag. The image digest is used to pull the layer from the registry.

Documentation

Index

• Constants
• Variables
• func CloneRepo(ctx context.Context, opts CloneRepoOptions) (bool, error)
• func DefaultWorkspaceFolder(repoURL string) (string, error)
• func HijackLogrus(callback func(entry *logrus.Entry))
• func Run(ctx context.Context, options Options) error
• func SendLogsToCoder(ctx context.Context, client *agentsdk.Client, ...) (func(log agentsdk.StartupLog), error)
• type BuildParameters
• type CloneRepoOptions
• type DevContainer
  • func ParseDevcontainer(content []byte) (*DevContainer, error)
  • func (d *DevContainer) Compile(fs billy.Filesystem, devcontainerDir, scratchDir string) (*BuildParameters, error)
• type DevContainerBuild
• type DockerConfig
• type Options
  • func OptionsFromEnv(getEnv func(string) string) Options

Constants

const (
	// WorkspacesDir is the path to the directory where
	// all workspaces are stored by default.
	WorkspacesDir = "/workspaces"

	// EmptyWorkspaceDir is the path to a workspace that has
	// nothing going on... it's empty!
	EmptyWorkspaceDir = WorkspacesDir + "/empty"

	// MagicDir is where all envbuilder related files are stored.
	// This is a special directory that must not be modified
	// by the user or images.
	MagicDir = ".envbuilder"
)

Variables

var (
	ErrNoFallbackImage = errors.New("no fallback image has been specified")
)

Functions

func CloneRepo

func CloneRepo(ctx context.Context, opts CloneRepoOptions) (bool, error)

CloneRepo will clone the repository at the given URL into the given path. If a repository is already initialized at the given path, it will not be cloned again.

The bool returned states whether the repository was cloned or not.

func DefaultWorkspaceFolder

func DefaultWorkspaceFolder(repoURL string) (string, error)

DefaultWorkspaceFolder returns the default workspace folder for a given repository URL.

func HijackLogrus

func HijackLogrus(callback func(entry *logrus.Entry))

HijackLogrus hijacks the logrus logger and calls the callback for each log entry. This is an abuse of logrus, the package that Kaniko uses, but it exposes no other way to obtain the log entries.

func Run

func Run(ctx context.Context, options Options) error

Run runs the envbuilder.

func SendLogsToCoder

func SendLogsToCoder(ctx context.Context, client *agentsdk.Client, logf func(format string, args ...any)) (func(log agentsdk.StartupLog), error)

SendLogsToCoder returns a function that will automatically queue and debounce logs to send to Coder.

Types

type BuildParameters

type BuildParameters struct {
	DockerfilePath string
	BuildContext   string
	BuildArgs      []string
	Cache          bool

	User string
	Env  []string
}

type CloneRepoOptions

type CloneRepoOptions struct {
	Path    string
	Storage billy.Filesystem

	RepoURL      string
	RepoAuth     transport.AuthMethod
	Progress     sideband.Progress
	Insecure     bool
	SingleBranch bool
	Depth        int
}

type DevContainer

type DevContainer struct {
	Image      string            `json:"image"`
	Build      DevContainerBuild `json:"build"`
	RemoteUser string            `json:"remoteUser"`
	RemoteEnv  map[string]string `json:"remoteEnv"`
}

func ParseDevcontainer

func ParseDevcontainer(content []byte) (*DevContainer, error)

ParseDevcontainer parses a devcontainer.json file.

func (*DevContainer) Compile

func (d *DevContainer) Compile(fs billy.Filesystem, devcontainerDir, scratchDir string) (*BuildParameters, error)

Compile returns the build parameters for the workspace. devcontainerDir is the path to the directory where the devcontainer.json file is located. scratchDir is the path to the directory where the Dockerfile will be written to if one doesn't exist.

type DevContainerBuild

type DevContainerBuild struct {
	Dockerfile string            `json:"dockerfile"`
	Context    string            `json:"context"`
	Args       map[string]string `json:"args"`
	Target     string            `json:"target"`
	CacheFrom  string            `json:"cache_from"`
}

type DockerConfig

type DockerConfig configfile.ConfigFile

DockerConfig represents the Docker configuration file.

type Options

type Options struct {
	// InitScript is the script to run to initialize the workspace.
	InitScript string `env:"INIT_SCRIPT"`

	// CacheRepo is the name of the container registry
	// to push the cache image to. If this is empty, the cache
	// will not be pushed.
	CacheRepo string `env:"CACHE_REPO"`

	// DockerfilePath is a relative path to the workspace
	// folder that will be used to build the workspace.
	// This is an alternative to using a devcontainer
	// that some might find simpler.
	DockerfilePath string `env:"DOCKERFILE_PATH"`

	// DockerConfigBase64 is a base64 encoded Docker config
	// file that will be used to pull images from private
	// container registries.
	DockerConfigBase64 string `env:"DOCKER_CONFIG_BASE64"`

	// FallbackImage is the image to use if no image is
	// specified in the devcontainer.json file and
	// a Dockerfile is not found.
	FallbackImage string `env:"FALLBACK_IMAGE"`

	// ForceSafe ignores any filesystem safety checks.
	// This could cause serious harm to your system!
	// This is used in cases where bypass is needed
	// to unblock customers!
	ForceSafe bool `env:"FORCE_SAFE"`

	// Insecure bypasses TLS verification when cloning
	// and pulling from container registries.
	Insecure bool `env:"INSECURE"`

	// GitURL is the URL of the Git repository to clone.
	// This is optional!
	GitURL string `env:"GIT_URL"`

	// GitCloneDepth is the depth to use when cloning
	// the Git repository.
	GitCloneDepth int `env:"GIT_CLONE_DEPTH"`

	// GitCloneSingleBranch clones only a single branch
	// of the Git repository.
	GitCloneSingleBranch bool `env:"GIT_CLONE_SINGLE_BRANCH"`

	// GitUsername is the username to use for Git authentication.
	// This is optional!
	GitUsername string `env:"GIT_USERNAME"`

	// GitPassword is the password to use for Git authentication.
	// This is optional!
	GitPassword string `env:"GIT_PASSWORD"`

	// WorkspaceFolder is the path to the workspace folder
	// that will be built. This is optional!
	WorkspaceFolder string `env:"WORKSPACE_FOLDER"`

	// Logger is the logger to use for all operations.
	Logger func(level codersdk.LogLevel, format string, args ...interface{})

	// Filesystem is the filesystem to use for all operations.
	// Defaults to the host filesystem.
	Filesystem billy.Filesystem
}

func OptionsFromEnv

func OptionsFromEnv(getEnv func(string) string) Options

OptionsFromEnv returns a set of options from environment variables.