build-tools

README

build-tools

A set of highly opinionated tools for creating and building components/services into docker images and deploying them to Kubernetes clusters.

By following the conventions set by the tools, building and deploying applications is made simpler.

The only hard requirement is to provide a Dockerfile which describes how to build and run your application.

The configuration needed is done by environment variables (most likely for CI/CD) and yaml files (for local use).

Available commands

build

push

deploy

Conventions

• Dockerfile must be present in the root of the project directory (TODO Override name of file). The Dockerfile will be used to build the project into a runnable docker image.
• The name of the directory will be used as the name of the docker image (TODO Override by ENV)
• The current commit id will be used as docker tag
• Kubernetes descriptor files must be located in the k8s folder under the root

Take a look at the build-tools-example repository (TODO link) to try it out.

Using in CI/CD pipelines

Example usage

After installing (TODO link) the tools, clone the build-tools-example repository (TODO link), cd into it and execute the build command.

$ build
Using CI none

no Docker registry found

Since we we haven't setup a.buildtools.yaml (TODO LINK in doc) file, nothing has been configured, and to be able to build a docker image we must specify where we (potentially) want to push it later. In other words, setting the tags of the created image. Luckily we can use environment variables as well, let's try:

$ DOCKERHUB_REPOSITORY=sparetimecoders build
Using CI none

Using registry Dockerhub

Login Succeeded
Using build variables commit 7c76db502b4a70df5480d6ff438ae10e374b420e on branch master

As we can see, the build command identified that we are using Dockerhub, and extracted the commit id and branch information from the local git repository. Notice that the name of the current directory is used as the image name. After the successful build the image is tagged with the commit id and branch.

Successfully tagged sparetimecoders/buildtools-examples:7c76db502b4a70df5480d6ff438ae10e374b420e
Successfully tagged sparetimecoders/buildtools-examples:master
Successfully tagged sparetimecoders/buildtools-examples:latest

Now that we have a docker image, let's publish it to the docker repository (this of course requires write access to the repository).

$ DOCKERHUB_REPOSITORY=sparetimecoders DOCKERHUB_PASSWORD=<PASSWORD> DOCKERHUB_USERNAME=<USERNAME> push
...
Pushing tag 'sparetimecoders/buildtools-examples:7c76db502b4a70df5480d6ff438ae10e374b420e'
...

TODO Link to more environment variables and stuff

Basic setup

• Clone this repository
• Add BUILD_TOOLS_PATH=<PATH TO THIS REPOSITORY> to your shell environment, typically in .bash_profile or something similar

Setup script

Script to be used for scaffolding a component/service. Handles repository, build-pipeline and basic files-scaffolding.

${BUILD_TOOLS_PATH}/service-setup --stack <stack> <name>

See Build project structure below

Common build scripts

Scripts to be used for building and deploying our services (typically when running in a CI/CD environment). They can also be used for local builds/deploy.

${BUILD_TOOLS_PATH}/build
${BUILD_TOOLS_PATH}/push

Build project structure

Configuration and setup is done in .buildtools files. Those files must be present in the project folder or upwards in the dicectory structure. This lets you create a common .buildtools file to be used for a set of projects. Example:

$ pwd
~/source/
$ tree
.
├── customer1
│   ├── project1
│   └── project2
└── customer2
    └── project1

Here we can choose to put a .buildtools file in the different customer directories since they (most likely) have different deployment configuration.

$ cat customer1/.buildtools
CI=azure
VCS=azure
REGISTRY=dockerhub
ORGANIZATION=com.organization
DOCKERHUB_REPOSITORY=repository
DOCKERHUB_USERNAME=user
DOCKERHUB_PASSWORD=$(lpass show devenv/docker.com --password)
AZURE_USER=user
AZURE_TOKEN=token
AZURE_ORG=organization
AZURE_PROJECT=project
PIPELINE_VARIABLES=(
["DOCKERHUB_REPOSITORY"]="repository"
["DOCKERHUB_USERNAME"]="user"
["DOCKERHUB_PASSWORD"]="secret:$(lpass show devenv/docker.com --password)"
["BUILDTOOLS_CONTENT"]="$(lpass show devenv/organization/team/buildtools --notes | base64 -w0)"
["KUBECONFIG_CONTENT"]="$(lpass show devenv/organization/team/kubeconfig --notes | base64 -w0)"
)
valid_environments=(
    ["local"]="--context docker-for-desktop --namespace default"
    ["staging"]="--context docker-for-desktop --namespace staging"
    ["prod"]="--context docker-for-desktop --namespace prod"
)

This defines three environments (local,staging,prod) all which are to be deployed to a local Kubernetes cluster but in different namespaces.

$ cat customer2/.buildtools
valid_environments=(
    ["local"]="--context docker-for-desktop --namespace local"
    ["prod"]="--context kube-cluster-prod --namespace prod"
)

Context and namespaces must be provided/created/configured elsewhere.

Configuration for service-setup

The service-setup script need to know where to scaffold things (i.e. which VCS, CI and container registry to use and how to connect to them). The environment variables CI, VCS and REGISTRY are used to define this. Depending on the values of those variables, other variables are required as well. See each section below for the required variables. Values from the associative array PIPELINE_VARIABLES will be created as pipeline-variables in the CI-system. If the value is prepended vith secret: the variable will be a secret (if the CI-system supports that).

Azure Devops

If CI and/or VCS is set to azure the following variables are required:

Variable	Value
AZURE_USER	Azure username
AZURE_TOKEN	Azure Personal Access Token
AZURE_ORG	The organization name in Azure
AZURE_PROJECT	The project name in Azure where repository and build-pipeline will be created

Buildkite

If CI is set to buildkite the following variables are required:

Variable	Value
BUILDKITE_TOKEN	Buildkite Access Token
BUILDKITE_ORG	The organization name in Buildkite where the build-pipeline will be created

GitlabCI

If CI or VCS is set to gitlab the following variables are required:

Variable	Value
GITLAB_TOKEN	Gitlab Access Token
GITLAB_GROUP	The group name in Gitlab where project will be created

Configuration using environment variables

The .buildtools file can be created by defining an environment variable in the build pipeline named BUILDTOOLS_CONTENT. The value should be a base64-encoded string. On MacOS the value can be created and copied to the clipboard using the following snippet:

cat - <<EOF | base64 -w0 | pbcopy
valid_environments=(
    ["local"]="--context docker-for-desktop --namespace local"
    ["prod"]="--context kube-cluster-prod --namespace prod"
)
EOF

The Kubernetes contexts to deploy to are configured by setting the KUBECONFIG_CONTENT variable. The value should be a base64-encoded string containing the Kubernetes config for the clusters you want to be able to deploy to. On MacOS the value can be created and copied to the clipboard using the following snippet:

cat ~/.kube/config.d/prod-cluster.yaml | base64 -w0 | pbcopy

The scripts assume that the project follow the directory layout described below.

Project structure

The project folder must be a Git repository, with a least one commit.

There must be a k8s directory in the root of your project file. This directory contains all the yaml files used to describe the Kubernetes deployment tasks needed to run this service. Environment specific files can be handled in two different ways depending on personal preference. They can either be located in sub-directories, for example k8s/local for local setup.

$ cd projecct
$ tree
.
└── k8s
    ├── deploy.yaml
    ├── local
    │   ├── ingress.yaml
    │   └── setup.sh
    └── prod
        └── ingress.yaml

Or they can be defined using a -<environment> suffix, i.e. ingress-prod.yaml

$ cd projecct
$ tree
.
└── k8s
    ├── deploy.yaml
    ├── ingress-local.yaml
    ├── ingress-prod.yaml
    └── setup-local.sh

Running in a CI/CD environment

The tools recognize which CI/CD environment they are executed in based on which environment variables are present.

Variable	CI/CD environment
VSTS_PROCESS_LOOKUP_ID	Azure Devops
BUILDKITE_COMMIT	Buildkite
GITLAB_CI	GitlabCI

Support for different Docker container registries

The container registry to use when running docker:push is also defined by environment variables.

Variable	Container registry	Example value
DOCKERHUB_REPOSITORY	Docker Hub	bitnami (resulting in bitnami/<image>
ECR_URL	AWS ECR	12345678.dkr.ecr.eu-west-1.amazonaws.com
CI_REGISTRY_IMAGE	Gitlab Registry	registry.gitlab.com/sparetimecoders/build-tools
QUAY_REPOSITORY	Quay.io	quay.io/bitnami

Other environment variables that need to be defined (either automatically by the CI/CD environment or manually in the build pipeline) for each of the container registries are defined below.

Docker Hub

Variable	Description
DOCKERHUB_USERNAME	Username
DOCKERHUB_PASSWORD	Password

AWS ECR

Variable	Description
ECR_REGION	Optionally specified region, will use eu-west-1 as default value

Gitlab Registry

Variable	Description
CI_BUILD_TOKEN	The build-token set by GitlabCI
CI_REGISTRY	The URL to the registry set by GitlabCI

Quay.io

Variable	Description
QUAY_USERNAME	Username
QUAY_PASSWORD	Password

Example Azure Devops pipeline (azure-pipelines.yml)

resources:
  containers:
  - container: build-tools
    image: registry.gitlab.com/sparetimecoders/build-tools:master

jobs:
- job: build_and_deploy
  pool:
    vmImage: 'Ubuntu 16.04'
  container: build-tools
  steps:
  - script: |
      build
      push
    name: build
    env:
      DOCKERHUB_PASSWORD: $(DOCKERHUB_PASSWORD)
      QUAY_PASSWORD: $(QUAY_PASSWORD)
  - script: deploy staging
    name: deploy_staging
    condition: succeeded()

Example Buildkite pipeline (.buildkite/pipeline.yml)

steps:
  - command: |-
      build
      push
    label: build

  - wait

  - command: |-
      ${BUILD_TOOLS_PATH}/deploy staging
    label: Deploy to staging
    branches: "master"

  - block: ":rocket: Release PROD"
    branches: "master"

  - command: |-
      ${BUILD_TOOLS_PATH}/deploy prod
    label: Deploy PROD
    branches: "master"

Example GitlabCI pipeline (.gitlab-ci.yaml)

stages:
  - build
  - deploy-staging
  - deploy-prod

variables:
  DOCKER_HOST: tcp://docker:2375/

image: registry.gitlab.com/sparetimecoders/build-tools:master

build:
  stage: build
  services:
    - docker:dind
  script:
  - build
  - push

deploy-to-staging:
  stage: deploy-staging
  when: on_success
  script:
    - echo Deploy to staging.
    - deploy staging
  environment:
    name: staging

deploy-to-prod:
  stage: deploy-prod
  when: on_success
  script:
    - echo Deploy to PROD.
    - deploy prod
  environment:
    name: prod
  only:
    - master

Developing

Generate test mocks

mockgen -package=mocks -destination=pkg/config/mocks/MockRepositoriesService.go gitlab.com/sparetimecoders/build-tools/pkg/config RepositoriesService