turbobob

README

⬆️ For table of contents, click the above icon

Modern, minimal container-based build/development tool to make any project´s dev easy and frictionless.

Think like GitHub actions, but actually runnable locally (and also runnable from GitHub actions).

Note: while Bob uses containers for builds (and dev), your programs themselves don't need to use containers!

In a nutshell

flowchart TD
    GitHub[GitHub actions] --> Build_from_ci[Build from CI]
    GitLab[GitLab CI] --> Build_from_ci
    OtherCI[... CI] -->|In each CI: small boilerplate\nCI-specific conf to ask\nBob to do the build| Build_from_ci
    Build_from_ci -->|$ bob build in-ci-autodetect-settings| TurboBob
    Build_locally[Build locally] -->|$ bob build| TurboBob
    Develop_locally[Develop locally] -->|$ bob dev| TurboBob
    TurboBob[Turbo Bob<small>\ncontainer-based\nbuild orchestration</small>] -->|$ docker run ...| Docker

Notes:

• Here's what the GitHub actions boilerplate looks like for just passing the build to Bob
  • You can think of these as CI-specific adapters for passing control to Turbo Bob
• Then here's the container image that gets run to do the build
  • This is reusable container to build all our Go-based projects, i.e. the build environment can be shared across many projects. Ship one improvement to the build environment -> many projects benefit.

Small demo screencast

Features

• Log line grouping in GitHub actions
• Automatically adds OCI-compliant metadata to built containers. (Example, click "latest")

Additional documentation

• Using Bob in your project (also covers making your own builder images - "buildkits")
• ENV vars passed to build containers
• Displaying pro-tips on entering the dev environment (also covers mapping network ports)
• Development-time HTTP ingresses (routing HTTP requests)
• Customizing dev container startup
• Language server support (code completion, analysis, refactoring support etc. for code editors)
• Quality helpers (multi-project quality scalability by automatically checking standards-compliance like having README, LICENSE, security policy etc.)

What is this?

Turbo Bob (the builder) is an abstraction for building and developing your software, whether it happens in your laptop or in a CI system.

Usage of Turbo Bob, in every project you're developing:

$ bob dev

This gives you a shell inside the build environment container with the working directory mounted inside the container so that you can directly edit your code files from your host system.

To build your project:

$ bob build

By keeping these commands consistent across each project we'll minimize friction with mental context switching, since the commands are the same for each project whether you're building a Docker-based image or running anything custom that produces build artefacts.

There's a document that your project can link to for build & help instructions. This explains Bob's value proposition quite well and serves as the first introduction for new Bob users. See an example of a project's build docs linking to Bob.

Philosophy

• Your project must support a simple build and dev interface. If you can't, you're probably doing something wrong and you should simplify it. The build command usually just runs your project's bin/build.sh (or equivalent) command inside a container. The dev command usually starts Bash terminal inside the container but doesn't execute bin/build.sh so you can manually invoke or debug the build process (or a subset of it).

• Build environment should be stateless & immutable. No longer missing build tools or mismatched versions within your team. Nothing to install on your CI server (except Docker).

• Decouple build-time dependencies from runtime dependencies (build container pattern), so build tools will not be shipped to production (smaller images & less attack surface).

• Dev/CI/production environment parity as close as possible. Dev environment is the same as build & CI environment. What's built on dev ($ bob build) is exactly the same or as close as possible ($ bob build --uncommitted) as to what will end up running in production.

• No vendor lock-in for a CI system. Bob can seamlessly build projects on your laptop, GitHub actions, Jenkins, GitLab etc. CI needs to only provide the working directory and Docker - everything else like uploading artefacts to S3, Bintray etc. should be a build container concern to provide full independence.

Install

Linux

Requires Docker for use, so currently only Linux is supported. Native Windows support might come later as Windows' Linux subsystem keeps maturing.

$ sudo curl --location --fail --output /usr/bin/bob https://function61.com/go/turbobob-latest-stable-linux-amd64 && sudo chmod +x /usr/bin/bob

Windows

I have used Turbo Bob in Windows through Vagrant (= run Linux in VM) quite succesfully for years.

I can edit my project files from the Windows host and those changes reflect inside the container just fine, though the Linux setup is definitely simpler and has less moving parts. The setup looks like this:

Mac

I have no experience whatsoever with Mac, but I hear Vagrant works on Mac so maybe it works the same way I described it works on Windows.

Supported build/CI platforms

Basic approach anywhere:

1. Have Docker installed
2. If you don't have Turbo Bob installed, download it
3. Run $ bob build (i.e. hand off build to it)

Your own computer

If your system can run Docker locally, you can build projects on your own computer.

GitHub actions

See example actions workflow file.

GitHub actions' design is pretty similar to Turbo Bob's design ("run stuff inside containers"). I started this project before actions was announced, so unfortunately there's currently no synergy with these projects. I'd like to research if Bob concepts could directly be mapped into actions' concepts (perhaps you could just generate actions' workflow file from turbobob.json).

GitLab

I've built projects on GitLab's public runners with Bob. See example configuration.

Other CI systems, vendor lock-in

Bob's approach is pretty generic (as documented under the above larger heading).

Here's a commit demonstrating how portable Bob is by moving from Travis CI -> GitHub actions - how it's just from small boilerplate to small boilerplate. This prevents vendor lock-in. (NOTE: GitHub actions boilerplate has since been updated).

If you've added support to other public CI systems, please add links to here for instructions!

Examples / how does it work?

NOTE: The easiest way to understand is to read "Using Bob in your project" first!

Examples

Look for the turbobob.json file in each of these example repos. Most of them use multiple container images ("buildkits") for builds:

• This project itself
• function61/james
  • uses buildkit-golang
  • uses buildkit-publisher
• function61/lambda-alertmanager
  • uses buildkit-golang
  • uses buildkit-js
  • uses buildkit-publisher
• function61/hautomo
  • uses buildkit-golang
  • uses buildkit-js (via build-alexaconnector.Dockerfile)
  • uses buildkit-publisher

How does turbobob.json work?

The process is exactly the same whether you use a different CI system. You can even run builds exactly the same way on your laptop by just running $ bob build.

This very project is built with Bob on GitHub actions. The workflow configuration is minimal. Here's what happens when a new commit lands in this repo:

• GitHub actions start processing the build workflow file which:
  • Downloads Turbo Bob
  • Hands off build process to Bob
• Bob reads turbobob.json, which instructs to:
  • Run container off of image fn61/buildkit-golang (repo) and run build-go-project.sh (defined by the buildkit) inside it. We could of course store the build script in our own repo, but it's advantageous to have it defined by the buildkit, so improvements can "automatically" ship to multiple projects.
  • For publishing step, run container off of image fn61/buildkit-publisher (repo) and run publish.sh rel/ inside it (again defined by the buildkit)

Why multiple buildkits?

If your project e.g. uses Go for backend and TypeScript for frontend, it's hygienic to keep the build tools separate so:

• They can't conflict with each other.
• One buildkit doing one thing enables reusability.
  • All of function61's projects use buildkit-golang. The decision has already paid itself back.
• It also allows the build environments to evolve independently (update another buildkit without breaking others).
• Increases your chances of finding community-provided buildkits so you don't have to maintain your own.

p.s. "buildkit" is not a Turbo bob concept per se. It just means "a container image with tooling". You can probably use images with Turbo Bob that aren't designed with Turbo Bob in mind.

Alternative software

These technologies have some overlap with Turbo Bob:

• Visual Studio Code Dev Container - tools inside container
• GitHub codespaces - Code editor in cloud + tools inside container?
• nektos/act - Run your GitHub Actions locally