README
¶
Docker Pushes Watcher
Docker pushes watcher is an app that with new task drivers and new jobs has replaced Google Cloud Build for Skia. The tracking bug for this work was skbug/9514.
Previous Cloud Build framework
Skia used to trigger builds on Google Cloud's Cloud Build framework using this cloudbuild.yaml file.
Using cloud build, the "skia-release" and "skia-wasm-release" images were created and pushed per commit in the Skia repository. The "infra" image was created and pushed per commit in the Buildbot repository.
Additionally cloudbuild.yaml created and pushed new images for various apps like fiddler, skottie, debugger, etc.
The continuous-deploy app then ran pushk on those apps and deployed them to k8s.
Replacement framework with Task Drivers and Docker Pushes Watcher app
The Cloud build framework worked but did not use Skia infra framework and was thus difficult to see failures and diagnose problems. The framework was replaced with task drivers and the docker-pushes-watcher app.
Now all failures when building apps cause the corresponding jobs to turn red on status.skia.org for the commits that break things, this makes failures much easier to find for the Skia Gardener. Failures in the docker-pushes-watcher app show up as alerts for the Infra Gardener to diagnose and fix.
Task Drivers
A task driver was written to build and push a specified docker image (build_push_docker_image).
The following jobs were then created in the Skia repository:
- Housekeeper-PerCommit-CreateDockerImage_Skia_Release job to create and push the "gcr.io/skia-public/skia-release" image using this Dockerfile.
- Housekeeper-PerCommit-CreateDockerImage_Skia_WASM_Release job to create and push the "gcr.io/skia-public/skia-wasm-release" image using this Dockerfile.
These jobs could run out of order because of backfilling. Due to this the "Docker Pushes Watcher" app (described below) calculates which image is the most recent and then tags it with the "prod" tag.
Task Drivers were also written to create and push Docker images of various apps that depend on the "gcr.io/skia-public/skia-release" and "gcr.io/skia-public/skia-wasm-release" images:
The following jobs were created in the Skia repo using the above task drivers:
- Housekeeper-PerCommit-PushAppsFromSkiaDockerImage job to create and push docker images for fiddler and api apps.
- Housekeeper-PerCommit-PushBazelAppsFromWASMDockerImage job to create and push docker images for jsfiddle, skottie, debugger, and particle apps.
All above task drivers send a pubsub message when a docker image is created and pushed.
Docker Pushes Watcher App
The docker pushes watcher app listens for pubsub messages for 2 main tasks:
- Tags images in the app's list with the "prod" tag when they correspond to the latest commit in the Skia/Buildbot repository. This is done to account for jobs running out of order because of backfilling.
- Deploys apps to k8s using pushk for images in the app's list when they correspond to the latest commit.
Directories