zarf

README

Zarf - DevSecOps for Air Gap Systems

Zarf simplifies the setup & administration of Kubernetes clusters, cyber systems & workloads that support DevSecOps "across the air gap."

It provides a static go binary (can run anywhere) CLI that can pull, package, and install all the things your clusters need to run and any necessary resources to standup infrastructure such as Terraform. Zarf also caches downloads (for speed), hashes packages (for security), and can even install the Kubernetes cluster itself if you needed.

Zarf runs on a bunch of operating systems. It aims to support configurations ranging from "I want to run one, simple app" to "I need to support & dependency control a bunch of internet-disconnected clusters."

Zarf was theorized and initially demonstrated in Naval Postgraduate School research:

A DEVSECOPS APPROACH FOR DEVELOPING AND DEPLOYING CONTAINERIZED CLOUD-BASED SOFTWARE ON SUBMARINES.

Why is Zarf Needed?

Most of the software ecosystem assumes your systems have access to the internet. The world (for good reasons) has become more and more dependent upon Software as a Service (SaaS), which assumes a robust connection to the internet and a willingness to inherently trust 3rd party providers. Although this makes sense for most of the world, certain secure systems must operate either fully disconnected, semi-disconnected, or might need the ability to disconnect in case of emergencies (like while under an active cyber attack). Although only a small percentage of systems, these secure systems make up some of the most vital systems globally, such as Aerospace and Defense, Finance, Healthcare, Energy, Water, Sewage, and many Federal, Local, and State Government systems.

Explain Zarf Like I'm Ten(ish)

Zarf allows you to bundle portions of "the internet" into a single package to be installed later following specific instructions. A Zarf package is just a single file that includes everything you need to manage a system or capability while fully disconnected. Think of a disconnected system as a system that always is or sometimes is on airplane mode.

You bring this single file (we call it a package) to the system you want to install or update new software on. The package includes instructions on how to assemble all the pieces of software (components) once on the other side. These instructions are fully "declarative," which means that everything is represented by code and automated instead of manual. The hardest part is assembling the declarative package on the connected side. But once everything is packaged, Zarf makes even very complex systems easy to install, update, and maintain within disconnected systems.

Such packages also become highly distributable, as they can now run on edge, embedded systems, secure cloud, data centers, or even in a local environment. This is incredibly helpful for organizations that need to integrate and deploy software from multiple, secure development environments from disparate development teams into disconnected IT operational environments. Zarf helps ensure that development teams can integrate with the production environment they are deploying to, even if they will never actually touch that environment.

Zarf makes DevSecOps for air gap possible.

If you're just getting into Zarf, you should...

Get Started Using Zarf	Experience just how easy it is to go from zero to chainsaw-wielding hero (of the Kubernetes cluster) using Zarf!	Read
Add Logging Zarf components	Sometimes running your app in-cluster is enough— usually, it's not. Find out how to inject commonly-required, uber-useful functionality through Zarf components.	Read
Join Slack Channels	Join our project channels on K8s slack. #zarf #zarf-dev	https://slack.k8s.io/

If you'd rather help develop Zarf, you should read...

Workstation Setup	Are you thinking about hacking on the Zarf binary itself? Or, perhaps you want to run the examples in an isolated environment (the same way we do)? Then, get your machine setup just right using these instructions!	Read
Build Your First Zarf	You've got a development workstation setup, so... now what? Why not build your own Zarf? Step-by-step instructions here.	Read
Contribution Guide	As with most collaborative efforts, there are guidelines for contributing to the project. Find out what they are & how to make them work here.	Read

Or, for details & design decisions, check out...

Supported OSes	Zarf is intended to run on a variety of Operating Systems— you can find out which and discover how to take Zarf for a test drive (in a VM of your favorite flavor) by clicking the link!	Read
Zarf Components	Need to understand what's happening in your cluster? Zarf can give you visibility by injecting a Logging component. Find out all the other stuff Zarf offers here.	Read
Usage Examples	There are a bunch of interesting ways to use Zarf beyond "space-marine-ing" your way through the pixel demon invasion. Browse our examples directory to find out what other neat things are available.	Read
Iron Bank Hardened image registry	Zarf can build deployment packages using images pulled from pretty much anywhere...but for gov't-approved & "hardened" container images, check out Platform One's Iron Bank.	Read

 

Zarf Nerd Notes

Zarf is written entirely in go, except for a single 400Kb binary for the injector system written in rust, so we can fit it in a configmap. All assets are bundled together into a single zstd tarball on each zarf package create operation. On the air gap / offline side, zarf package deploy extracts the various assets and places them on the filesystem or installs them in the cluster, depending on what the zarf package says to do. Some important ideas behind Zarf:

• All workloads are installed in the cluster via the Helm SDK
• The OCI Registries used are both from Docker
• Currently, the Registry and Git servers are not HA, see #375 and #376 for discussion on this
• To avoid TLS issues, Zarf binds to 127.0.0.1:31999 on each node as a NodePort to allow all nodes to access the pod(s) in the cluster
• A mutating webhook we call the Zarf Agent handles updating pod's ImagePullSecrets so the deployed resources use the NodePort binding
• Note: Existing namespaces in the cluster prior to zarf init are marked as ignored by the Zarf Agent so you should not use them with Zarf-deployed resources. Most common would be the default or kube-system namespaces
• Zarf uses a custom injector system to bootstrap a new cluster. See the PR #329 and ADR for more details on how we came to this solution. The general steps are listed below:
  • Get a list of images in the cluster
  • Attempt to create an ephemeral pod using an image from the list
  • A small rust binary that is compiled using musl to keep the size the max binary size of ~ 672 KBs is injected into the pod
  • The mini zarf registry binary and docker:2 images are put in a tar archive and split into 512 KB chunks; larger sizes tended to cause latency issues on low-resource control planes
  • An init container runs the rust binary to re-assemble and extract the zarf binary and registry image
  • The container then starts and runs the zarf binary to host the registry image in an embedded docker registry
  • After this, the main docker registry chart is deployed, pulls the image from the ephemeral pod, and finally destroys the created configmaps, pod, and service
• We have published a Homebrew Tap for this project that you can install with brew tap defenseunicorns/tap && brew install zarf
  • This recipe is updated with each new version release via GoReleaser as part of the release pipeline that runs on GitHub Actions.  

Zarf Architecture

Source DrawIO