What's what with Wolfi, the Linux "undistribution," and ARM

Want to really secure a containerized Linux? Don't include the kernel. Seriously.

Author

Steven Vaughan-Nichols
January 11, 2023

There are many ways to get serious about securing Linux on a container. Heck, Microsoft has one, Common Base Linux (CBL)-Mariner. Others include Alpine Linux, Flatcar Container Linux, Red Hat Enterprise Linux CoreOS (RHCOS), and RancherOS. The name of this game is to shrink down Linux to the smallest possible size so that its attack surface is tiny. Then, there's Chainguard's Wolfi. Here, there's no Linux kernel at all.

How do they manage that? Simple. Wolfi relies on a kernel being provided by an underlying container runtime. Secure that, and you're safe from most kernel-based attacks.

Chainguard CEO and founder Dan Lorenc told me at Open Source Summit Europe in Dublin, A Linux container is "a distro that boots up on hardware and gets you to a container runtime. Alpine is probably the most heavily used such distro. Wolfi is the opposite of this. It's distroless. It's minimal to the point of not even having a package manager." It has just enough to run your containerized application, and that's it.

Wolfi provides useful packages and languages such as curl, go, rust, python, and libraries like libxml, libssl, and libcap. With other distributions, you might install these and additional code with a package manager such as apt or rpm. Wolfi, which was built largely by Alpine Linux developers, uses Alpine's apk package format and package manager. The Wolfi packages are built from upstream source code, using Chainguard's custom apk package build tool Melange.

Its most important features include the following:

  • Based on the Alpine Package (APK) format

  • Packages are of an appropriate granularity and independence to support minimal images

  • Comes with a high-quality, build-time software bill of materials (SBOM) for all packages

  • Fully declarative and reproducible build system

At a high level, Chainguard engineer Jason Hall explained, "For example, Wolfi's curl package fetches source from https://curl.se and builds it with make. But where's make come from? Well, make is also built from source, using make. Wait what?!! Where does that make come from?!"

Then, it gets "complicated." The short version is that with Stage 1, the process starts by making and compiling a tiny number of core packages, such as gcc and glibc, using Alpine's tools and libraries.

Next, in Stage 2, using those basic packages, Chainguard creates useful packages such as busybox, gzip, OpenSSL, sed, and wget. To do that, it also calls on other Alpine programs, such as Perl. At this point, Chainguard also rebuilds the Stage 1 packages, such as gcc. Think of Stage 1 as the first stage of a rocket. It's left behind as Stage 2 fires to fly the packages higher.

"Stage 3 is where, " Hall wrote, "it starts to get fun. As with Stage 2, we'll start to build a few more packages than the previous stage, using packages provided by the previous stage. Unlike Stage 2, Stage 3 has no remaining dependencies on Alpine packages."

Stage 3's aim is to produce a minimal set of packages built using only Stage 2 Wolfi packages. All the previous packages are rebuilt, and a few more essential packages are built. You can check these Stage 3 programs out on GitHub.

Wolfi's not done yet! At long last, we're ready to go into program orbit. The "real" Wolfi OS repo packages are built using Stage 3 packages, including busybox, wget, gcc, and make. All these are rebuilt for the final time from the source code. "These packages," Hall said, "are the ones intended for wide usage by Wolfi users."

Then, Wolfi finally has enough programs to build almost everything else. Wolfi uses the bootstrapped packages to build packages for OpenJDK, Ruby, NodeJS, Python, etc., etc. Need another package? Let Chainguard know, and they'll get on it.

There are two other languages, Go and Rust, that take another path into Wolfi. With these, Hall explained, "Wolfi first fetches a Go toolchain pre-built by the Go team, then uses that to build the Go toolchain from source. Like the previous bootstrap packages, the so-called "go-stage0" package isn't meant to do anything but build the "real" Go package. Wolfi does the same thing for Rust, too – use a pre-built Rust toolchain to build the Rust toolchain from source. The built-from-source Rust toolchain is the one Wolfi users use, and rust-stage0 is just there to show our work."

Finally, Chainguard uses another tool, apko, to produce its Chainguard Images. You can then use these as a base container image for your applications, or in your build pipelines. Like Wolfi OS packages, Chainguard Images are also built transparently with GitHub configs and build logs.

By now, you're doubtlessly asking, "What's the point of all this?" It's all about spelling out exactly what's in Wolfi's software supply chain. As you doubtlessly know, knowing what's what in your software supply chain is essential for security. 

The logic behind all this, Lorenc told me in Dublin, "We believe in minimizing dependencies as much as possible, which simplifies auditing, updating, and transferring images, as well as reducing the potential attack surface. Wolfi [named for the smallest and most flexible octopus] is designed from the ground up to take full advantage of these containerized environments while maximizing security."

Another reason for this approach is that it gives you Chainguard the ability to add security patches quickly. It also lets them add features, such as enabling yjit in Ruby 3, as soon as they're available in the source code.

These Wolfi images are rebuilt daily from upstream sources. The images are signed via Sigstore, the standard for signing and verifying code, and described in an SBOM. This signature can be verified to show that the image is the one you wanted and is free of any tampering.

For you, this means you get the freshest images, and you can be sure that you know exactly what's in them. Of course, getting the newest code can sometimes go badly wrong. Just ask the poor people who were using PyTorch dailies late last December. But, that's the exception to the rule.

If you like, you can build the images for yourself. Every single package in these images is reproducible by default. That means you'll get the same image if you build the package yourself from the source code. This is guaranteed by using Supply Chain Levels for Software Artifacts (SLSA, pronounced salsa). This is a source-to-service security framework for ensuring the integrity of software artifacts by protecting against unauthorized software package changes.

Finally, while Wolfi has been available for quite a while on the x64 platform, you can now use it on ARM64 and aarch64 packages. Hall explained, "With Arm packages built, we can start to provide multi-platform Chainguard Images with support for Arm container runtime environments, like those available on all three major clouds, AWS, GCP, and Azure, so that you can take advantage of Arm's power consumption and cost benefits." This will help Chainguard Image users optimize their cloud deployments, so "they don't cost an arm and a leg."

Mea culpa. I like the idea of having the newest code and full supply chain security baked into my x86 and ARM images. If you like the approach too, give Chainguard a try,

Other noteworthy Linux and open-source stories: