BitBootcampBitBootcamp ← all explainers 🔄 Review ⚙ Settings

CASE-16 · Case Study — XZ Utils: The Backdoor That Almost Shipped

What happened

In March 2024, a Microsoft engineer named Andres Freund was chasing a tiny performance mystery: SSH logins on a test machine were about half a second slower than they should be, and a background process was burning unexpected CPU. Pulling that thread, he uncovered one of the most audacious software supply-chain attacks ever caught — and caught it, by luck and diligence, days before it would have spread to millions of servers.

The target was XZ Utils, a compression library (liblzma) so ordinary that almost no one thinks about it. It ships in nearly every Linux distribution. Over roughly two years, an account using the name "Jia Tan" had patiently built up trust in the small, under-resourced project — submitting helpful patches, being friendly, and eventually being handed co-maintainer authority by the original, burned-out maintainer. Then, in versions 5.6.0 and 5.6.1, that account slipped in a backdoor: heavily obfuscated malicious code hidden not in the visible source, but in the test files and the release build scripts. When the library was built as part of a distribution package, the malicious payload activated and hooked into the authentication path of the SSH daemon, aiming to let the attacker bypass authentication on any affected server. It was assigned CVE-2024-3094 with the maximum severity score of 10.0.

See it — how the backdoor got in

sequenceDiagram
  autonumber
  participant A as "Jia Tan" account
  participant P as XZ project
  participant R as Release tarball
  participant D as Linux distros
  A->>P: Years of helpful patches, build trust
  P->>A: Grants co-maintainer rights
  A->>R: Hide obfuscated payload in build scripts
  R->>D: Ship 5.6.0 / 5.6.1
  D->>D: Backdoor hooks sshd via liblzma

The root cause

The root cause was misplaced trust in a person, amplified by an invisible build step. The visible source code in the Git repository looked clean; the malice lived in the release tarball's build machinery and test fixtures, which few people audit and which differed from what was in the repo. A single volunteer maintainer, overworked and pressured by a chorus of seemingly-helpful strangers, handed the keys to an attacker who had spent years earning them. Critical global infrastructure rested on one unpaid person's judgment — and on a build process nobody was watching.

The control that would have stopped it

The strongest control here is reproducible builds paired with scrutiny of the gap between source and shipped artifact: if the binary that ships can be rebuilt bit-for-bit from the public source, a payload hidden only in the release tarball has nowhere to live. Reducing the "bus factor" of critical projects — funding maintainers, requiring more than one trusted reviewer to approve releases, and treating build scripts and test blobs as security-sensitive code — would have removed the single point of failure the attacker exploited.

The blast radius

Because it was caught before the affected versions reached stable releases, the real-world damage was remarkably limited — mostly bleeding-edge and testing branches of a few distributions. But the potential blast radius was staggering: a silent authentication bypass in a library present on the majority of the world's Linux servers. Had it shipped into stable releases, it could have handed an unknown actor a master key to a large share of the internet's infrastructure. It was a near-miss measured in days.

How it unfolded

timeline
  title The slow-motion infiltration
  2021-2022 : "Jia Tan" account begins contributing
  2023 : Gains maintainer trust and commit rights
  Feb 2024 : Backdoored 5.6.0 released
  Mar 2024 : 5.6.1 released; distros begin adopting
  Mar 29 2024 : Freund reports it; CVE-2024-3094 issued

The lesson

Open-source trust is a supply chain, and its weakest link can be a single tired human. The attack succeeded socially long before it succeeded technically — the code was the last step, not the first. Fund and staff the critical libraries everyone depends on, demand reproducible builds so the shipped artifact matches the audited source, and remember that "it compiles from clean-looking source" is not the same as "it is safe." This is exactly the danger Ken Thompson warned about in Reflections on Trusting Trust, made real four decades later.

Where it rhymes

It rhymes with SolarWinds — both poisoned the trusted build pipeline rather than attacking victims directly — and with the classic paper Reflections on Trusting Trust, which argued you cannot fully trust code you did not build yourself, all the way down the toolchain.