Echo — left-pad (2016)

What happened

In March 2016, a developer named Azer Koçulu unpublished all of his packages from npm after a dispute over the name of one of them, kik. One of those packages was left-pad — a tiny utility, around eleven lines of code, that pads a string on the left. Because left-pad sat deep in the dependency trees of hugely popular projects — build tooling that much of the JavaScript ecosystem depended on — its sudden disappearance from the registry broke installs and builds for a large number of projects almost immediately. npm ultimately took the unusual step of restoring the package to stop the breakage.

The mechanism

The failure was not malicious code — it was removal. npm's publishing model at the time let a maintainer unpublish a version, and countless downstream projects fetched their dependencies live from the public registry at build time rather than from a pinned, vendored, or mirrored copy. So a single maintainer's unilateral decision propagated straight into other people's builds: when the registry no longer served left-pad, every unpinned build that transitively required it failed to resolve. Almost no one had chosen left-pad; it arrived as a transitive dependency of a dependency, one node in a graph nobody had fully inventoried. The trust boundary of a production build quietly extended to every maintainer who held the power to remove a package.

Blast radius

The impact was broad rather than deep: a very large number of projects — including widely used, name-brand tooling — suddenly could not build or install, all at once and without warning. The outage lasted for hours until the package was restored, and the alarm came from the community and from broken CI (Continuous Integration) everywhere, not from any one team's own dependency tooling. The precise count of affected projects varies by account, but the ecosystem-wide scope is not in dispute.

The same-class lesson

The transferable reflex is the same one Drill 60 drills: do not let your production builds depend on the live good behavior of the public registry or of every upstream maintainer. Pin exact versions with lockfiles so nothing changes without a reviewed commit; vendor or mirror dependencies through a registry proxy so a build resolves from a copy you control, not from whatever the public registry serves this second; and keep an inventory of the transitive graph so you actually know what a removal — or a compromise — would take out. left-pad was availability, not a poisoned release, but it teaches the same thing: an unpinned dependency is an unattended decision handed to a stranger. Pin it, mirror it, and inventory it, so one person's bad day upstream is not an outage in your pipeline.

Rhymes with

• Drill 60 — The Package That Turned
• Drill 59 — The Deprecation Email

Named concepts: dependency pinning, transitive dependencies, registry immutability, vendoring.