BitBootcampBitBootcamp ← all explainers 🔄 Review ⚙ Settings

Drill 49 — The Zombie Model

Theme: Compliance, Privacy & Model Governance · Format: 20-min drill · Difficulty: intro

The story

An internal audit walks the decision flow for personal-loan applications and finds a score, risk_v2, consulted on every application. Nobody in the room recognizes it. Archaeology reveals it was deployed in 2019 by a team disbanded in a 2021 reorg; the documentation wiki was migrated twice and the pages didn't survive, the training data was purged in a storage cleanup, and the owner field resolves to a deleted email alias. It has been making production lending decisions, unwatched, for four years — and nobody dares turn it off, because nobody knows what depends on it.

⏸ Stop here and ask the room: the audit report is due in three weeks. What do you even write?

Run it — facilitator script

Timing: ~2 min story · ~10 min questions · ~3 min reveal · ~5 min transfer.

Read the story aloud, pause at the blockquote, and let the silence sit for a moment — the room's discomfort is the drill. Then work the questions in order.

How does a production decision-maker lose its owner without anyone noticing?

Listen for: ownership attached to people and teams instead of roles; documentation living in systems that die during migrations; reorg checklists that reassign headcount but never artifacts; and the key observation that every step — the reorg, the wiki move, the storage cleanup, the alias deletion — was locally reasonable, and no step had a check asking "what does this orphan?"

Weak signal: blaming the 2019 team ("they should have documented it better"). The drill is about the absence of a system that survives teams, not about one team's diligence — any team's docs would have died in those migrations.

What's the minimum set of artifacts that must outlive the team that built a model?

Listen for: a registry entry with an owner role; a statement of purpose and scope; a training-data snapshot or durable pointer; evaluation results; upstream and downstream dependencies; a review date and a retirement plan — all held in a system of record, not a wiki page or a shared drive.

Weak signal: either extreme — "everything, full documentation" with no sense of a minimum, or storage locations that die with teams (personal drives, team wikis, someone's laptop). The answer is a short list in a durable place.

Why is "just turn it off" not an option — and what does that tell you?

Listen for: unknown dependencies mean unknown blast radius — and the fear of the off switch is itself the diagnostic, because it proves the institution can't reason about its own decision path. The safe route is shadow-mode measurement first: log the score, compare decisions with and without it, map who actually consumes it.

Weak signal: bravado — "turn it off and see what breaks" runs that experiment on live loan applicants. In a regulated decision flow that isn't debugging; it's a second audit finding.

The reveal

Remediation takes two quarters. The team has to reconstruct what the model does from its live inputs and outputs — probing it, logging it, correlating its score against known cases — reverse-engineering their own system as if it were a competitor's. The audit finding isn't "the model is bad"; nobody can say whether it's bad. The finding is "the institution cannot explain a system that decides who gets credit."

Why this happens — the mechanics

Governance has an inverted cost curve, and this drill is its purest form. On deployment day, an inventory entry is a form and the documentation is a few pages — hours of work, done by the people who know the answers by heart. Four years later the same information costs two quarters, because it must be reconstructed from the model's behavior instead of copied from the builders' heads. Nothing enforces the cheap moment: the model scores identically whether or not the form was filled in, so the deadline that would force the work never arrives — until an audit invents one, at the expensive end of the curve.

Documentation debt is invisible because nothing pages when knowledge dies. A wiki migration that drops pages looks like a successful migration; a storage cleanup that purges training data frees space and closes a ticket; a deleted email alias just stops delivering. Each loss is silent individually and catastrophic jointly — and there is no monitor for "the institution just forgot something" unless you deliberately build one, which is exactly what a model inventory with a quarterly owner-resolution check is.

Underneath both is the founding problem of Model Risk Management (MRM — the discipline of governing models like the risk-bearing assets they are, anchored in SR 11-7, the Federal Reserve's supervisory guidance on model risk): models are long-lived organizational commitments maintained by short-lived organizational structures. Teams reorganize on a two-to-three-year cycle; a lending model can run for a decade. Unless ownership is attached to something that survives the reorg — a role, a registry, a lifecycle policy — the default outcome is this story: the model outlives everyone who understood it, and keeps deciding anyway.

Diagrams

Problem — how the failure unfolds:

timeline
    title Four years of an unowned model
    section Deployment
        2019 : risk_v2 ships to production : Docs live on the team wiki
    section Silent decay
        2021 : Reorg disbands the team (silent) : Owner alias deleted (silent)
        2022 : Wiki migration drops docs (silent)
        2023 : Training data purged (silent)
        2021-2025 : Model scores every application
    section Discovery
        2025 : Audit asks who owns this : Nobody in the room knows
        2025-2026 : Two quarters of remediation : Reverse-engineering own system

Solution — where the guardrails go:

flowchart TD
    A["Model built"] --> R["Model inventory entry:<br/>owner role, docs, data<br/>snapshot, dependencies"]
    R --> G1["Shipping gate: no registry<br/>entry, no production<br/>traffic"]
    G1 --> D["Model deployed"]
    R --> L["Lifecycle policy: review<br/>date and retirement plan<br/>assigned at birth"]
    O["Reorg dissolves the team"] --> C["Org-change checklist<br/>reassigns the owner role"]
    L -.-> M["Quarterly inventory<br/>review: every owner<br/>resolves to a live human"]
    C -.-> M
    M -.-> AL["Orphaned model flagged<br/>within one quarter"]
    AL --> E["Audit question answered in<br/>an afternoon"]

What good looks like

The lesson

A model inventory is the foundation of Model Risk Management (MRM — the discipline of governing models like the risk-bearing assets they are): every production model registered, with an owner by role so reorgs can't orphan it, documentation as a shipping requirement rather than a follow-up, and a lifecycle policy that assigns every model a review date and a retirement plan on the day it's born. A model nobody owns isn't running unsupervised — it's running ungoverned, which an examiner reads as the same thing.

Named concepts: model inventory, ownership registries, model lifecycle management, MRM (Model Risk Management).

Real-world echoes

Transfer

List every model score consulted in our production decision paths today. For each: who owns it, where are its docs, where is its training data? Any blank cell is a zombie in incubation.

Make it a ticket: Run the inventory sweep this sprint — enumerate every score in the production decision path, fill the owner/docs/training-data columns, and open one remediation ticket per blank cell, assigned by default to the team that consumes the score.

Related