Drill 1 — The Model That Scored Everyone the Same

Theme: Silent Failures · Format: 20-min drill · Difficulty: core

The story

A feature service starts timing out intermittently after a dependency upgrade. The scoring service handles it gracefully — missing features are imputed with training-set means, exactly as designed. Result: 40% of applications are scored with mostly-imputed vectors, and scores compress into a narrow band around 0.52. Approvals become close to random. Latency: normal. Error rate: zero. Uptime: 100%.

⏸ Stop here and ask the room: it runs like this for 19 days. Who notices first, and how?

Run it — facilitator script

Timing for a 20-minute session: 2 min story · 10 min staged questions · 3 min reveal · 5 min transfer.

What's the earliest observable symptom?

Listen for: signals at the data layer, upstream of the score — the feature-null rate spiking the moment the dependency upgrade ships, and the imputation rate climbing right behind it. The score-distribution compression follows within hours; the approval-mix shift is the last symptom, not the first.
Weak signal: answers that reach for latency, error rates, or CPU — the drill's whole point is that every infrastructure metric stayed healthy. Someone naming only technical signals hasn't yet made the shift from "is it up?" to "is it right?"

What monitor would have caught it on day one?

Listen for: named SLIs (Service Level Indicators — the numbers that measure whether the service does its job) with thresholds and alarms: per-feature null rate, imputation rate per request, and a population-stability monitor on the score histogram — each firing in minutes, not surfacing in a weekly review.
Weak signal: "we should have a dashboard for that" or "someone should look at the score distribution weekly" — passive observation is human-in-the-loop detection, and humans check dashboards on business cadence. That's how you get 19 days.

Was the graceful degradation a good idea?

Listen for: nuance — degrading gracefully is fine, degrading silently is the bug. A strong answer separates the fallback behavior (impute and keep serving) from the missing signal (a degradation SLI, plus a cap: above X% imputed features, flag the decision or refuse to score).
Weak signal: binary verdicts — "yes, it protected uptime" or "no, always fail fast" — without noticing that the design choice isn't degrade-vs-crash, it's silent-vs-observable.

The reveal

A portfolio analyst notices the approval mix shifted. Nobody in engineering ever saw it: every technical signal was healthy, because the system was doing exactly what it was built to do — degrade silently.

Why this happens — the mechanics

The technical root is that imputation is designed behavior. The scoring service has no concept of "wrongness" — filling missing features with training-set means is exactly what its code says to do, so from the inside there is nothing to log at error level, nothing to count as a failure, nothing to page about. The 200 OK contract is honored on every request. Monitoring inherits the infrastructure's point of view, and from that point of view a model scoring garbage is indistinguishable from a model scoring well.

The organizational root is a seam in ownership. Engineering owns uptime, latency, and error rates; the analytics or portfolio side owns decision quality — approval mix, loss rates, score distributions. The failure lives precisely in the gap: it is invisible to the metrics engineering watches and only slowly visible in the metrics analytics watches, because business reviews run on weekly or monthly cadence. Detection latency is set by the cadence of whoever happens to own the symptom.

Competent, busy teams fall into this because graceful degradation is a best practice. The fallback path passed review on its merits — it prevents outages, it was probably praised. What review didn't ask is "when this path activates, who finds out, and how fast?" A fallback without a signal converts a loud failure you'd have fixed in an hour into a quiet one you'll fix in three weeks. The mean time to detect (MTTD — how long a failure goes unnoticed) for this class is measured in business-review cycles, and every day of it is a day of near-random lending decisions compounding on the book.

Diagrams

Problem — how the failure unfolds:

sequenceDiagram
    autonumber
    participant FS as Feature service
    participant SS as Scoring service
    participant DB as Dashboards
    participant AN as Portfolio analyst
    Note over FS: Dependency upgrade ships
    FS--xSS: Intermittent timeouts -<br/>40% of features arrive null
    Note over SS: Imputes training-set<br/>means (silent)
    Note over SS: Scores compress to 0.52 -<br/>approvals near random (silent)
    SS->>DB: Latency normal, zero errors
    Note over DB: Green for 19 days
    AN->>SS: Why did the approval<br/>mix shift?
    Note over AN,SS: Day 19 - found by a human,<br/>not a monitor

Solution — where the guardrails go:

flowchart TD
    A["Dependency upgrade ships"] --> B["Feature service times out<br/>intermittently"]
    B --> C["Feature-null rate spikes"]
    C --> G1["Null-rate SLI with threshold"]
    G1 -.-> AL["Alarm fires in minutes"]
    C --> D["Imputation rate climbs"]
    D --> G2["Imputation-rate<br/>degradation SLI"]
    G2 -.-> AL
    D --> E["Score distribution compresses"]
    E --> G3["Population stability<br/>monitor on score histogram"]
    G3 -.-> AL
    AL --> H["On-call rolls back the<br/>upgrade same day"]

What good looks like

Per-feature null-rate SLIs on the feature service: alarm when any top-20-by-importance feature's null rate exceeds 3× its trailing 7-day baseline for 15 minutes.
An imputation-rate degradation SLI on the scoring service: the share of requests scored with more than K% imputed features — warn at 2%, page at 5%. If the fallback path activates at scale, that is the incident.
A degradation cap, not just a signal: above a written imputation threshold per vector, the score is flagged as degraded downstream (route to manual review, or decline to auto-decide) rather than silently treated as first-class.
Population-stability monitoring on the score histogram: compute PSI (Population Stability Index) or a KS-statistic hourly against a trailing baseline — warn at PSI 0.1, page at 0.25. Score compression to a narrow band screams here on day one.
A business-metric backstop: approval rate per segment with anomaly detection at daily granularity — the monitor of last resort for the failure modes you didn't imagine.
Post-deploy verification for dependency upgrades of feature services: null rates and imputation rates checked automatically in the hour after rollout, same as for code deploys.
Monitor tests: inject a synthetic null-flood in staging quarterly and confirm the null-rate, imputation-rate, and stability alarms all fire. An alert that has never fired is unverified code.

The lesson

Define SLIs at the level of what the service is for: score-distribution stability, imputation rate, feature-null rates — each with alarms. Graceful degradation without a degradation SLI is just silent failure with good manners.

Named concepts: SLI/SLO design, semantic monitoring, population stability monitoring.

Real-world echoes

Rhymes with Knight Capital, 2012: a malfunctioning, unmonitored deployment executed unintended orders for 45 minutes while the systems looked operational — roughly $440M lost. When a system's wrongness has no monitor, the cost is set by detection latency, not by the bug — Knight's was 45 minutes at market speed; this drill's is 19 days at lending speed.

Transfer

Which of our systems can serve garbage with a 200 status? What's its degradation SLI?

Make it a ticket: Pick the highest-decision-volume scoring path, write its imputation-rate and score-stability SLIs with numeric thresholds, and wire both alarms to on-call this sprint.

Drill 3 — The Average That Hid the Outage — the same blindness one layer over: healthy aggregates instead of healthy infrastructure metrics.
Drill 6 — The Health Check That Checked Nothing — "the process is up" vs. "the service works," at the endpoint level.
Drill 16 — The Upgrade That Broke Nothing (Visibly), theme 3 — the same silent-degradation pattern when the component is an LLM.