Drill 87 — Train-Serve Skew

Theme: Model Lifecycle & MLOps · Format: 20-min drill · Difficulty: core

The story

A credit-decisioning model has been the quiet workhorse of a lender's underwriting for a year. Its single most important feature, by a wide margin, is income_to_debt — annual income divided by annualized debt service. In the training pipeline this feature is computed in Python over the data warehouse: annual income over monthly_debt × 12, with missing debt imputed to the training-set median. The model learned that borrowers with a ratio above roughly 4 are good risks.

The scoring service that runs at application time is a different codebase, in a different language, owned by a different team. It computes income_to_debt too — as annual income over monthly_debt. No × 12. And where debt is missing, it defaults to zero, which makes the ratio explode to infinity and get clipped to the feature's max. Nobody wrote this divergence on purpose; the two implementations were written eighteen months apart from the same one-line description in a design doc, and they drifted.

So the model was trained on a feature centered around 4 and, in production, is fed a feature centered around 48 — the same borrowers, a number twelve times too large, with a slug of maxed-out values where debt was null. Accuracy on the metrics anyone watches looks fine: the service returns a score, latency is flat, errors are zero. Approvals have crept up over the year, which everyone read as a healthier applicant pool.

⏸ Stop here and ask the room: the model is "working" — it returns a score for every application. How would anyone on this team ever find out the score is being computed on a feature the model never saw in training?

Run it — facilitator script

Timing for a 20-minute session: 2 min story · 10 min staged questions · 3 min reveal · 5 min transfer.

The model is up and scoring — so what is actually broken?

• Listen for: the model is being served inputs from a different distribution than it trained on — training-serving skew. The score is a confident number computed on garbage; a valid-looking output is not a correct one. The failure is in the feature layer, upstream of the model, and it is silent because nothing about a 12× feature is malformed enough to throw.
• Weak signal: "the model degraded / needs retraining." The model didn't change at all — the inputs did. Retraining on the skewed serving feature would just bless the bug.

Why didn't any monitor catch a feature that's 12× off?

• Listen for: because nobody compares the distribution of features at serving time against the distribution at training time. You need to log served feature values and monitor them — mean, spread, null rate, and a population-stability check against the training snapshot — per feature. A feature whose serving mean is 12× its training mean should page on day one.
• Weak signal: "add more accuracy metrics" — you often can't measure accuracy live (labels arrive months later, for approved loans only). The catchable signal is the input distribution, now, not the outcome later.

Two implementations of the same feature — how do you make that impossible?

• Listen for: a single source of truth for the computation — a shared feature definition / feature store that serves the same transformation to both training and serving, so parity is structural, not a coincidence maintained by two teams reading the same prose. Failing that, a contract test that computes the feature both ways on the same inputs and asserts they match.
• Weak signal: "write better documentation" or "have the teams sync" — the divergence happened because parity depended on two humans keeping two codebases aligned by hand.

The reveal

An analyst chasing the year's approval creep pulled the served feature values and found income_to_debt sitting around 48 instead of 4. The model hadn't drifted; it had been fed a mis-scaled feature since the day the serving path shipped. The fix was to compute the feature once — in a shared feature pipeline consumed by both training and serving — and to log every served feature so its distribution could be compared to training. The × 12 and the null-handling divergence both vanished the moment there was one implementation instead of two. Approvals snapped back; the "healthier applicant pool" had been a scoring artifact all along.

Why this happens — the mechanics

A model is a function fit to the distribution of its training features. It carries no memory of what those features meant, no units, no sanity range — it interpolates whatever vector you hand it and returns a confident number. Feed it the same feature on a different scale and it will not error; it will quietly extrapolate into a region it never learned, and the score is meaningless in a way that no HTTP status can express. This is the defining property of ML failures: the system is behaving exactly as coded and still producing garbage, because correctness lives in the data, not the control flow.

The organizational root is that training and serving are almost always two different worlds — a batch pipeline in one language over the warehouse for training, a low-latency service in another language over live data for serving. Any feature that isn't computed from a single shared definition is really two features that happen to share a name, and they drift the instant either side is touched. Training-serving skew is the most common way a model that tested beautifully offline underperforms in production, and it is invisible to every infrastructure metric because the infrastructure is healthy — it's the meaning that's broken.

Diagrams

Problem — how the skew is introduced:

sequenceDiagram
    autonumber
    participant TP as Training pipeline<br/>(Python, warehouse)
    participant M as Model
    participant SS as Scoring service<br/>(other lang, live)
    participant DEC as Underwriting
    Note over TP: income / (monthly_debt×12)<br/>null → median  (mean ≈ 4)
    TP->>M: Train on ratio ≈ 4
    Note over SS: income / monthly_debt<br/>null → 0  (mean ≈ 48)
    SS->>M: Score on ratio ≈ 48
    M-->>DEC: Confident score on a<br/>feature it never saw
    Note over SS,DEC: 200 OK, zero errors,<br/>approvals creep for a year

Solution — where the guardrails go:

flowchart TD
    A["One shared feature<br/>definition (feature store)"] --> B["Same transform in<br/>training AND serving"]
    C["Log served feature<br/>values in production"] --> G1["Skew monitor: serving vs<br/>training distribution"]
    G1 -.-> AL["Page when mean/spread/<br/>null-rate diverges"]
    B --> G2["Parity contract test<br/>in CI"]
    G2 -.-> AL
    AL --> H["Block deploy / roll back<br/>the feature path"]

What good looks like

• A single feature definition consumed by both training and serving (a feature store or a shared library), so parity is structural — one implementation, not two kept in sync by hand.
• Serving-time feature logging: log the actual feature vector scored in production, sampled, so its distribution is observable.
• A training-serving skew monitor per feature: compare serving mean, standard deviation, and null rate against the training snapshot; page when any top-importance feature's serving mean drifts beyond, say, 2× its training value or its null rate exceeds baseline by 3×.
• A population-stability check (PSI or KS) on each served feature against training — warn at PSI 0.1, page at 0.25.
• A parity contract test in CI: compute each feature the training way and the serving way on a shared fixture set and assert they match within tolerance; a failing parity test blocks the deploy.
• Post-deploy feature verification: after any change to a feature pipeline or the serving service, auto-check served distributions against training in the first hour — the same discipline you'd apply to a code deploy.

The lesson

A model only knows the distribution it was trained on; serving it a feature computed differently is feeding it a stranger. Compute each feature once and monitor the served distribution against training — training-serving skew is silent, and only a distribution comparison, not an error rate, will surface it.

Named concepts: training-serving skew, feature parity, feature store, feature distribution monitoring.

Real-world echoes

• Google's "Rules of Machine Learning." Google's widely-cited ML engineering guide names training-serving skew explicitly (Rules 29, 31, 37) as a top cause of models that look good offline and underperform live, and prescribes exactly this drill's fix: log features at serving time and compare them to training, and share the feature-generation code between the two paths.

Transfer

For your highest-stakes model, is each feature computed by one piece of code, or by a training implementation and a separate serving implementation? Pick the top feature by importance and prove the two paths produce the same number on the same input.

Make it a ticket: log served feature values for the top model, stand up a skew monitor comparing serving-vs-training distribution for the top-10 features by importance, and add a parity contract test to CI — page on any top feature diverging beyond a written threshold.

Related

• Drill 1 — The Model That Scored Everyone the Same — the same silent-scoring blindness one layer over: a degraded input the infrastructure can't see.
• Drill 93 — The Metric That Lied — the evaluation-side cousin: an offline number that didn't survive production.
• Drill 88 — The Drift Nobody Watched, theme 15 — skew is a static mismatch; drift is the same problem arriving slowly over time.
• Pattern #138 — Schema / Data Contracts — the reusable enforcement for feature parity.