AIM-01-011-P · Primer — Sampling Bias

The idea in one sentence

Sampling bias is when the data you collected isn't a fair picture of the world you want to predict, so your model quietly answers a different question than the one you asked.

Why it exists

You almost never get to see the whole population — you see whoever showed up, whoever responded, whoever survived long enough to be recorded. The classic story is World War II bombers: engineers wanted to add armor where returning planes had the most bullet holes, until Abraham Wald pointed out those were the planes that made it back. The holes marked the survivable hits; the armor belonged where the returning planes had no holes, because planes hit there never came home. The data was real and carefully measured — it just came from a filtered sample, and the filter, not the truth, was doing the talking.

How it works

The mechanism is a mismatch between two distributions: the distribution your training data was drawn from, and the distribution your model will actually face in production. Say you build a loan-default model, but you can only train on people the bank already approved — you never observe how rejected applicants would have behaved. Your model learns the pattern of approved borrowers and calls it the pattern of all borrowers. Concretely: if your training set is 5% defaults among a pre-screened crowd, but real applicants default at 15%, the model's whole notion of "risky" is calibrated to the wrong world, and every threshold you set downstream inherits that skew. The bias lives in how the rows were chosen, not in any single value, which is why you can't spot it by staring at the numbers.

See it — the selection filter, not the truth, shapes what the model learns

flowchart TD
    A["True population<br/>(all applicants)"] --> B{"Selection filter<br/>(only approved)"}
    B -->|kept| C["Sampled data<br/>5% defaults"]
    B -->|dropped| D["Never observed<br/>(rejected)"]
    C --> E["Model learns<br/>'risky' from 5%"]
    E --> F["Deployed on<br/>real applicants<br/>15% defaults"]
    F --> G["Mismatch:<br/>wrong question<br/>answered"]

The one gotcha

The gotcha is that your data answers a different question than the one you asked — and the model looks perfectly healthy while doing it. High accuracy on a biased sample just means you nailed the wrong population; the metrics are honest about the sample and silent about the gap. You cannot fix this after the fact by tuning the model, because the missing data was never collected. The only real fixes are upstream: change how you sample, reweight to match the target population, or explicitly model the selection process.

See it (visualization plan)

CompareScene or HistogramScene: put two distributions side by side — a wide, true population histogram in blue and a narrow, shifted "who we sampled" histogram in amber — with a badge showing the model trained on the amber one. Then a CardsScene for the bomber example: "add armor where the holes are" (wrong) vs "add armor where the holes aren't" (right), captioned "the planes with holes there never came back."

The one-line takeaway

Ask who's missing before you trust who's there.

Where you'll meet it

Every survey, A/B test, and web-scraped dataset; scikit-learn's train_test_split won't warn you that your rows came from a biased pipe, and PyTorch will happily train on a skewed loader. It shows up as class imbalance's sneakier cousin, in survey non-response, in survivorship bias in finance backtests, and in feedback loops where a recommender only ever sees clicks on items it already showed.

Named concepts: sampling bias, selection bias