BitBootcampBitBootcamp ← all explainers 🔄 Review ⚙ Settings

Drill 52 — The Proxy Variable

Theme: Compliance, Privacy & Model Governance · Format: 20-min drill · Difficulty: advanced

The story

A client's compliance team commissions a fair-lending analysis of the credit model. The analysis finds that one heavily-weighted feature — "distance from nearest branch" — correlates strongly with a protected class in that geography: branch locations trace decades of neighborhood history, and neighborhood traces demographics. The disparate impact numbers are bad. The feature was never tested for this, because it was "obviously" not demographic — it's just geometry, a distance in kilometers.

⏸ Stop here and ask the room: the feature never mentions race, age, or gender. Is the model discriminating?

Run it — facilitator script

Timing: ~2 min story · ~10 min questions · ~3 min reveal · ~5 min transfer.

Read the story, pause at the blockquote, and force a yes/no vote before discussion — the split in the room usually maps exactly to who has and hasn't internalized impact-based doctrine. Then work the questions.

How can a feature encode a protected class without naming one?

Listen for: correlation is all a model needs — it optimizes for signal wherever it hides, with no concept of which correlations are lawful; geography as a demographic carrier, because branch siting, housing policy, and redlining history wrote demographics into the map decades ago; and the doctrinal point that under impact-based law, what the feature measures matters, not what it's called.

Weak signal: "we removed the protected attributes, so we're compliant." Removing the labeled columns while keeping their correlates is precisely the failure in the story — sometimes called fairness through unawareness, and it doesn't work.

Whose job was it to test for this, and at what point in the model's life?

Listen for: the model owner, at design time, with fairness sign-off as a deployment gate — plus ongoing monitoring in production, because impact drifts with populations and portfolios. The sharp observation to reward: the client's compliance team finding it means our control ran two years late and in someone else's building.

Weak signal: pointing at legal or compliance alone. The test requires training data, model access, and statistics — it is a data-science deliverable that compliance consumes, not a memo compliance writes.

What would a design-time check for proxy features look like — concretely, in a day?

Listen for: mechanics, not vibes — estimate protected-class membership where it isn't collected (e.g., BISG, Bayesian Improved Surname Geocoding); compute per-feature correlation or mutual information against those estimates; rank features by weight times correlation; compute model-level adverse impact ratios on training and holdout data; and for flagged features, search for a less-discriminatory alternative before launch. All scriptable, all runnable in a day.

Weak signal: hand-waving — "run a bias audit" with no mechanics — or the claim that testing is impossible because protected attributes aren't in the data. Estimation methods exist precisely for that case, and regulators use them.

The reveal

The model is pulled and rebuilt — months of work, a client relationship strained, and exposure under fair-lending law for every decision the model made. The bitter part: the exact analysis that found the problem could have been run in one day at design time, on the training data, before a single applicant was scored.

Why this happens — the mechanics

Proxies exist because history is already encoded in the features. Branch locations weren't placed randomly — they trace decades of neighborhood wealth, housing policy, and redlining; neighborhood composition traces demographics; so "distance from nearest branch" arrives at the model pre-loaded with protected-class signal, wearing the costume of geometry. A model trained to predict risk will find and use that signal because correlation is the only thing it can see — it has no concept of which correlations are legally radioactive. The feature's innocent name is a fact about human labeling, not about the joint distribution the model actually learns from.

The legal frame makes intent irrelevant. Under ECOA (the Equal Credit Opportunity Act — the US law prohibiting credit discrimination), liability attaches through two doctrines: disparate treatment (using a protected attribute directly) and disparate impact (a facially neutral practice that disproportionately harms a protected group without adequate business justification). The second is the one that catches proxies — nobody intended to discriminate, no protected attribute appears anywhere, and the exposure is real regardless. This is why "obviously neutral" is a category error: neutrality is an empirical property of a feature's correlations in a specific population, measurable only by testing, never by inspection of the feature's name.

And the exemption mechanism is social, not technical. Features get waved past fairness testing exactly when they look neutral — which means the testing regime skips precisely the features most likely to be undetected proxies, while the obviously-sensitive ones were never proposed in the first place. The inverted cost curve does the rest: the one-day analysis not run at design time returns two years later as a client-run fair-lending finding, a model rebuild, and legal exposure on every decision in between.

Diagrams

Problem — how the failure unfolds:

sequenceDiagram
    autonumber
    participant FT as Feature
    participant MD as Model
    participant PG as Protected group
    participant CC as Client compliance
    Note over FT: Distance from nearest<br/>branch - looks like<br/>pure geometry
    FT->>MD: Added untested - obviously<br/>non-demographic (silent)
    Note over FT: Branch geography encodes<br/>neighborhood - a proxy<br/>(silent)
    MD->>MD: Weights the feature heavily
    MD--xPG: Decisions skew for two<br/>years (silent)
    CC->>MD: Fair-lending analysis
    CC-->>FT: Heavy weight correlates<br/>with protected class
    Note over CC,FT: Disparate impact is bad -<br/>model pulled and rebuilt

Solution — where the guardrails go:

flowchart TD
    A["Feature proposed: distance<br/>from nearest branch"] --> T["Design-time disparate<br/>impact test across ALL<br/>features"]
    T --> P["Proxy correlation with<br/>protected classes measured"]
    P --> L["Less-discriminatory<br/>alternative searched<br/>before launch"]
    L --> G["Fairness sign-off is a<br/>deployment gate"]
    G --> D["Model deployed"]
    D -.-> M["Ongoing fairness<br/>monitoring per segment"]
    M -.-> AL["Impact-ratio drift pages<br/>compliance early"]
    AL --> R["Feature reworked in a day<br/>— not the model in a<br/>quarter"]

What good looks like

The lesson

Under ECOA (the Equal Credit Opportunity Act — the US law prohibiting credit discrimination), impact is what counts, and a feature doesn't need to name a protected class to encode one. So the control is disparate impact testing across all features — pre-launch and ongoing — with no feature exempted for being "obviously" neutral, followed by a search for less-discriminatory alternatives that preserve predictive power, and fairness monitoring per segment in production, because impact drifts as populations and portfolios do.

Named concepts: proxy discrimination, disparate impact testing, ECOA (Equal Credit Opportunity Act) compliance, less-discriminatory alternatives.

Real-world echoes

Transfer

Take our current model's top ten features by weight. Which have actually been tested for proxy correlation with protected classes — and which were waved through because they were "obviously" neutral? The second list is the audit.

Make it a ticket: Pull the top ten features by weight, run the proxy-correlation screen and adverse-impact-ratio tests against estimated protected-class membership, and file the results — pass or fail — into the model's validation record.

Related