Drill 36 — The $999 Pattern

Theme: Feedback Loops & Adversaries · Format: 20-min drill · Difficulty: intro

The story

Transactions over $1,000 trigger extra verification — a rule, stable for a year. An analyst plotting transaction amounts notices a sharp spike between $950 and $999 that didn't exist three months ago. The team raises the threshold to $1,500; within six weeks, the spike relocates to $1,400–$1,499.

⏸ Stop here and ask the room: two questions: what's happening, and how did they know the exact number?

Run it — facilitator script

Timing: 2 min story · 10 min questions · 3 min reveal · 5 min transfer.

Sketch the histogram on the whiteboard as you read — the spike is worth a thousand words. Pause at the blockquote and take guesses on "how did they know the exact number" before moving on.

"Is the threshold leaking — insider, probing, or trial-and-error — and how would you distinguish?"

Listen for: hypotheses tested with data. An insider leak produces bunching that appears abruptly and precisely; probing leaves fingerprints first — clusters of small, escalating transactions bracketing the line, spread across accounts, before the bunching starts. Check the timeline for the reconnaissance phase.

Weak signal: defaulting to "must be an insider" and proposing an HR investigation as the fix. It's the comfortable answer — and the reveal shows it wasn't needed.

"Is moving the threshold ever the fix?"

Listen for: recognition that moving the line just reruns the adversary's binary search from a new starting point — six weeks of relief, purchased by teaching the ring your reaction function. Legitimate uses exist: a deliberate, monitored move to confirm the probing hypothesis, or a move paired with jitter and layered signals.

Weak signal: treating the raise as the resolution ("we bought time"). Time was bought for the ring too — it now knows this team relocates thresholds instead of detecting gaming.

"What's the detection opportunity hiding in the spike itself?"

Listen for: the bunching is a confession in histogram form — mass accumulating just below any visible line identifies the accounts doing the accumulating. Alert on the shape, then pivot from the histogram to the account list: the spike is a pre-assembled ring investigation.

Weak signal: treating the spike purely as a nuisance to make disappear. The instinct to smooth the histogram back to normal is the instinct to destroy evidence.

The reveal

Systematic probing: the ring binary-searched the limit with small transactions and shared the finding. No insider needed — visible behavior is discoverable behavior.

Why this happens — the mechanics

A deterministic, visible threshold is an oracle the adversary can query for free. Every transaction is an experiment that returns one bit — extra verification triggered, or not — and a binary search over a $0–$2,000 range needs only about eleven such bits to pin the line exactly. The ring doesn't need an insider because the system itself answers questions honestly, one transaction at a time; the entire reconnaissance costs a handful of small purchases.

The economics then amplify the leak. One member runs the search; the number is shared; the whole ring's throughput moves to $999 — the discovery cost is amortized across every future transaction. Meanwhile the defenders' rule keeps "working" by its own definition: everything over $1,000 still gets verified, so the rule's metrics stay green. Each individual $980 transaction is unremarkable; the attack exists only at the population level, in the shape of the distribution — a place nobody was looking, because the rule was stable and quiet.

This is why raising the threshold fails structurally rather than incidentally. The move resets one parameter of a system whose mechanism — deterministic response to an observable input — is unchanged. The adversary reruns a cheap experiment; the defender pays migration costs, review costs, and six weeks of false comfort. Until the oracle stops answering cleanly (jitter) or stops being the only gate (layered signals the adversary can't observe), the equilibrium is always the same: mass just under the line.

Diagrams

Problem — how the failure unfolds:

sequenceDiagram
    autonumber
    participant FR as Fraud ring
    participant RE as Rules engine
    participant FT as Fraud team
    loop Probe the limit
        FR->>RE: Small test transactions<br/>bracket the line
        RE-->>FR: Verification fires above<br/>$1,000 - not below
    end
    Note over FR: Exact number shared<br/>across the ring (silent)
    FR->>RE: Transactions bunch<br/>at $950-999
    FT->>RE: Analyst spots the spike -<br/>threshold raised to $1,500
    FR->>RE: Bunching relocates to<br/>$1,400-1,499 in six weeks
    Note over FR,FT: The ring reruns its<br/>experiment faster than the<br/>team reruns theirs

Solution — where the guardrails go:

flowchart TD
    A["Verification threshold"] --> B["Adversaries probe the limit"]
    A --> J["Threshold jitter: probing<br/>yields noisy answers"]
    B --> D["Mass accumulates just<br/>below the line"]
    D --> BD["Bunching detector under<br/>every visible threshold"]
    BD -.-> AL["Alarm: gaming detected,<br/>ring investigated"]
    A --> L["Layered signals<br/>adversaries cannot observe"]
    L --> C["Skating under one line no<br/>longer clears the others"]

What good looks like

A standing bunching detector under every visible threshold: alarm when mass in the band just below the line exceeds k× its trailing-quarter baseline.
Distribution review as a quarterly calendar event — plot amount histograms around the top thresholds, because the adversary reruns their experiment continuously.
Threshold jitter: randomize the trigger point within a band, or trigger probabilistically near the line, so probing returns noisy answers and the binary search never converges.
Layered controls: pair the amount rule with signals invisible from outside — device linkage, velocity features, network relationships — so skating under one line doesn't clear the others.
When the bunching alarm fires, pivot to the accounts: the sub-threshold band is a pre-built case file of ring members, not just a chart anomaly.
A "how would we detect the gaming?" line item in the launch checklist for every new rule or visible control.
Monitor probe patterns directly: sequences of small, escalating transactions bracketing a threshold across related accounts are the reconnaissance fingerprint.

The lesson

Any deterministic, visible threshold will be found and skated under. Responses that work: the bunching itself is a detector (alert on mass just below any threshold — it's a confession in histogram form); add randomness/jitter so probing yields noisy answers; layer thresholds with signals adversaries can't observe. And ask "how would we detect the gaming" for every rule you ship.

Named concepts: adversarial adaptation, detection of gaming, randomized controls, defense in depth.

Real-world echoes

Structuring under the $10,000 reporting threshold — US banks must file currency transaction reports for cash transactions over $10,000; splitting cash into just-under-threshold deposits ("structuring," or "smurfing") became so predictable that the evasion pattern itself was criminalized and is now a standard anti-money-laundering detection target. The bunching is the detector, at industry scale.

Transfer

Plot the distributions just below our three biggest thresholds. Any bunching? (Do this quarterly — the adversaries rerun their experiment continuously.)

Make it a ticket: Identify the three highest-impact visible thresholds; ship a bunching alarm under each and put the histogram review on the quarterly calendar with a named owner.

Drill 37 — The Metric That Got Gamed — the same optimization pressure aimed at an internal metric instead of an external rule.
Drill 41 — The Coached Application — the adversary graduates from probing one threshold to curating entire applications.
Drill 46 — The Poisoned Feedback (Theme 7) — adversaries who attack the learning loop itself rather than the decision boundary.