Drill 45 — The Vendor's Bad Week

Theme: Security & Trust Boundaries · Format: 20-min drill · Difficulty: core

The story

Your document-processing vendor — the one every applicant's bank statements pass through — discloses a breach: an attacker had access to their logging environment for three weeks. Their logs, it turns out, contained full request payloads. Your customers' financial documents were in someone else's logs, and now in someone else's hands. Your name is on the breach notification letters, not theirs.

⏸ Stop here and ask the room: what should have been asked before the contract was signed, and what do you do this week?

Run it — facilitator script

Timing for a 20-minute session: 2 min story · 10 min staged questions · 3 min reveal · 5 min transfer.

What data minimization was possible? (Did the vendor need full statements?)

• Listen for: field-level thinking — the vendor's output uses a handful of extracted fields, so that's what it should receive; account numbers and SSNs tokenized before transmission; images masked where the integration allows. The strongest answers notice that minimization is the one control that still works after the vendor fails.
• Weak signal: "the integration needs the whole document" asserted without anyone having checked what the vendor's output actually consumes. "Send everything" is a default someone accepted, not a requirement someone verified.

What did the security questionnaire actually verify vs. record?

• Listen for: the distinction between attestation and behavior — a SOC 2 (Service Organization Control 2, an auditor's point-in-time attestation of control design and operation) answers "were the controls designed properly during the audit window," not "what did debug logging write to disk last quarter." Questionnaires record answers; evidence, log-handling specifics, and audit rights verify them.
• Weak signal: "they had a clean SOC 2, we did our diligence" — outsourcing judgment to a snapshot. If the room treats certification as a guarantee, they've just re-signed the same contract.

What's the incident plan when the incident is inside a vendor?

• Listen for: an upstream-breach runbook that exists before the disclosure — who calls whom, a data-sent inventory that scopes exposure in an hour, the notification decision tree, and a per-vendor exit or rotation plan. Someone should say out loud that customers and regulators will treat this as your breach.
• Weak signal: the room starts designing the plan live, in the drill — which is the honest tell that there isn't one. Improvising the runbook during the incident is the failure mode, not the answer.

The reveal

The vendor's SOC 2 report was current and clean. It attested to their controls' design — not to what their debug logging did under pressure last quarter.

Why this happens — the mechanics

Your obligations travel with the data; your controls don't. The moment full statements cross the wire, they're governed by the vendor's logging defaults, debug habits, and retention hygiene — and vendors log full payloads for the same innocent reason your own teams do: debugging is easier with everything. Nothing in a standard integration ever surfaces what the vendor's logs contain, so the gap between "what we sent them" and "what they're storing, where, for how long" stays invisible until someone hostile finds it first.

Attestation gaps are structural, not scandalous. A SOC 2 (Service Organization Control 2) report is an auditor's snapshot of control design and operation over a window — it cannot see the debug flag an engineer enabled under incident pressure after the audit closed, and it was never designed to. The failure isn't that the certificate lied; it's that the buyer read a snapshot as a live feed. Questionnaires compound this: they record what the vendor says, and almost never ask the one question that mattered here — what do your logs contain when a request fails?

And accountability doesn't outsource. Customers handed their statements to you; regulators licensed you; the notification letters carry your name, because from the outside there is no vendor — there is only your promise about the data, kept or broken. That's why minimization is the load-bearing control: it's the only one that keeps working when everything on the vendor's side of the boundary fails at once.

Diagrams

Problem — how the failure unfolds:

sequenceDiagram
    autonumber
    participant YOU as You
    participant VN as Vendor
    participant LG as Vendor logs
    participant AT as Attacker
    participant CU as Your customers
    YOU->>VN: Full bank statements sent<br/>for processing
    VN->>LG: Full request payloads<br/>logged (silent)
    AT->>LG: Inside the logging<br/>environment for three<br/>weeks (silent)
    Note over AT,LG: Customers' financial<br/>documents exfiltrated
    VN->>YOU: Breach disclosed - SOC 2<br/>was current and clean
    Note over YOU: Attestation covered design,<br/>not behavior under pressure
    YOU->>CU: Breach notification letters -<br/>your name on them

Solution — where the guardrails go:

flowchart TD
    A["Document needs vendor<br/>processing"] --> B["Data minimization: redact<br/>and tokenize before<br/>transmission"]
    B --> C["Vendor receives only the<br/>fields it actually needs"]
    D["Contract: log-handling and<br/>breach-notification terms<br/>with teeth"] --> C
    C --> E["Vendor breach happens anyway"]
    E --> F["Upstream-breach incident<br/>runbook triggers"]
    F -.-> M["Exposure scoped fast:<br/>tokens leaked, not<br/>statements"]
    F --> G["Exit and rotation plan per<br/>critical vendor, ready in<br/>advance"]

What good looks like

• A data-sent inventory per vendor — the exact fields transmitted, refreshed quarterly, owned by a name. It's the document that scopes a breach in an hour instead of a week.
• Tokenize before transmission — account numbers and SSNs replaced with vault tokens the vendor round-trips; when their logs leak, they leak tokens, not statements.
• Redact down to what the vendor consumes — if their output uses six fields, they receive six fields; every extra field shipped is exposure purchased for zero business value.
• Contract terms with teeth — payload-logging prohibitions, log-retention maximums, breach notification on a fixed clock, and audit rights — verified with evidence, not questionnaire checkboxes.
• Ask the debug-log question in diligence — "what do your logs contain when a request fails, and who can read them?" — the question this story turns on and no attestation answers.
• An upstream-breach runbook, exercised — named owner, notification decision tree, data-sent inventory attached, run as a tabletop annually so the first rehearsal isn't the real one.
• An exit plan per critical vendor — a second vendor warm or a documented migration path, so "leave" is an available decision rather than an empty threat during renegotiation-by-incident.

The lesson

Your trust boundary includes every vendor that touches your data — customers don't distinguish, and neither do regulators. Practical controls: send the minimum (redact/tokenize before transmission), contractual breach-notification and log-handling terms with teeth, an exit/rotation plan per critical vendor, and your own incident runbook for "the breach is upstream" — because certification is a snapshot, not a guarantee (Drill 39's dependency lesson, at maximum stakes).

Named concepts: third-party risk, data minimization, vendor security posture vs. attestation, supply-chain incident response.

Real-world echoes

MOVEit, 2023 — a vulnerability in Progress Software's MOVEit Transfer tool was exploited at scale, and hundreds of downstream organizations — many in financial services — sent breach letters for customer data that had been sitting in a vendor's file-transfer layer. Most affected customers had never heard of MOVEit; the letters carried names they had heard of. Same shape as this drill: the breach was upstream, the accountability wasn't.

Transfer

For our top three data-touching vendors: exactly what fields do we send, and could any be tokenized or dropped without breaking the integration?

Make it a ticket: For our single most data-heavy vendor, produce the exact field inventory we transmit and mark each field keep / tokenize / drop — the deliverable is the redaction change request, filed against the integration.

Related

• Drill 48 — The Logs That Remembered Everything — the same payloads-in-logs failure inside your own walls; this drill is that one, outsourced.
• Drill 43 — Shadow AI — data landing in a third party's logs via an individual's browser tab instead of a signed contract; same boundary, no procurement trail at all.
• Drill 39 — The Vendor's Silent Update, theme 6 — the dependency lesson this drill escalates to maximum stakes: the thing you depend on failing is a when, and the plan has to predate it.