Drill 43 — Shadow AI

Theme: Security & Trust Boundaries · Format: 20-min drill · Difficulty: core

The story

A DLP alert flags it: an operations analyst has been pasting customer account histories — names, balances, SSNs — into a free public chatbot to "summarize the case faster." For four months. The analyst is one of the most conscientious people on the team; the chatbot was genuinely making their work better. The data is now in a third party's logs under a consumer terms-of-service.

⏸ Stop here and ask the room: before the disciplinary instinct: why did this happen, and what does punishing it teach the other forty analysts?

Run it — facilitator script

Timing for a 20-minute session: 2 min story · 10 min staged questions · 3 min reveal · 5 min transfer.

Facilitator caution: run this one with amnesty framing, and say so out loud before the first question — the goal is surfacing shadow usage, not confession hunting. The moment the room smells a hunt for names, the honest answers go underground, which is exactly the failure mode the drill is about. If someone volunteers their own shadow tool, thank them; that disclosure is the drill working.

What need was the analyst meeting?

Listen for: case-load pressure plus a genuinely effective tool plus no sanctioned alternative — the analyst wasn't cutting corners, they were closing cases. The strongest answers treat the usage as a demand signal: four months of daily use is the clearest product-requirements document the organization will ever receive.
Weak signal: "they wanted a shortcut" — framing an unmet need as a character flaw. If the room settles on motive instead of mechanism, the other forty analysts' tools stay hidden.

What's the disclosure obligation?

Listen for: PII (Personally Identifiable Information) now sits in a third party's logs under consumer terms — someone should name legal and privacy as the first call, the regulatory notification analysis that follows, and the fact that the clock arguments start from discovery, which is today.
Weak signal: "the chatbot probably doesn't keep the data" — a compliance position built on the unread terms-of-service of a free consumer product. Nobody vouching for those terms has read them.

If you ban it, what happens next — compliance, or better hiding?

Listen for: prohibition without provision moves usage where you can't see it — personal phones, home laptops, paraphrased pastes that slip past DLP (Data Loss Prevention). The durable answer is the paved road: a sanctioned tool that wins on merit, so hiding stops being worth the effort.
Weak signal: confidence that a policy memo changes behavior — "we'll send a reminder about acceptable use." The need is untouched by the memo; only the visibility changes.

The reveal

Half the team, quietly asked, admits to something similar. The organization provided no sanctioned way to get AI help with the workload — so people built their own, invisibly.

Why this happens — the mechanics

Shadow AI — unsanctioned use of external AI tools on work data — is demand-side economics, not misconduct. The analyst's performance is measured in cases closed; a tool that meaningfully cuts handling time is worth real risk to a person who never sees that risk priced. The exposure lands on the organization, the benefit lands on the analyst's queue, and nothing in between requires bad intent — only a gap between workload and tooling that someone eventually fills from the open internet.

Prohibition without provision changes visibility, not volume. Ban the chatbot and the case load — the thing generating the demand — is untouched, so usage continues one step further from view: personal devices, home networks, retyped summaries that no DLP pattern will ever match. Each enforcement action teaches better hiding, and hidden usage is strictly worse — same data leaving the boundary, zero chance of redaction, logging, or an honest count of the exposure.

The four-month gap is the third mechanic: consumer AI tools don't announce themselves to any inventory. There's no procurement record, no integration, no vendor questionnaire — the "integration" is a browser tab. That leaves exactly two discovery channels, DLP and honesty, and an organization that punishes honesty has chosen to rely on DLP alone — a pattern-matcher, against people it has just trained to paraphrase.

Diagrams

Problem — how the failure unfolds:

timeline
    title Four months of shadow AI
    section Month 1
        The pressure : Case load keeps climbing : No sanctioned AI tool exists
        The find : Free public chatbot helps : Cases close faster
    section Months 2-4
        Every day : Customer PII pasted in (silent) : Names, balances, SSNs
        Every day : Data lands in third-party logs : Consumer terms apply (silent)
    section Month 4
        The alert : DLP alert finally fires : Disclosure question now open

Solution — where the guardrails go:

flowchart TD
    A["Analyst under case-load<br/>pressure"] --> B["Paved road: sanctioned<br/>model under enterprise<br/>terms"]
    C["Data classification rules<br/>people can actually follow"] --> B
    B --> D["Easier than the contraband<br/>path — integrated, fast,<br/>allowed"]
    D --> E["Built-in redaction strips<br/>PII before the model sees<br/>it"]
    E --> F["Usage logged and reviewable"]
    B -.-> M["DLP still watches for off-<br/>road traffic"]
    M -.-> AL["Shadow usage flagged in<br/>days, not months"]

What good looks like

A sanctioned endpoint under enterprise terms — no training on your inputs, contractual retention limits, audit logging — deployed before the ban, not announced alongside it.
Redaction in the path, not in the policy — a gateway strips names, SSNs, and account numbers before any prompt leaves the boundary, so the safe path is safe by construction rather than by discipline.
A paved road that is genuinely faster — single sign-on, embedded in the case tooling, no copy-paste round trip. If the sanctioned path adds clicks over the contraband one, expect contraband.
Data classification on one page — three tiers people can recall under deadline pressure ("green: paste freely / yellow: redact first / red: never leaves"), not a forty-page standard nobody opens mid-case.
DLP (Data Loss Prevention) tuned to AI egress — known chatbot domains and paste-shaped uploads flagged in days, and treated as discovery signal about unmet needs, not as case files on individuals.
Quarterly amnesty surveys — "what tools are you using that we don't know about?", no names attached, results feeding the sanctioned-tooling roadmap.
Adoption as the KPI (Key Performance Indicator) — measure paved-road usage growth, not violations caught. If adoption stalls, the sanctioned tool lost a fair fight and needs fixing, not defending.

The lesson

Shadow AI is unmet demand, not misconduct. The durable fix is a sanctioned path — an approved model behind enterprise terms, with logging and redaction — that's genuinely easier than the contraband one, plus data-classification rules people can actually follow. Prohibition without provision just moves usage where you can't see it (Drill 31's visibility lesson in security clothing).

Named concepts: shadow IT/AI, data classification, DLP, paved-road security (make the safe path the easy path).

Real-world echoes

Samsung, 2023 — engineers pasted confidential source code and internal meeting notes into ChatGPT to speed up their own work, and the company responded by restricting generative-AI use on corporate devices. The employees weren't exfiltrating — they were debugging and summarizing, faster. Which is the drill's whole point: the most conscientious people find the most effective tools first.

Transfer

What AI tools would our team reach for if unsupervised — and is our sanctioned alternative actually better, or just official?

Make it a ticket: Stand up (or honestly evaluate) the sanctioned AI path, then run one amnesty survey this month — the deliverable is the anonymized list of unmet needs it surfaces, filed as roadmap input with no names attached.

Drill 45 — The Vendor's Bad Week — the same data-in-someone-else's-logs problem, entered through a signed contract instead of a browser tab.
Drill 48 — The Logs That Remembered Everything — when the third party holding unminimized data is your own observability stack.
Drill 31 — The Postmortem That Named Names, theme 5 — the visibility lesson this drill re-runs in security clothing: punish honesty and behavior doesn't stop, it just leaves your field of view.