Drill 19 — The Clause on Page 47

Theme: LLM & Agent Incidents · Format: 20-min drill · Difficulty: core

The story

A contract-review pipeline summarizes loan agreements and flags risky clauses. It works beautifully in testing. In production, a deal goes bad on a clause the system never flagged — a liability term on page 47 of a 60-page document. Investigation: the ingestion step truncated documents at the context limit, silently, keeping the first N tokens. The model summarized what it received, confidently and well.

⏸ Stop here and ask the room: nothing errored. Rank everything that failed here.

Run it — facilitator script

Timing: 2 min story · 10 min questions · 3 min reveal · 5 min transfer.

Where should truncation be impossible to do silently?

Listen for: At ingestion — every input is token-counted against the window before the call, and over-length becomes a routed, logged decision, never a default slice. And at the record — coverage ("analyzed 100% of input") stored as a queryable field on every output, so silence has nowhere left to live.

Weak signal: "The API or SDK should warn us" — some do, but relying on the provider is hope, not design; or "use a bigger context window" — which moves the cliff and keeps the silence.

Should the model answer at all when it hasn't seen the whole input?

Listen for: For risk-flagging, no — a confident partial answer is worse than no answer, because it discharges the human vigilance that would otherwise exist. Partial analysis is tolerable only when it's explicitly labeled as partial and every downstream consumer treats it that way.

Weak signal: "Some answer is better than nothing" — this incident is that answer; no recognition that a confident output transfers responsibility from the machine to the reader.

How do you handle documents that genuinely exceed the window?

Listen for: A named, tested strategy: chunk-and-merge with overlap and per-chunk provenance; hierarchical (map-reduce) summarization; retrieval over sections for targeted questions — or refuse and route to a human, which is a perfectly respectable engineering answer. Whichever it is, coverage is recorded and the strategy itself is in the test set.

Weak signal: "Just chunk it" with no merge or consistency story — clause-level risk can vanish at chunk boundaries too; or choosing a strategy without asking what its failure mode is.

The reveal

Every long document had been reviewed partially for months. Short documents — most of the test set — were fine, which is why testing missed it.

Why this happens — the mechanics

A context window is a hard cap on tokens per call, so when input exceeds it, something must give. SDK (Software Development Kit) conveniences and naive slicing default to keep-the-first-N-tokens — turning a capacity limit into a silent data drop. Worse, contracts front-load boilerplate: the pages most likely to be cut are the negotiated, riskiest ones at the back. Page 47 was not unlucky; it was structurally the most likely victim.

The model cannot know what it didn't see. There is no signal distinguishing "the document ends here" from "the input was cut here" — the truncated text simply is the document, as far as the model is concerned. So it summarizes what it received fluently and accurately, and that competence is the hazard: output quality is the reviewer's heuristic for input completeness, and here the heuristic lies.

Testing missed it because of stratification. Most documents are short, so aggregate accuracy looked excellent while the failure lived entirely in the long tail — literally. Months of partial reviews is what MTTD (Mean Time To Detect) looks like when the detector is a loss event: the most expensive monitor there is.

Diagrams

Problem — how the failure unfolds:

sequenceDiagram
    autonumber
    participant DOC as Document
    participant ING as Ingestion
    participant MDL as Model
    participant DT as Deal team
    DOC->>ING: 60-page loan agreement
    Note over ING: Truncates at the context<br/>limit - keeps the first<br/>N tokens (silent)
    ING->>MDL: First 40 pages only
    Note over DOC: Liability clause on page 47<br/>never reaches the model
    MDL-->>DT: Confident summary -<br/>no risk flag
    Note over DT: Months of partial reviews<br/>on every long doc (silent)
    DT--xDOC: Deal goes bad on the<br/>unseen clause
    Note over DOC,DT: Found by the loss,<br/>not by a monitor

Solution — where the guardrails go:

flowchart TD
    A["Document enters the pipeline"] --> B["Token accounting on every call"]
    B -->|"fits the window"| C["Model analyzes the full<br/>document"]
    B -->|"over length"| D["Explicit routing: chunk-<br/>and-merge or refuse and<br/>route to a human"]
    C --> E["Coverage recorded per<br/>document as a queryable<br/>field"]
    D --> E
    E -.-> F["Coverage SLI (Service<br/>Level Indicator) across<br/>the corpus"]
    F -.-> G["Any partial-coverage<br/>document surfaces<br/>immediately"]

What good looks like

Count tokens on every call with the model's own tokenizer, against the window minus prompt overhead — before dispatch, not after failure.
Over-length input is an explicit branch: chunk-and-merge, hierarchical summarization, or refuse-and-route — chosen per use case, written down, never defaulted.
Every output row records input_tokens, tokens_analyzed, and coverage_pct; anything under 100% is loud in the interface the deal team actually reads.
A corpus-level coverage SLI (Service Level Indicator) with an alarm on any partial-coverage document in flows where partial analysis is prohibited.
The eval set is stratified by length — including documents at 1x, 2x, and 5x the window — and asserts the routing behavior, not just summary quality.
For chunked flows, planted canary clauses at chunk boundaries (and on "page 47") verify that risk detection survives the split.
Contract-review outputs carry their coverage statement ("analyzed 100% of 60 pages"), so the claim being made is visible to the human relying on it.

The lesson

Input-completeness is a hard precondition: token-count accounting on every call, an explicit strategy for over-length inputs (chunk-and-merge, hierarchical summarization, or refuse and route to human), and coverage recorded per document ("analyzed 100% of input") as a queryable field. A confident answer about 60% of a contract is worse than no answer.

Named concepts: input validation, completeness SLIs, fail-explicit over fail-silent, stratified test sets (by length!).

Real-world echoes

None cited — silent-truncation incidents are common in practice but rarely public; they end up in loss reviews, not press releases.

Transfer

Which of our LLM inputs can exceed the window? What happens — provably — when they do?

Make it a ticket: For each LLM pipeline, feed a deliberately over-length document in staging and record what actually happens; any pipeline that answers without recording partial coverage gets a token-accounting and routing ticket this sprint.

Drill 16 — The Upgrade That Broke Nothing (Visibly) — the sibling silent-coverage failure, at the output boundary instead of the input.
Drill 17 — The Policy That Never Existed — confident text without grounding; here the grounding was cut, there it never existed.
Drill 1 — The Model That Scored Everyone the Same (Theme 1) — graceful degradation without a degradation SLI, the pattern this drill repeats at the token level.