Pattern — Timeout Budgets

The problem it solves

A request fans out through a chain of services, and each hop sets its own fixed timeout with no idea how long the caller has already been waiting. So a request that should have been abandoned after eight seconds keeps marching downstream — the edge gives up and returns 504, but three hops down a worker is still parked on a call for a client that left long ago. Worse, when a client sets no timeout at all (the library default is usually infinite), a slow dependency — one still returning 200 OK — holds a worker thread forever, a fixed pool fills in seconds, and unrelated features starve. This is the failure dramatized in Drill 76 — The Timeout That Wasn't Set: a KYC vendor went slow (not down), threads with no read timeout piled up on a shared 200-thread pool, and login and the dashboard went dark even though they never called the vendor. A timeout budget is the primitive that bounds how long the whole chain is allowed to wait, and makes every hop honor what's left.

The pattern

Attach a single deadline to the request at the edge — an absolute "give up by" time — and propagate it down every hop. Each outbound call gets timeout = min(remaining budget, its own local cap), so the deadline only ever shrinks as it travels. Any hop that sees the budget already spent fails fast instead of starting a call it can't finish. One request, one shrinking clock that every service reads.

How to apply it

Grounding the numbers in Drill 76 (edge request budget 8 s; vendor p99 400 ms; per-call caps connect 2 s, read 5 s):

1. Set the budget at the edge. The inbound signup handler stamps a deadline of now + 8000 ms. Everything the request touches must finish inside that window.
2. Carry it as remaining-milliseconds, not wall-clock. Propagate a relative grpc-timeout: 7900m / X-Deadline-Ms: 7900 header (or a Go context.WithTimeout), and recompute it at each hop from a local monotonic clock. Relative durations dodge cross-host clock skew; absolute wall-clock timestamps do not.
3. Compute each call's timeout as min(remaining − margin, cap). Before calling the vendor with remaining = 6200 ms: min(6200 − 100, 5000) = 5000 ms read timeout, connect capped at 2000 ms. Subtract a ~100 ms per-hop safety margin so the reply has time to travel back before the parent's own deadline fires.
4. Reject early below a floor. If remaining < 100 ms, don't start the call — return 504 immediately. Starting a 400 ms call with 50 ms left just wastes a worker and a downstream call.
5. Never fall back to the library default. An explicit read timeout of 5 s — a small multiple of the vendor's 400 ms p99 — replaces the infinite default that caused the outage. A call that can't finish inside the budget is abandoned, not parked.
6. Pair it with a bulkhead + breaker. A budget bounds one call's wait; put the vendor on its own 25-thread pool and a circuit breaker (opens at 50 % failures over a rolling 20-request window, stays open 30 s) so repeated deadline-exceeds trip it open and stop marching every request up to the dead-slow dependency.

flowchart LR
  A["Edge: budget = 8000 ms"] --> B["Hop 1 (remaining 7900)<br/>cap 5000 → wait 5000"]
  B --> C["Hop 2 (remaining 6200)<br/>min(6200-100, 5000) = 5000"]
  C --> D["Vendor call (remaining 4800)<br/>read 4700, connect 2000"]
  D --> E["remaining < 100 ms<br/>→ fail fast (504)"]

When to use it — and when not

Use it for synchronous, user-facing fan-out where a caller is actively waiting and there's a real end-to-end SLA. Do not reach for it when:

• The work is async / background — a batch pipeline or queue consumer has no waiting caller, so it should retry with backoff, not honor a stale sub-second deadline and drop work.
• The stream is long-lived — websockets, SSE, and gRPC streaming run indefinitely; an end-to-end deadline is meaningless (use idle/keepalive timeouts instead).
• The downstream write is non-idempotent and can't be safely cancelled — cancelling mid-charge can leave ambiguous state. Make the operation idempotent first, or a deadline just converts "slow" into "did it happen or not?"
• The operation is legitimately long (video encode, large export) — a tight budget guarantees failure; give those their own generous, isolated timeout.

The cost: budgets add plumbing to every hop, are sensitive to clock handling (propagate remaining ms, not timestamps), and turn a slow leaf into a wave of cancellations you have to expect.

Signals it's working

• No endpoint's p99.9 exceeds its configured budget. If signup's budget is 8 s, its tail latency is clamped near 8 s — no 30- or 60-second waits in the logs.
• Deadline-exceeded errors surface at the right layer — the leaf that overran, not a generic edge 504 — and each hop logs a monotonically decreasing remaining_ms.
• A count of "insufficient budget, skipped" rejections exists and is non-zero under load — proof the floor check is firing instead of starting doomed calls.
• Pool utilization stays bounded during a downstream slowdown (active ÷ pool size flat, not pinned at 100 %).
• A latency game day passes: inject 10 s of vendor latency in staging; callers time out at ~budget, the breaker opens, and only signups degrade while login and the dashboard stay green.

Seen in the wild

• gRPC deadlines — set once by the client as an absolute deadline and automatically propagated across every hop; the canonical implementation of this pattern.
• Go context.Context (context.WithTimeout / WithDeadline) — the deadline rides the context through every call and cancels the whole tree when it fires.
• Envoy / Istio route timeouts and the x-envoy-upstream-rq-timeout-ms header for per-request budget overrides.
• Google's deadline propagation (documented in the SRE book and Dapper) and Twitter Finagle's request deadlines / Linkerd — production RPC stacks that ship budgets by default.

See it

The ChainScene showing the budget shrink 8000 → 7900 → 6200 → 4800 ms across hops, then the fail-fast floor at < 100 ms, carries the whole mechanism in one line.

Named concepts: timeout budget, deadline propagation, client-side timeouts