Pattern — Backpressure

The problem it solves

A fast stage feeds a slow stage through a queue that nobody bounds, so a spike converts directly into an unbounded backlog. The edge keeps returning success — 202 Accepted — while work it can never finish piles up until the process OOM-kills or the pipeline stalls. This is the shape behind Drill 84 (the order gateway that returned 202 to 15,000 orders a second while the processing tier drained 4,000) and Drill 81 (the in-memory enrichment buffer that grew until the heap died in a crash loop). Both fail because load flows one way only: the slow stage has no channel to reach back and say stop.

The pattern

Backpressure is a return channel: the saturated stage publishes its pressure (queue depth, in-flight concurrency), and every upstream stage acts on it — by blocking, or by rejecting fast with a first-class "slow down" response — so arrivals can never outrun service for long. Acceptance is coupled to capacity. When the system cannot finish work, it refuses it cheaply at the edge instead of accepting it and dropping it later.

How to apply it

Ground every number in the measured drain rate of the slow stage (Drill 84: about 4,000 orders/s).

• Token-bucket admission control at the edge. Refill rate = the measured drain rate, 4,000 tokens/s; burst = about 2x, 8,000. No token means an immediate HTTP 429 with a Retry-After header. Reject in under 5 ms — before the work item is even created. Never accept-then-drop.
• Bound the queue. Hard cap sized to a few seconds of drain: 30 s x 4,000/s = 120,000. At capacity the enqueue fails closed and the caller gets a 429. No queue is ever unbounded; prefer a durable, disk-backed broker (Kafka, SQS) so a slowdown yields a recoverable backlog, not an in-memory OOM.
• Concurrency limit per worker. Cap each processor at, say, 50 in-flight calls; when in-flight hits the cap, shed. This also protects shared dependencies (a pricing service, an execution venue) you cannot scale infinitely.
• Autoscale on the queue-depth SLI, not CPU. Add workers when depth > 70% of capacity for 2 min; remove when depth < 30% for 10 min; cap max workers to what the slowest downstream dependency can actually serve.

When to use it — and when not

Use it on any edge that returns success for work a slower tier must finish. Do not reach for it when the right answer is a bigger buffer for a transient spike that will drain on its own — backpressure trades throughput and caller convenience for survival, and rejecting work that a 10-second buffer would have absorbed just pushes retries and pain onto clients. It is also the wrong tool when the fix is genuinely more capacity (a real, sustained traffic increase) rather than protection against overload; backpressure keeps you alive, it does not make you faster. And it needs somewhere for the shed load to go — a caller that can back off, a priority tier to drop, or a durable store — otherwise you are just choosing where the data loss lands.

Signals it's working

• The 429 rate is a first-class SLI, and it rises under a spike — non-zero 429s are healthy shedding, not an outage. Flat-zero 429s under a 3x spike means backpressure is not actually wired.
• Queue depth stays bounded below its cap and the depth alarm fires at 70% before memory pressure, not after the OOM.
• p99 latency for admitted work stays flat through the spike (Drill 84: admitted orders still execute in about 300 ms) while excess is rejected at the edge.
• A game-day load test holds arrivals at 3x the drain rate for 10 minutes and asserts a bounded queue, honest 429s, and stable admitted-p99 — degradation, not collapse.

Seen in the wild

• Netflix concurrency-limits — each service measures its own latency, infers saturation, and sheds with a fast rejection instead of queueing unboundedly (adaptive backpressure).
• AWS S3 503 Slow Down and API gateway 429 Too Many Requests — the standardized "stop" signal a downstream returns so a spike does not become an outage.
• TCP flow control (the receive window) and reactive-streams request(n) demand signalling — backpressure built into the transport and the library.

See it

The CompareScene carries the point: an unbounded queue growing to hundreds of thousands and OOM-ing, versus a bounded 120,000-cap queue shedding at the edge with a 429 while admitted-p99 holds at 300 ms.

Named concepts: backpressure signalling, bounded queues, token-bucket admission control, concurrency limits.