BitBootcampBitBootcamp ← all explainers 🔄 Review ⚙ Settings

Pattern — Exponential Backoff + Jitter

The problem it solves

A retry is a good idea in isolation and a bomb in aggregate. In Drill 80, a card platform runs about 400 pods that all call one ledger on the hot path; a routine 2-second GC pause makes every in-flight call time out at the same instant, and every client retries immediately and in lockstep. The ledger comes back to normal load plus the whole backlog of first retries plus new traffic — roughly 5× real demand — so it stays pinned. The trigger cleared in seconds; the outage ran for 18 minutes and survived restarts. That is a metastable failure: the retries themselves are now the load, a positive-feedback loop decoupled from the thing that started it. The fix is not more capacity — added replicas just invite the synchronized wave to land somewhere new. It is retries that spread out in time and are capped as a fraction of traffic.

The pattern

Never retry on a fixed schedule. Compute a growing backoff — base · 2^attempt, clamped to a cap — and then pick the actual sleep uniformly at random inside [0, backoff] (full jitter). Exponential growth stops a struggling dependency from being hammered on a tight loop; the random jitter de-phases clients so two that failed together do not retry together. Wrap the whole thing in a retry budget (a token bucket that caps retries at a small share of real traffic) and a circuit breaker (which stops calling a dependency that is plainly down), so a blip can add a bounded few percent of load instead of a 400% wave.

How to apply it

  1. Backoff schedule — real numbers. base = 100ms, multiplier = 2, cap = 10s, max_attempts = 5. Attempt n waits min(cap, base · 2^n) → 100ms, 200ms, 400ms, 800ms, 1.6s… clamped at 10s. These are exactly the gRPC retryPolicy knobs (initialBackoff: 0.1s, maxBackoff: 10s, backoffMultiplier: 2).
  2. Full jitter — the load-bearing line. Do not sleep for the backoff itself; sleep for random_uniform(0, min(cap, base · 2^n)). Equal-jitter (half + random(0, half)) and "backoff ± 10%" both leave a synchronized spike; full jitter flattens the arrival histogram and, in AWS's own experiments, minimizes both competing work and completion time. Two clients that time out on the same GC pause now retry at random, unrelated moments.
  3. Retry budget — cap the multiplier. A token bucket that refills at 10% of successful requests: each retry spends a token, and when the bucket is empty the client fails fast instead of retrying. This guarantees retries can add at most ~10% load, never 5×. (Envoy exposes this as a retry budget; Finagle and the Google SRE book use the same 10% rule of thumb.)
  4. Circuit breaker — zero retries at a dead dependency. Trip open when the error rate exceeds 50% over a 10s window (minimum ~20 requests so noise can't trip it), fail fast for a 5s cooldown, then go half-open and probe with a single request before closing. A dependency that is clearly down gets zero retries, not five.
  5. Only retry retryable failures. Retry timeouts, connection resets, 503, 429. Do not retry 400/401/409 validation errors — backoff can't fix a bad request. Honor a Retry-After header when the server sends one, and cap total wait by the caller's deadline, not just max_attempts.
flowchart TD
    A["Call fails (timeout / 503 / 429)"] --> B{"Retryable? budget<br/>token available?"}
    B -->|"no"| F["Fail fast (shed the retry)"]
    B -->|"yes"| C["backoff = min(cap, base * 2^n)"]
    C --> D["sleep = random_uniform(0, backoff)"]
    D --> E["Retry once, then re-evaluate"]
    G["Error rate > 50% over 10s"] --> H["Circuit OPEN — 0 retries<br/>for 5s, then half-open probe"]

When to use it — and when not

Reach for it on every client call to a shared, remote dependency that can be retried — RPCs, queue consumers, HTTP to a downstream service. Two hard preconditions: the request must be safe to retry (naturally idempotent, or carrying an idempotency key — Pattern 01), and the failure must be transient. When not to: don't retry a non-idempotent write with no idempotency key — you'll turn one lost response into two charges. Don't put a 10s cap on a user-interactive path with a 300ms budget — bound retries by the deadline instead, and often just fail fast. And backoff+jitter alone doesn't save a server that's already melting: pair it with server-side load shedding (drop low-priority work above ~80% CPU) so the recovering dependency protects a healthy core.

Signals it's working

Emit a retry-amplification SLI: retries / total_requests. Healthy is under ~10% (the budget); alarm when it exceeds 20% for 1 minute — a climbing ratio is the earliest sign a storm is forming. After you inject a stall in a game-day, the arrival histogram should stay flat (jitter working, no synchronized wave) and load should return to baseline within the budget once the stall clears — self-recovery, no restart. Watch circuit-breaker open events: against a dead dependency, retries should collapse to ~0, not stay pinned at 5×.

Seen in the wild

See it

The TilesScene shows the schedule (base 100ms · ×2 · cap 10s); the CompareScene puts lockstep retries next to full-jitter to show the wave flattening; the CardsScene carries the guardrail numbers (10% budget, breaker at 50%/10s, 5s cooldown) and the MetricLineScene shows the retry-amplification alarm at 20%.

Named concepts: exponential backoff, full jitter, retry budget, circuit breaker.