BitBootcampBitBootcamp ← all explainers 🔄 Review ⚙ Settings

Drill 58 — The Flag That Forked Reality

Theme: Migrations & Releases · Format: 20-min drill · Difficulty: core

The story

A migration flag routes 50% of traffic to the new pipeline — "temporarily, while we validate." Months pass; other priorities arrive; the flag stays at 50%, unowned and unremembered. The two paths drift: a fix lands on the new pipeline but not the old, a feature lands on the old but not the new, and soon bug reports come in that are irreproducible — "works for half our users." What was one product is now, effectively, two products that nobody fully understands.

⏸ Stop here and ask the room: an incident is open right now on this system. What's the first question nobody in the room is asking?

Run it — facilitator script

Timing: 2 min story · 10 min questions · 3 min reveal · 5 min transfer.

"Who owns a flag after the sprint that created it ends?"

Listen for: the honest answer — nobody, unless a mechanism says otherwise — followed by the fix: ownership is recorded at creation (a name, not a team) in a flag registry, and it survives sprint boundaries because reviews and pages route to that name. Flags belong to a moment; owners have to be attached deliberately.

Weak signal: "The team owns it." A team owning a flag is how it reaches month six at 50% — collective ownership of a small thing is nobody's job. Push until a mechanism produces a specific human.

"What does 50%-forever cost that 0% or 100% doesn't?"

Listen for: at 0% or 100% there is one product; at 50% there are two — double the test matrix, fixes and features landing on one path only, and every bug report carrying a hidden coin flip that makes "works for half our users" irreproducible by default. Sharp answers include the incident tax: every investigation starts one question deeper than anyone remembers to go.

Weak signal: "It's a bit more maintenance." It's a fork, not an overhead line — if the room prices it as maintenance, ask them to estimate the test matrix and the 3x incident from the reveal, then re-price.

"What mechanism — not intention — forces a flag to die?"

Listen for: machinery, not resolve — a deadline and deletion ticket created the day the flag is born, a ratchet plan (25%, then 50%, then 100%) with diff monitoring at each step, a weekly flag review, and a flag past its deadline paging its owner. Best answers make staleness loud: the flag SDK (software development kit) warns or fails builds once the expiry date passes.

Weak signal: "We'll clean them up in a hardening sprint." That's intention wearing a calendar — cleanup sprints lose to roadmaps for exactly the reasons the flag reached month six. If deletion isn't attached to the flag itself, it doesn't happen.

The reveal

The incident takes three times longer to debug than it should have, because nobody thought to ask which pipeline the affected requests took. The answer — all of them went down one path, none down the other — would have localized the bug in minutes, but the flag was so old it had fallen out of the team's mental model entirely.

Why this happens — the mechanics

"Temporary" is the most permanent state in software because removal is priced wrong. Deleting a flag delivers no feature, wins no praise, and carries pure downside — the deletion can break something, while leaving the flag alone breaks nothing today. Every individual week, the rational choice is to leave it; the cost only exists in aggregate, months later, spread across irreproducible bugs and a doubled test matrix where no single line item is worth escalating. That's the structure of debt — and unlike financial debt, nobody sends a statement.

Ownership evaporates on a schedule, too. The flag was created by a sprint, for a sprint's purpose — "while we validate" — and when the sprint ended, the validation question dissolved without being answered. The people who remember why it exists rotate to other work; six months later the flag isn't a decision anyone is deferring, it's furniture. That's why the reveal stings: the flag didn't just fork the product, it fell out of the team's mental model, so the one diagnostic question that would have localized the bug in minutes — which path? — was no longer in anyone's vocabulary.

And 50% is the worst possible resting point. At 0% the new path is dead code; at 100% the old path is deletable; at 50% both paths are live, both accumulate changes independently, and the divergence compounds — each path-specific fix makes the eventual merge harder, which makes finishing the migration more expensive, which makes deferring it more rational. A fork left at 50% doesn't hold still; it widens. The lesson's framing is the countermeasure: a flag is a loan, and loans come with a repayment schedule the day they're issued, not when the collector shows up.

Diagrams

Problem — how the failure unfolds:

timeline
    title Months of a temporary flag
    Sprint 1 : Flag routes 50% to new path : Labeled temporary (silent)
    Months pass : No deadline, no owner : Fixes land on one path only : Features land on the other
    Quarter end : Bug reports irreproducible : Works for half our users
    Incident day : Two products nobody knows : Debugging takes 3x longer : Nobody asks which pipeline

Solution — where the guardrails go:

flowchart TD
    A["Flag created"] --> B["Born with a deadline, an<br/>owner, and a deletion<br/>ticket"]
    B --> C["Ratchet plan: 25%, then<br/>50%, then 100%"]
    C -.-> V["Diff monitoring compares<br/>the two paths at each step"]
    V -.-> AL["Divergence pauses the ratchet"]
    C --> D["Every request logs which<br/>path it took"]
    D --> E["100% reached by the deadline"]
    E --> F["Old path deleted, ticket<br/>closed"]
    B -.-> W["Weekly flag review: any<br/>flag past deadline pages<br/>its owner"]

What good looks like

The lesson

Migration flags are loans, not settings: every one gets a deadline, a ratchet plan — 25%, then 50%, then 100% — and a deletion ticket created the day the flag is born. Log the path taken with every request, so "which pipeline?" is a query and not an archaeology project. "Temporarily both" is the most permanent state in software.

Named concepts: flag debt, migration ratchets, path divergence, delete-the-old-path discipline.

Real-world echoes

Transfer

List every flag in our codebase older than 90 days. For each: what's its deadline, who owns it, and does its deletion ticket exist? The blanks in that table are our fork count.

Make it a ticket: Run the 90-day flag audit this sprint and stand up the registry — owner, deadline, rollout percentage, deletion-ticket link per flag — plus the CI check that blocks unregistered flags; the audit table with zero blanks is the acceptance criterion.

Related