Interactive Lab · Decoding & Generation
When a language model writes, it picks tokens one at a time. The obvious strategy, greedy decoding, always take the single most likely next word, is short-sighted: a word that looks best right now can lead somewhere unlikely. Beam search hedges by keeping the top few candidate sentences alive at once, letting them branch and pruning the weak ones, so it can find a sentence that's more probable overall. Slide the beam width and watch it rescue a better answer.
▶ The playground, grow the beam tree
Beam search is a decoding strategy, how a model turns next-token probabilities into a finished sentence. It's used in translation and speech, and it's the counterpart to temperature/top-k sampling: the choice between “most likely” and “most creative” output. This is the “how it writes” step.
The code that's actually running
Start at B = 1 (greedy) and read the winning sentence and its probability. Now nudge B to 2. Watch what happens: a candidate greedy had thrown away at step 1 (because its first word was a hair less likely) survives, and by the end it wins, with a higher total probability. That's the whole point: the locally-best choice isn't always on the globally-best path. Widen B further and the answer stops changing, you've searched enough of the tree to be confident. Real decoders balance a bigger B (better sequences) against the extra compute it costs.
The takeaway