Interactive Lab · Search & Planning

It knows no strategy, it just imagines thousands of games

Monte-Carlo Tree Search was never taught a single rule of tic-tac-toe beyond "here are the legal moves." To choose, it plays out thousands of random games in its head from each option and counts which one wins most often, then plays that. Slide the simulations up and watch a hopeless random flailer turn unbeatable. It's the exact idea (scaled up with a neural net) behind AlphaGo.

▶ The playground, you're X, beat the search

your move X MCTS O MCTS ran sims result your turn

bar in a cell = how many of MCTS's imagined games tried that move

Simulations per move 400

10 · plays randomlyunbeatable · 2000

🧭 Where this fits

Monte-Carlo Tree Search is how an agent plans by imagining many futures before committing to a move. It's the search half of AlphaGo/AlphaZero and powers game AI, planning, and, increasingly, LLM “reasoning” (search over chains of thought). Use it when you can simulate outcomes and need to look ahead.

The code that's actually running

🔮 Predict, then test

Drop the simulations to 10 and start a new game. Now MCTS barely gets to imagine anything, so it often misses an obvious block or win, you can beat it easily. Crank it to 2000 and try again: it will always block your threats and take any winning move, so the best you can get is a draw. Same code, same random rollouts, the only thing that changed is how many games it got to imagine. That's the whole bet of MCTS: thinking time buys strength.

The takeaway

Your new mental model of MCTS

Four phases, repeated. Select a promising path (using UCB1, which balances exploiting good moves vs exploring rare ones), expand a new node, simulate a random game to the end, and backpropagate the win/loss up the path.
Wins are just counted. No evaluation function, no hand-coded tactics. A move looks good purely because random games starting with it tended to win.
UCB1 focuses the search. It spends most simulations on the current-best move but keeps sampling the rest, so a sleeper move can still be discovered. That's the explore/exploit trade-off again.
AlphaGo = MCTS + a neural net. Replace "random rollouts" with a network that judges positions and suggests moves, and this exact loop beats the world champion at Go.

Learning from reward: Q-Learning →Deep Q-Learning 📖 Lecture notes