PAPERS-03-007 · Paper — AlphaZero

The paper

The paper “AlphaZero” by Silver et al., published in 2017 at the journal Science, is a landmark because it showed that a single reinforcement‑learning algorithm could master multiple complex board games without any human‑crafted knowledge.

Before this paper

Before AlphaZero, the strongest game‑playing AIs such as AlphaGo still relied on large databases of human expert games and hand‑tuned features specific to each game. This dependence limited their ability to generalize to new domains and created a wall: progress required extensive human data and game‑specific engineering. Researchers struggled to find a truly general method that could learn from scratch.

The key idea

The key insight of AlphaZero is to let the system learn entirely from self‑play, using only the rules of the game. A single neural network serves both as a policy (suggesting moves) and a value estimator (predicting the outcome). No human data or handcrafted features are needed, making the approach game‑agnostic.

How it works

AlphaZero starts with a randomly initialized network and plays games against itself. After each move it runs a Monte Carlo tree search (MCTS) guided by the current network to explore possible continuations and produce strong move probabilities. The outcomes of these self‑play games become training targets for the network, updating both policy and value heads. This loop—play, search, train—is repeated many times, gradually improving the network. The same algorithm, without any modification, was applied to chess, shogi, and Go, each time learning from scratch.

See it — learning from pure self-play

flowchart LR
  A["Only the rules"] --> B["Play itself<br/>millions of games"]
  B --> C["Train one net<br/>policy + value"]
  C --> B
  C --> D["Superhuman<br/>play"]

What they showed

With no human knowledge beyond the rules, AlphaZero mastered chess, shogi, and Go, defeating the strongest programs — Stockfish in chess, Elmo in shogi — and its predecessor AlphaGo in Go.

What it changed

AlphaZero demonstrated that a single, general reinforcement‑learning algorithm could reach superhuman skill across very different domains without any human examples. This broke the belief that each game required bespoke engineering and opened the door to universal game‑playing agents. It shifted the field toward learning‑only approaches and inspired many follow‑up works. Next: MuZero.

The one-line takeaway

A single self‑play algorithm can become superhuman in any board game without human data.

Read it / meet it

Reading the AlphaZero paper reveals the elegant simplicity of learning from scratch and the power of combining neural networks with MCTS. Its legacy lives on in today’s self‑play training pipelines that now extend far beyond board games, shaping modern AI research.