Interactive Lab · Deep Reinforcement Learning
The Q-Learning agent kept a value for every cell. But this world has no cells, the agent lives at a real-valued position (x, y), so there are infinitely many states and no table can hold them. DQN's fix: replace the Q-table with a small neural network that takes (x, y) and predicts the value of each move. Press Train and watch it learn a path from raw coordinates, then flip the two switches from the paper and watch training fall apart.
▶ The playground, train an agent that has no map
Store past moves in a memory and train on a random mix of them, instead of only the latest, correlated step.
Aim at a frozen copy of the network that only updates occasionally, so the agent isn't chasing a target that moves every step.
DQN is how an agent learns to act from raw reward, no labeled “right answer,” just trial, error, and a score. It's the lineage behind game-playing AI (Atari, Go), robotics, recommendation, and the RL that fine-tunes LLMs. Reach for RL when you can't hand a model examples, only a goal and feedback.
The code that's actually running
Train once with both switches ON until it says policy found, note how the value tint spreads smoothly across the whole board even though the net only ever visited a handful of points (that's the network generalizing). Now Reset, turn Target network OFF, and retrain. Before you do: predict what happens to the success rate. Why does aiming at a target that shifts every single update make the learning wobbleeven diverge, instead of settling?
The takeaway
Q(s,a), so it can generalize to states it never visited. You saw the value tint fill cells the agent never stepped on.r + γ·maxQ(s′), but if that estimate comes from the same net you're changing, you're chasing a moving target. A frozen copy, synced occasionally, holds it still. You watched training wobble the moment you removed it.