Interactive Lab · Neural Networks

Watch a neural network draw the line

A classifier's whole job is to find a boundary that separates the classes. A straight line handles easy cases, but the spiral below is hopeless for any line. Give the network hidden neurons and it bends the boundary into whatever shape the data needs. Press Train and watch the colored regions warp to fit the dots, then shrink the network and watch it fail.

▶ The playground, pick a shape, build a net, press Train

epoch 0 train acc test acc loss status untrained

class A class B shaded region = what the network predicts there · dots = the training data

Dataset the shape to separate

spiral circle xor two blobs

Input features what you feed the net (add x·y to make XOR easy)

x y x² y² x·y sin x sin y

Hidden layers 2

0 · just a linedeep · 3

Neurons / layer 8

2 · tinywide · 8

Learning rate 0.030

0.002 · slowfast · 0.1

Regularization L2 0.000

0 · fit freely0.05 · keep it simple

Noise 0.06

0 · cleanmessy · 0.3

Activation

tanh ReLU

🧭 Where this fits

This is the “hello world” of deep learning: a small network bending a decision boundary to separate data. Every classifier, vision model, and LLM is this same idea scaled up, layers of weighted sums and nonlinearities, trained by gradient descent. When your data isn't linearly separable, you add width and depth; watch that trade-off live here.

The code that's actually running

🔮 Predict, then test

Switch to xor and set Hidden layers = 0 (just a line). Train, it stalls near 60%, because no single straight line can split XOR. Now, still at 0 layers, turn on the x·y feature. Before you press Train: predict what happens. Why does multiplying two inputs let a linear model suddenly solve a problem that was impossible a moment ago? (This is the whole idea of feature engineering, and why hidden layers, which invent their own features, are so powerful.)

The takeaway

Your new mental model of neural networks

Hidden layers bend the boundary. Zero layers = a straight line. Each layer lets the net compose simple pieces into curves, spirals, islands, whatever separates the data.
Capacity is a dial. Too few neurons and it underfits (can't capture the shape); too many with no regularization and it starts memorizing noise. You watched the tiny net fail on the spiral.
Regularization keeps it simple. L2 pulls weights toward zero, smoothing the boundary. Crank it up and even a big net is forced back to a near-straight line.
Features matter as much as depth. A linear model can't do XOR, until you hand it x·y. Hidden layers are powerful because they learn features like that automatically.

How the weights update →Gradient descent 📖 Lecture notes