Interactive Lab · Classification by proximity

Can an algorithm classify without ever "learning"?

Meet the laziest algorithm in machine learning. It does no training at all, it just memorizes every example. To classify a brand-new point, it asks the k nearest neighbors and takes a majority vote. That's the whole idea. Drag the query point anywhere, and watch the entire background repaint into the regions where each class wins.

🎬 Watch the 45-second explainer, then turn the knobs yourself below ↓

▶ The playground, drag the query, then turn the k knob

k = 5 vote prediction

k, neighbors that vote 5

1 · jagged, overfitsmooth, underfit · 15

🧭 Where this fits

k-NN is the algorithm behind 'similar items', 'people also bought', and every vector-database query, RAG retrieval is literally k-NN over embeddings. The bias-variance dial you just turned (k=1 jagged, k=15 smooth) is the same tradeoff tuned in recommender systems, and its cost at scale is why approximate indexes (HNSW, IVF) exist.

The code that's actually running

🔮 Predict, then test

Set k = 1 and slowly drag the query across the boundary, notice how the prediction flickers the instant it passes a lone noisy dot, and how the background shatters into little islands. Now set k = 15: smoother and calmer, but what happens to a small cluster near the edge? (It gets outvoted and swallowed.) Before you slide k, predict: does raising k make the boundary simpler or more detailed? Why is choosing k a bias/variance tradeoff?

The takeaway

Your new mental model of k-Nearest Neighbors

There is no training. kNN never fits weights or builds a model, it just stores every point and measures distance at query time. All the work happens when you ask.
k is a smoothness dial. Small k = low bias, high variance, it chases every noisy point into jagged islands (overfit). Large k = high bias, low variance, a smooth but blurry boundary that can swallow small classes (underfit).
The boundary is where the vote flips. The tinted regions are the classifier, nothing is drawn there until a query lands and its neighbors are counted.
It's fragile in the real world. kNN is sensitive to feature scale (one big-range feature dominates the distance) and collapses in high dimensions, the curse of dimensionality, where "nearest" stops meaning anything.

📖 Read the lecture notes Try: The Perceptron Continue the journey →