ML-01-014-P · Primer — Overfitting vs Underfitting: Finding the Balance

The idea in one sentence

A model underfits when it's too simple to capture the pattern (bad on data it trained on and new data), overfits when it's so complex it memorizes the training data instead of learning the pattern (great on training, terrible on new data), and you diagnose which one you have by comparing the training score to the test score.

Why it exists

The whole point of machine learning is to do well on data you haven't seen yet — tomorrow's customers, next week's transactions, the photo you haven't taken. But you only ever get to train on data from the past, so there's a sneaky trap: a model can look amazing on the exact examples it studied while being useless on anything new, the same way a student who memorized last year's answer key can ace that test and still fail the real exam. That failure is called overfitting, and its mirror image is underfitting — a model so simple it can't even nail the examples it did see, like answering every exam question with "42." Left unchecked, both quietly wreck real projects: the overfit model dazzles in the demo and collapses in production, the underfit model is honestly bad from day one. This primer exists because every model you ever build lands somewhere on this spectrum, and the single most important skill in applied ML is reading where — so you know whether to make your model fancier, simpler, or leave it exactly where it is.

How it works

The core move is to split your data into a training set (the model learns from it) and a test set (held back, used only to grade the model on data it never saw), then score the model on both. Underfitting shows up as both scores being low — the model is too weak to capture the pattern anywhere. Overfitting shows up as a large train-vs-test gap — a near-perfect training score sitting next to a poor test score, meaning the model memorized quirks and noise instead of the real signal. The "just right" zone is where both scores are high and close together. As you dial a model's complexity up (here, the degree of a polynomial we fit), you sweep from underfit, through a sweet spot, into overfit — and watching the two scores diverge is exactly how you spot the balance point. Below we fit the same recipe at three complexities on one tiny curved dataset and print R2 (1.0 is perfect, 0.0 is no better than guessing the average, and negative is worse than guessing the average):

import numpy as np
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score

# Tiny inline dataset: a gentle U-shaped curve (quadratic truth) + a little noise
rng = np.random.RandomState(0)
X = np.linspace(-3, 3, 24).reshape(-1, 1)                        # 24 points, one feature
y = 0.5 * X.ravel()**2 - X.ravel() + rng.normal(0, 1.2, size=24)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.35, random_state=4)                       # hold back 35% as "unseen"

# Same recipe, three complexities: a line, a curve, and a wild wiggle
for degree in [1, 2, 12]:
    model = make_pipeline(PolynomialFeatures(degree), LinearRegression())
    model.fit(X_train, y_train)
    train_r2 = r2_score(y_train, model.predict(X_train))
    test_r2  = r2_score(y_test,  model.predict(X_test))
    print(f"degree {degree:>2}:  train R2 = {train_r2:5.2f}   test R2 = {test_r2:8.2f}")

# degree  1:  train R2 =  0.65   test R2 =     0.28   <- UNDERFIT: both mediocre, too simple
# degree  2:  train R2 =  0.81   test R2 =     0.76   <- JUST RIGHT: both high, tiny gap
# degree 12:  train R2 =  0.98   test R2 = -12619.19  <- OVERFIT: aced train, fails on new data

Read the three rows like a story: the straight line (degree 1) is too dumb to bend around a curve so it's mediocre everywhere; the gentle curve (degree 2) matches the true shape and scores high on both sets; the wild degree-12 wiggle threads perfectly through the training dots (train R2 0.98) but whips off to nonsense between them, so on unseen points it does far worse than just guessing the average — that's the catastrophic negative test score.

The one gotcha

A high training score means nothing on its own — you can always push training accuracy up by adding complexity, right off a cliff. The number that lured you (train R2 = 0.98 above) is the exact number a memorizing model produces, so judging a model by how well it fits its own training data is like judging a chef by whether they can re-eat a meal they already cooked. The honest signal is the test score, and specifically the gap between train and test: a tiny gap with both high is health; a big gap with train much higher than test is overfitting, no matter how gorgeous that training number looks. This is also why you must decide on model complexity using data the model never trained on — if you keep tweaking until the test set looks great, you've quietly started memorizing the test set too, which is why serious workflows use a separate validation set or cross-validation to choose complexity and save the test set for one final, untouched grade.

See it — diagnosing a model from two scores

flowchart TD
    A["Split data:<br/>train + test set"] --> B["Fit model,<br/>score on both"]
    B --> C{"Training<br/>score high?"}
    C -->|"No"| D["UNDERFIT<br/>(too simple)"]
    D --> E["Add complexity,<br/>better features"]
    C -->|"Yes"| F{"Test score<br/>close to train?"}
    F -->|"No: big gap"| G["OVERFIT<br/>(memorized noise)"]
    G --> H["Simplify, add data,<br/>or regularize"]
    F -->|"Yes: small gap"| I["JUST RIGHT<br/>(ship it)"]

The one-line takeaway

Never trust the training score alone — compare it to the test score, and let the train-vs-test gap tell you whether to make your model fancier (underfit), simpler (overfit), or leave it be.

Where you'll meet it

This is the single most-used diagnostic in all of applied ML, so you'll meet it on day one of every project and never stop. The moment you call .score() on training data and again on test data, you're running this check — a spam classifier that's 100% on training email and 70% on new email is overfitting in plain sight. You'll see it as a validation curve, plotting train and test scores as you sweep a knob like a tree's max_depth, a k-NN's n_neighbors, or a model's regularization C/alpha — the point where the test curve peaks and the gap starts widening is your sweet spot. It's the reason train_test_split and cross_val_score exist, the reason "regularization" is a word you'll hear constantly (it's the standard cure for overfitting), and the reason a stunning demo metric should make you suspicious, not proud, until you've seen the held-out number. Master reading these two scores and you've got the instinct that separates people who run models from people who trust them — and that instinct is exactly what the deeper AIM track sharpens into cross-validation, learning curves, and the bias-variance tradeoff underneath it all.

Named concepts: overfitting, underfitting, train vs test gap