ML-01-010-P · Primer — Linear Regression: Your First Regressor

The idea in one sentence

Linear regression draws the single best-fitting straight line through your data so that, given an input, it can predict a continuous number — like a price, a temperature, or an exam score — and the line's slope and starting height tell you exactly how the input drives the output.

Why it exists

A huge share of real questions ask "how much?" rather than "which category?" — how much will this house sell for, how many minutes will the delivery take, what score will a student get. Those answers are numbers on a continuous scale, not labels, so they call for a regressor rather than a classifier. Linear regression exists because the simplest honest guess for "how does y move as x moves?" is a straight line, and a straight line has just two dials you can read off and reason about: a slope (how much y changes per unit of x) and an intercept (where the line sits when x is zero). That transparency is its superpower — unlike a black box, it hands you an equation you can explain to your boss ("each extra hour of study adds about 9 points"). It's also fast, needs almost no tuning, and rarely overfits, which is why it's the very first regressor everyone learns and the baseline every serious model must beat.

How it works

You give the model a table of inputs X (one row per example, one column per feature) and a matching list of target numbers y. Fitting means finding the slope(s) and intercept that make the line pass as close as possible to every point — specifically, the line that makes the total squared vertical distance from points to line as small as it can be (this is why it's called "least squares"). Once fit, the model stores those learned numbers in .coef_ (the slope for each feature) and .intercept_ (the height where the line crosses x = 0), and predict simply plugs a new x into the line's equation y = coef · x + intercept. In scikit-learn the whole thing is three lines — create, fit, predict — and .score(X, y) returns R², a 0-to-1 grade of how much of the target's variation the line explains. Here it is on a tiny "hours studied → exam score" dataset:

import numpy as np
from sklearn.linear_model import LinearRegression

# One feature (hours studied) -> one continuous target (exam score)
X = np.array([[1], [2], [3], [4], [5]])   # shape (5 samples, 1 feature)
y = np.array([52, 60, 71, 79, 88])         # the numbers we want to predict

model = LinearRegression()
model.fit(X, y)                            # find the best-fit line

print(model.coef_)                         # [9.1]   -> +9.1 points per study hour
print(round(model.intercept_, 1))          # 42.7    -> predicted score at 0 hours
print(round(model.predict([[6]])[0], 1))   # 97.3    -> predict a NEW input (6 hours)
print(round(model.score(X, y), 3))         # 0.998   -> R^2: line explains ~99.8%

Read that output like a sentence: "start at 42.7, add 9.1 points for every hour," and the predict([[6]]) call is the model doing exactly that arithmetic for an input it never saw.

The one gotcha

A coefficient's size depends entirely on the units of its feature, so a small number does not mean "unimportant." The slope is "change in y per one unit of x" — change the unit and the number changes with it, even though the model is identical. Measure study time in minutes instead of hours and the coefficient shrinks by a factor of 60, yet every prediction stays exactly the same:

import numpy as np
from sklearn.linear_model import LinearRegression

X_hours = np.array([[1], [2], [3], [4], [5]])
X_mins  = X_hours * 60                      # same data, different units
y = np.array([52, 60, 71, 79, 88])

coef_hours = LinearRegression().fit(X_hours, y).coef_[0]
coef_mins  = LinearRegression().fit(X_mins,  y).coef_[0]

print(round(coef_hours, 3))                 # 9.1
print(round(coef_mins,  4))                 # 0.1517  -> 60x smaller, NOT weaker!
print(round(coef_hours / coef_mins, 1))     # 60.0    -> exactly the unit change

So never rank features by raw coefficient magnitude across different units. If you want to compare "which feature matters more," standardize your features first (put them on a common scale) — then, and only then, do the coefficient sizes become comparable.

See it — how a linear regressor learns then predicts

flowchart TD
    A["Inputs X and<br/>target numbers y"] --> B["fit: find the line<br/>with least squared<br/>error"]
    B --> C["Learned dials:<br/>coef_ (slope) and<br/>intercept_ (height)"]
    C --> D{"New input x<br/>to score?"}
    D -->|"Yes"| E["predict: y =<br/>coef * x + intercept"]
    E --> F["Continuous number<br/>(e.g. exam score)"]
    D -->|"Check fit"| G["score: R^2 grades<br/>how well line fits"]

The one-line takeaway

Linear regression fits one straight line to predict a continuous number, and its .coef_ (slope) and .intercept_ (starting height) are a plain-English equation you can read and explain.

Where you'll meet it

You'll meet linear regression the moment a task asks for a number instead of a label: forecasting sales from ad spend, estimating a house price from square footage, predicting delivery time from distance, or projecting energy use from temperature. It's the default baseline in almost every regression project — you fit it first, note its R², and treat any fancier model (random forests, gradient boosting, neural nets) as needing to earn its keep by beating that number. Its readable coefficients make it a favorite wherever people must justify a prediction — think pricing, medicine, and finance, where "the model is a black box" is not an acceptable answer. You'll also see its ideas everywhere downstream: the fit / predict / score rhythm is identical across all of scikit-learn, standardizing features to compare coefficients is your first taste of preprocessing, and R² is the grade you'll reach for on every regressor you build. Nail this one, and the whole regression toolkit — regularized lines like Ridge and Lasso, then the powerful nonlinear models in the deep AIM track — clicks into place on top of it.

Named concepts: LinearRegression, predicting continuous values, coefficients and intercept