ML-01-008-P · Primer — Logistic Regression: Predicting Class Probabilities

The idea in one sentence

Logistic regression draws a straight dividing line between two classes and, instead of just saying "left or right," tells you how confident it is with a probability between 0 and 1.

Why it exists

Plain linear regression is great for predicting numbers, but it happily predicts things like 1.7 or -0.3 when all you wanted was a yes/no answer — nonsense for a probability. Logistic regression fixes this by squashing the straight-line output through an S-shaped sigmoid function that always lands neatly between 0 and 1. That gives you two things at once: a hard prediction (class 0 or class 1) and a soft probability you can trust and act on. It is the workhorse first classifier of machine learning — fast, interpretable, and hard to beat as a baseline.

How it works

Under the hood, the model learns a weight for each feature and computes a weighted sum (a straight line, just like linear regression). That sum is fed through the sigmoid, turning it into P(class = 1). Anywhere the probability crosses 0.5, you get the decision boundary — the flat line that separates the two classes. During training, fit nudges the weights until the predicted probabilities best match the real labels. At prediction time, predict gives you the class and predict_proba gives you the full probability for each class. Watch it learn that more study hours pushes the pass-probability up:

from sklearn.linear_model import LogisticRegression
import numpy as np

# Tiny dataset: hours studied -> passed the exam? (0 = fail, 1 = pass)
X = np.array([[1.0], [2.0], [3.0], [4.0], [5.0], [6.0]])
y = np.array([0, 0, 0, 1, 1, 1])

clf = LogisticRegression()
clf.fit(X, y)

print(clf.predict([[2.5], [4.5]]))         # [0 1]  -> fail, pass
print(clf.predict_proba([[2.5], [4.5]])[:, 1].round(3))  # [0.246 0.754] = P(pass)
print(clf.score(X, y))                     # 1.0 -> all 6 training points correct

The one gotcha

The word "regression" fools everyone — logistic regression is a classifier, not a numeric predictor. And predict only ever tells you the side of the 0.5 line; the interesting information lives in predict_proba. A point at 51% and a point at 99% both predict as class 1, but they are worlds apart in confidence. Always reach for predict_proba when the strength of a decision matters.

See it — from study hours to a pass/fail decision

flowchart TD
  A["Input feature<br/>(hours studied)"] --> B["Weighted sum<br/>(a straight line)"]
  B --> C["Sigmoid squashes it<br/>to 0 to 1"]
  C --> D["predict_proba<br/>P(pass) = 0.754"]
  D --> E{"Is P above 0.5?<br/>(decision boundary)"}
  E -->|Yes| F["predict: pass (1)"]
  E -->|No| G["predict: fail (0)"]

The one-line takeaway

Logistic regression fits a straight decision boundary and hands you calibrated class probabilities via predict_proba — not just a yes/no.

Where you'll meet it

Spam vs. not-spam, will-this-customer-churn, is-this-transaction-fraud, medical risk scores — anywhere you need a fast, explainable "how likely is class 1?" answer. It is the default baseline in nearly every real ML project, and the conceptual on-ramp to the deep classification and neural-network models in the AIM track.

Named concepts: LogisticRegression, predict_proba, decision boundary