AIM-07-004-P · Primer — Logistic Regression in Depth

The idea in one sentence

Logistic regression models the log-odds of a binary outcome as a linear function of the features, then squashes that through the sigmoid to produce a calibrated probability between 0 and 1.

Why it exists

You can't fit a straight line to a 0/1 outcome — it would predict probabilities above 1 and below 0 and assume a fixed effect on probability that makes no sense near the extremes. Logistic regression solves this by being linear in the right space (log-odds) rather than in probability space. It's the honest default for binary classification: fast, interpretable, and it outputs genuine probabilities you can threshold or calibrate, not just labels.

How it works

The model is log(p / (1-p)) = beta0 + beta1*x1 + ... , where p / (1-p) are the odds and the left side is the log-odds (logit). Inverting gives p = 1 / (1 + e^-(linear part)), the S-shaped sigmoid that maps any real number to (0,1). Each beta is the change in log-odds per unit of x; exponentiate it and e^beta is the odds ratio — the multiplicative change in odds. Worked example: for churn, beta for 'monthly_charges' = 0.03 means each extra dollar multiplies the odds of churn by e^0.03 = 1.03, a 3% odds increase per dollar. We fit by maximum likelihood (there's no closed form), iterating with Newton-Raphson / IRLS until convergence. The decision boundary is where the linear part equals 0, i.e. p = 0.5 — and because the linear part is linear in features, that boundary is a straight line (or hyperplane) in feature space. A key subtlety: a coefficient's effect on probability is not constant — it's largest near p = 0.5 and flattens near 0 or 1, which is why you can't read a logistic coefficient the way you read a linear one. Perfectly separable classes make the MLE blow up to infinity, so regularization (an L2 penalty) is standard practice.

See it

flowchart LR
  A["Features x"] --> B["Linear part<br/>beta0 plus beta x"]
  B --> C["Log-odds<br/>logit"]
  C --> D["Sigmoid<br/>squash to 0 to 1"]
  D --> E["Probability p"]
  E --> F{"p above<br/>threshold"}
  F --> G["Predict class 1"]
  F --> H["Predict class 0"]

In practice

Customer churn: score each subscriber's probability of canceling next month, rank them, and target retention offers at the highest-probability segment while reporting each driver as an interpretable odds ratio.

The one thing to remember

A logistic coefficient is a constant effect on log-odds, not on probability — exponentiate it to get an odds ratio, and remember the same coefficient moves probability a lot near 0.5 and barely at all near the extremes.