AIM-07-003-P · Primer — Lasso and Elastic Net (L1)

The idea in one sentence

Lasso penalizes the sum of absolute coefficient values, which drives some coefficients exactly to zero and thereby performs feature selection, while Elastic Net blends L1 and L2 to keep that selection stable when features are correlated.

Why it exists

When you have hundreds of candidate features but suspect only a handful truly matter, you want the model itself to throw the rest out — a sparse model is easier to explain, cheaper to serve, and often generalizes better. Ridge shrinks but never eliminates; Lasso does both. The geometric reason is that L1's diamond-shaped constraint region has sharp corners on the axes, so the optimum tends to land exactly on a corner where some coefficients are precisely zero.

How it works

Lasso minimizes RSS + lambda * sum(|beta_j|). As lambda increases, coefficients hit zero one by one, giving you a natural ranking of feature importance along the 'regularization path.' Worked example: start with 200 marketing-touchpoint features; at a cross-validated lambda, Lasso keeps 12 nonzero coefficients and zeros the other 188 — instant, interpretable feature selection. The gotcha: when two features are highly correlated, Lasso arbitrarily picks one and zeros the other, and which one it keeps can flip across resamples — bad for stability and for anyone reading the model. Elastic Net fixes this by minimizing RSS + lambda*(alpha*sum(|beta|) + (1-alpha)*sum(beta^2)); the L2 part encourages correlated features to enter or leave together (a 'grouping effect') while the L1 part still yields sparsity. You tune two knobs: lambda (overall strength) and the mixing alpha (1 = pure Lasso, 0 = pure ridge), typically via grid search with cross-validation. Standardize features first, exactly as with ridge.

See it

flowchart TD
  A["Many candidate<br/>features"] --> B{"Penalty<br/>type"}
  B --> C["L1 Lasso<br/>zeros some coeffs"]
  B --> D["L2 Ridge<br/>shrinks all"]
  C --> E["Correlated feats<br/>unstable pick"]
  D --> F["Keeps all<br/>features"]
  E --> G["Elastic Net<br/>blend L1 and L2"]
  F --> G
  G --> H["Sparse and<br/>stable model"]

In practice

Genomics or high-dimensional churn modeling: from tens of thousands of candidate signals, Elastic Net selects a compact, reproducible subset of predictive features that a domain expert can actually review.

The one thing to remember

Reach for Lasso when you want a sparse, self-selecting model — but if predictors are correlated, use Elastic Net, because pure Lasso will pick one of a correlated group at random and its choice won't be stable.