AIM-01-002-P · Primer — Features & Labels

The idea in one sentence

Features are what the model gets to see, and the label is the one thing it is trying to predict — every supervised learning problem is just learning the mapping from one to the other.

Why it exists

Supervised learning needs a crisp split of every example into "inputs" and "the answer." Picture a stack of flashcards: the front is the question the model reads, the back is the answer it is graded against. Features are the front — the columns of a row: a customer's monthly spend, their login count, how long they've been a customer. The label is the back — the single target column: did they churn, yes or no. Without that split there is nothing to learn from and nothing to learn toward.

How it works

Each training row is one example: a vector of feature values plus one label. In a churn dataset a row might be monthly_spend = $42, logins_last_30d = 3, tenure_months = 14, support_tickets = 2, and the label churned = 1:

\[ x^{(i)} = \left[\begin{array}{ll} 42 & \text{monthly\_spend} \\ 3 & \text{logins\_last\_30d} \\ 14 & \text{tenure\_months} \\ 2 & \text{support\_tickets} \end{array}\right], \qquad y^{(i)} = 1 \;\;(\text{churned}) \]

During training you feed the model thousands of such rows where the answer is already known, and it tunes its parameters until its predicted label matches the true label as often as possible — that is, it pushes \(\hat{y}^{(i)} \approx y^{(i)}\) across as many rows \(i\) as it can. Labels come in flavors: a yes/no label makes it a classification problem, a numeric label like next month's revenue makes it regression. Features come in flavors too — numbers, categories, text, dates — each needing its own preprocessing before the model can read it. Conventionally the feature matrix is written capital \(X\) and the label vector lowercase \(y\):

\[ X = \begin{bmatrix} x^{(1)\top} \\ x^{(2)\top} \\ \vdots \\ x^{(n)\top} \end{bmatrix} \in \mathbb{R}^{n \times d}, \qquad y = \begin{bmatrix} y^{(1)} \\ y^{(2)} \\ \vdots \\ y^{(n)} \end{bmatrix} \in \mathbb{R}^{n}, \qquad \hat{y} = f_\theta(X) \]

See it — one churn row splits into features (X) and label (y)

flowchart LR
  Row["one training row"] --> X["features (X):<br/>spend=$42<br/>logins=3<br/>tenure=14mo"]
  Row --> Y["label (y):<br/>churned = 1"]
  X --> Model["model learns<br/>X to y mapping"]
  Model --> Pred["predicted label"]
  Y -.compare.-> Pred
  Pred --> Tune["tune params<br/>until match"]
  Tune -.repeat.-> Model

The one gotcha

Your label is not handed to you by nature — it is a choice, and a bad or leaky one dooms everything downstream. "Churn" sounds obvious until you ask: does it mean they cancelled, went 30 days without logging in, or simply didn't renew? Each definition is a different label and trains a different model. Worse is label leakage: a feature that secretly encodes the answer. If one feature is "filled out the exit survey" — which only churned customers ever do — the model learns the shortcut exit-survey = churn and posts \(99\%\) training accuracy, then collapses in production because that feature isn't known until after the customer has already left. You leaked the answer into the question, and no amount of fancier modeling can fix a target that was wrong from the start.

See it (visualization plan)

A PipelineScene of features → model → label establishes the mapping; a CardsScene/TilesScene shows one concrete churn row with real numbers and its churned = 1 label. For the gotcha, a TimelineScene places the prediction moment before the label-known moment and marks the leaky "exit survey" feature as arriving after the answer, then a BigNumberScene counts 99% training accuracy up and labels it a lie.

The one-line takeaway

Your label is a choice — pick it carefully, and never let it leak.

Where you'll meet it

The first line of real ML code you write: X, y = ... in scikit-learn, Dataset and DataLoader targets in PyTorch, the label column you point a LightGBM or XGBoost model at. Every training API in existence wants features and a label; getting y right — well-defined and leak-free — matters more than any model you'll pick to fit it.

Named concepts: features, labels, label leakage