ML-01-007-P · Primer — Decision Tree Classifier: Learning If/Else Rules
The idea in one sentence
A decision tree learns a flowchart of yes/no questions — like "is the size above 6.5?" — that funnels each sample down to a predicted class.
Why it exists
Most beginners already think in if/else rules: if it's small, it's a lime; if it's big, it's an orange. The magic of a decision tree is that you don't hand-write those rules — the algorithm reads your labeled examples and discovers the questions and the cut-off numbers for you. That makes it one of the most intuitive models in machine learning: you can literally print the tree and read it like a checklist. And unlike a black box, every prediction comes with a trail of the exact rules that produced it, so you can explain why the model said what it said.
How it works
The tree looks at every feature and every possible cut-off point, then picks the single split that best separates the classes — the one that leaves each side as "pure" (one-class) as possible. It repeats this on each resulting group, growing branches until the groups are clean or a stopping rule kicks in. Each internal node stores a feature and a threshold; to predict, you drop a sample in at the top and follow the yes/no answers down to a leaf, which holds the class. Below, six labeled sizes teach the tree that small items are class 0 and large items are class 1 — and we print the exact rule it learned.
from sklearn.tree import DecisionTreeClassifier, export_text
# Tiny inline dataset: one feature (item size), two classes
X = [[2.0], [3.0], [10.0], [12.0], [1.5], [11.0]] # sizes
y = [0, 0, 1, 1, 0, 1] # 0 = small, 1 = large
clf = DecisionTreeClassifier(max_depth=1, random_state=0)
clf.fit(X, y)
print(clf.predict([[4.0], [9.0]])) # [0 1] -> small, then large
print(clf.score(X, y)) # 1.0 -> all 6 correct
print("threshold:", clf.tree_.threshold[0]) # threshold: 6.5
print(export_text(clf, feature_names=["size"]))
# |--- size <= 6.50
# | |--- class: 0
# |--- size > 6.50
# | |--- class: 1The tree found the split point 6.5 entirely on its own — no human picked that number.
The one gotcha
Left unchecked, a tree keeps splitting until every training point sits in its own tiny leaf — memorizing noise instead of learning the pattern, so it looks perfect on training data and flops on new data. That's overfitting, and the simplest guardrail is max_depth, which caps how many questions deep the tree may go (max_depth=1 above allows exactly one split). Start shallow, then increase depth only if validation accuracy actually improves.
See it — a sample flows down the learned tree
flowchart TD
A["New item size = 9.0"] --> B{"size <= 6.5 ?"}
B -->|"yes"| C["Predict class 0<br/>(small)"]
B -->|"no"| D["Predict class 1<br/>(large)"]
The one-line takeaway
A decision tree turns labeled examples into a readable ladder of if/else rules, and max_depth keeps that ladder short enough to generalize.
Where you'll meet it
Credit approvals, medical triage checklists, churn flags, and fraud screens all lean on trees precisely because a regulator or teammate can read every rule — and trees are the building block behind Random Forests and gradient boosting, the workhorses of the deep AIM track.
Named concepts: DecisionTreeClassifier, splits and thresholds, max_depth