PD-01-014-P · Primer — groupby & agg

The idea in one sentence

groupby splits your rows into buckets that share a key, and agg boils each bucket down to one summary row per group.

Why it exists

Real data usually arrives one event per row — one sale, one click, one patient visit — but the questions we ask are almost always per category: total sales per region, average score per class, deal count per rep. Without grouping you would loop over every distinct key by hand, filter the rows, and compute a number for each — slow to write and easy to get wrong. groupby + agg names that pattern directly: say which column defines the buckets, then say which summary you want. pandas does the splitting, the math, and the reassembly for you, fast and correctly.

How it works

The mental model has three beats — split, apply, combine. First groupby("region") splits the rows into one group per distinct value of region (it does not compute anything yet; it just remembers the buckets). Then agg applies a reducing function — sum, mean, count, min, max, and friends — to a column within each bucket, turning many rows into one number. Finally pandas combines those numbers back into a tidy result whose index is the group keys. The clearest way to write this is named aggregation: new_col=("source_col", "func"), which lets you build several summaries at once with readable output column names.

import pandas as pd

sales = pd.DataFrame({
    "region": ["East", "West", "East", "West", "East"],
    "rep":    ["Ada",  "Bo",   "Cy",   "Bo",   "Ada"],
    "amount": [100,     250,    140,    300,    60],
})

# Split rows by region, then combine per-group summaries
summary = sales.groupby("region").agg(
    total_amount=("amount", "sum"),
    avg_amount=("amount", "mean"),
    num_deals=("amount", "count"),
)
print(summary)

# Expected output:
#         total_amount  avg_amount  num_deals
# region
# East             300       100.0          3
# West             550       275.0          2

The one gotcha

The group keys become the index of the result, not a normal column — so summary["region"] raises a KeyError. When you want region back as an ordinary column (for merging, plotting, or exporting), either call .reset_index() afterward or pass groupby("region", as_index=False). Reach for reset_index() by default; it saves a surprising number of downstream headaches.

See it — split, apply, combine at a glance

flowchart TB
  A["Raw rows<br/>5 sales, mixed regions"] --> B["groupby('region')<br/>split into buckets"]
  B --> C["East bucket<br/>100, 140, 60"]
  B --> D["West bucket<br/>250, 300"]
  C --> E["agg per group<br/>sum / mean / count"]
  D --> E
  E --> F["Combined result<br/>1 row per region"]

The one-line takeaway

df.groupby(key).agg(name=(col, func)) turns per-row data into one clean summary row per group.

Where you'll meet it

Anywhere you answer a "per-category" question: sales per region, revenue per month, average latency per service, students per grade, sessions per user. It is the backbone of dashboards, reports, and cohort analysis — and the first thing you'll reach for the moment someone asks "...but broken down by ___?"

Named concepts: groupby, split-apply-combine, agg, grouped aggregation