DA-01-006-P · Primer — Spread: Standard Deviation, IQR, and Range

The idea in one sentence

Spread measures how far apart your values are, so you can tell a reliable 30-minute delivery from a sometimes-12, sometimes-48 one that happens to average 30.

Why it exists

An average is a single number pretending to speak for a whole column, and it lies by omission. Two drivers can both "average 30 minutes" while one is metronome-steady and the other is a coin flip — the mean hides that completely. The moment you make a promise, a plan, or a comparison, what you actually care about is consistency, and consistency is spread. So we keep a second number next to every average that answers a different question: "and how much do the values wander?" Range gives the crude headline, standard deviation gives the typical wander, and the IQR gives the wander of the sturdy middle. Together they stop a good-looking average from fooling you.

How it works

The question spread asks is: "if I only knew the average, how surprised would a single new value make me?" Range = max − min: the full width, dead simple, but it's decided entirely by the two most extreme points. Standard deviation (std) is the typical distance of a value from the mean — small std means everyone hugs the average, large std means they scatter. IQR = Q3 − Q1: the width of the middle 50% of the data, which quietly ignores the wildest points, so it's the spread measure that keeps its head when outliers are around. Watch three values collapse to identical means but tell three different stories through their spread.

import pandas as pd

# Two drivers, SAME average delivery time — very different reliability
data = pd.DataFrame({
    "anna": [28, 31, 29, 32, 30],   # steady
    "ben":  [12, 48, 15, 45, 30],   # wild swings
})

means = data.mean()
q1, q3 = data.quantile(0.25), data.quantile(0.75)

summary = pd.DataFrame({
    "mean":  means.round(1),
    "range": (data.max() - data.min()),
    "std":   data.std().round(1),      # sample std (ddof=1) by default
    "iqr":   (q3 - q1).round(1),
})
print(summary)
# mean is IDENTICAL (30.0, 30.0), but ben's std (16.6) and IQR (30.0)
# dwarf anna's (1.6, 2.0): same average, wildly different consistency.

The one gotcha

std and range are drama queens about outliers — one freak value can triple them. pandas std() also defaults to the sample formula (ddof=1), dividing by n−1, not n, so it will not match a hand calc or NumPy's default (ddof=0) on tiny data; pick one convention and state it. When your data has outliers or long tails, trust the IQR — it's built from the middle 50% and shrugs off the extremes, which is exactly why box plots draw the box from Q1 to Q3.

See it — picking a spread measure

flowchart TD
  A["Have a mean.<br/>How much do values<br/>wander?"] --> B{"Outliers or<br/>long tails?"}
  B -->|"No, tidy data"| C["Report std<br/>(typical distance<br/>from mean)"]
  B -->|"Yes"| D["Report IQR<br/>(middle 50%,<br/>ignores extremes)"]
  C --> E["Range for the<br/>crude headline<br/>(max minus min)"]
  D --> E
  E --> F["Mean plus spread:<br/>a promise you<br/>can defend"]

The one-line takeaway

Never quote an average without a spread beside it — std for tidy data, IQR when outliers lurk, range for the crude headline.

Where you'll meet it

Every "average" claim in a report — average response time, average spend, average score — is only trustworthy once you add spread: SLAs live or die on the IQR of latency, A/B tests compare std to know if a lift is real or noise, and box plots exist purely to draw the IQR. When a stakeholder says "but on average it's fine," spread is your evidence for whether "on average" is the whole story or a comforting lie.

Named concepts: standard deviation, interquartile range, range and variability