DA-01-001-P · Primer — The EDA Mindset: Asking Good Questions of Data

The idea in one sentence

Exploratory data analysis (EDA) is the habit of interrogating a dataset with genuine curiosity — asking "what does this reveal, and what would fool me?" — before you rush to a chart, a metric, or a conclusion.

Why it exists

The fastest way to be confidently wrong about data is to skip straight to the answer you already expected. You pull the numbers, drop them into a bar chart, see the bar you hoped for, and stop — never noticing the duplicate rows, the one gigantic outlier, or the fact that "average" is being dragged around by a single weird day. EDA exists as the discipline that slows that reflex down. Its whole purpose is to understand the shape of the data — where it's clean, where it's broken, what's typical, what's surprising — so that any later claim rests on something real. John Tukey, who coined the term in 1977, framed it as detective work: you're not proving a case yet, you're gathering clues and letting the data suggest which questions are even worth asking. Get this mindset right and everything downstream (dashboards, models, decisions) inherits your care; skip it and you're just decorating a guess.

How it works

Start with a question, not a chart — something a human actually cares about, like "Which day of the week is unusual, and is that a real pattern or a fluke?" A good question is falsifiable: it tells you what evidence would confirm it and what would knock it down. This is the split between exploratory analysis (open-ended: "what's going on here?") and confirmatory analysis (narrow: "is signups-on-Sunday-really-higher true, yes or no?") — you almost always do the first to find hypotheses and the second to test them. In pandas the loop is fast: peek at the raw rows, compute a summary like a median or describe(), draw one simple plot, and — critically — ask what could have faked the pattern you see (an outlier? a duplicate? too little data?). Each answer sharpens the next question, which is iterative refinement: your understanding tightens one small pass at a time. Below, a single "surprising" bar turns into a new question rather than a conclusion.

import pandas as pd
import matplotlib.pyplot as plt

# A question, then data — not the other way around:
# "Which day is unusual, and is it a real signal or a fluke?"
df = pd.DataFrame({
    "day":     ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"],
    "signups": [12, 15, 9, 14, 11, 2, 47],
})

median = df["signups"].median()          # a robust "typical day" = 12.0
df["gap"] = df["signups"] - median       # how far each day sits from normal
print(df)
print("Biggest surprise:",
      df.loc[df["gap"].abs().idxmax(), "day"])   # -> Sun (47, way above 12)

plt.bar(df["day"], df["signups"])
plt.axhline(median, linestyle="--")      # dashed line = "a normal day"
plt.title("Daily signups vs. the typical day")
plt.tight_layout()
plt.show()
# The chart shows Sun towering over the dashed line. That is a QUESTION,
# not an answer: real launch? a bot? a holiday? one week isn't enough to know.

The one gotcha

The dangerous moment is the instant the chart confirms what you hoped — that's exactly when you stop looking. Here the Sunday bar could mean "Sundays are strong" or it could be one bot signup spree, one marketing email, or simply that seven days is far too little data to call any pattern real. EDA's core move is to keep asking "what would fool me?" for one more beat: check for duplicates, look at more weeks, see if the median (which ignored the outlier) disagrees with the mean (which got yanked up to ~16 by Sunday). A single striking bar is a lead to investigate, never a verdict to ship.

See it — from curious question to sharper next question

flowchart TD
  A["Ask a real question<br/>(what is unusual here?)"] --> B["Peek at raw rows<br/>+ a summary (median)"]
  B --> C["Draw ONE simple chart"]
  C --> D{"Surprising?"}
  D -->|"No"| E["Ask a different<br/>question"]
  D -->|"Yes"| F["What could<br/>fool me here?"]
  F --> G{"Outlier, dup,<br/>or too little data?"}
  G -->|"Maybe"| H["Refine: more data,<br/>re-check the claim"]
  G -->|"Ruled out"| I["Now confirm it<br/>(confirmatory test)"]
  H --> A

The one-line takeaway

EDA is asking good, falsifiable questions of your data and staying suspicious of your own charts — explore to find the pattern, then confirm to trust it.

Where you'll meet it

Every real analysis begins here: before any dashboard, A/B test, or model, you'll open a fresh dataset and spend your first hour just asking what's typical, what's broken, and what's surprising. You'll feel it when a stakeholder says "our numbers spiked Tuesday!" — the EDA reflex is to ask what would fool me? before agreeing. It's the groundwork under describe(), groupby, outlier hunts, and the "let me plot it first" instinct that separates careful analysts from chart-decorators.

Named concepts: exploratory vs confirmatory analysis, question-driven exploration, iterative refinement