Spotting Misleading Statistics

A listener's checklist for catching the most common ways true numbers are used to tell a misleading story.

Most misleading statistics are not false. The numbers are real, but they are chosen, framed, or compared in a way that leaves a wrong impression. This final lesson brings together the skills from this module into a practical checklist you can run in your head whenever you hear a number used to make an argument.

Cherry-picking the time window

Cherry-picking means selecting only the data that supports a point while leaving out the rest. The most common version is choosing a convenient start or end date. If a stock index fell sharply in one year and recovered the next, someone can describe it as “up 40 percent” by starting at the bottom, or “flat over two years” by starting before the fall. Both are true. Ask: why did the comparison start there, and what happens if you move the start date?

Base effects

A base effect happens when a growth rate looks unusually high or low because the period being compared against was unusual. After an economy shrinks sharply, as many did during the 2020 pandemic lockdowns, the following year’s growth rate can look spectacular simply because it is measured from a very low starting point. Hearing “the fastest growth in decades” right after a crash should prompt the question: “Compared with what?”

A shop's record growth

Suppose a shop sells 1,000 items a month in normal times. During a month when a flood closes the road, it sells only 200. Next year, in that same month, it sells 1,000 again. Compared with the flood month, sales rose 400 percent - a thrilling headline. But the shop is simply back to normal. The dramatic growth rate is entirely a base effect from the unusually low comparison month.

Framing and missing context

Framing is the way a number is presented. The same fact can be described as “a 90 percent success rate” or “a one-in-ten failure rate,” and listeners react differently. Earlier lessons covered other framing choices: percentages versus percentage points, mean versus median, totals versus per capita, nominal versus real, and annualized versus year-on-year. Each can make a number sound bigger or smaller without changing the facts.

Missing context is the silent partner of framing. A number with no comparison - “the government spent 5,000 crore rupees on this scheme” - is hard to judge. Is that large or small relative to the budget, the population served, or last year?

Survivorship bias

Survivorship bias happens when we only look at the cases that made it through some filter and ignore those that didn’t. Stories about how successful start-up founders dropped out of college ignore the many dropouts whose businesses failed. Reports of investment funds with strong long-term returns can quietly exclude funds that performed badly and were closed. Ask: who or what is missing from this picture?

A quick listening checklist

When a statistic is used to persuade you, run through these questions: Percent or percentage points? Which average? Out of how many? Adjusted for inflation and for the season? Compared with what, and starting when? Is it a survey, and is the change bigger than the margin of error? Correlation or cause? Who produced the number, and could it be revised? You won’t need every question every time, but even two or three will catch most misleading claims.

Rejecting all statistics as lies

A common mistake, after learning how numbers can mislead, is deciding that all statistics are untrustworthy. That throws away one of the best tools we have for understanding the economy. The goal is not cynicism but careful listening: ask good questions, and trust numbers more when they survive them.

Key takeaways

Most misleading statistics use true numbers chosen or framed selectively.
Cherry-picked time windows can make the same data tell opposite stories.
Base effects make growth look dramatic after an unusually low period.
Survivorship bias hides the cases that didn't make it.
A short checklist of questions catches most misleading claims without rejecting data altogether.