Reading Economic Data
Margins of Error
Why every survey-based number is really a range, and how to tell whether a reported change is meaningful or just noise.
Because surveys ask only a sample of people, their results are estimates, not exact counts. Ask a different random sample and you would get a slightly different answer. The margin of error tells you how far off an estimate might reasonably be because of this chance variation. When a report says “42 percent, with a margin of error of plus or minus 3 percentage points,” it means the true figure for the whole population is most likely somewhere between 39 and 45 percent.
Hearing a number as a range
A useful habit is to mentally turn every survey figure into a range. That range is often called a confidence interval. Statisticians commonly use a 95 percent confidence level, which roughly means that if the survey were repeated many times with fresh random samples, about 95 out of 100 of those intervals would contain the true value.
The margin of error mainly depends on the size of the sample. Larger samples give smaller margins, but with diminishing returns. A well-designed random sample of around 1,000 people typically has a margin of error of roughly plus or minus 3 percentage points for a result near 50 percent. To cut that margin in half, you would need roughly four times as many people.
When is a change real?
This matters most when you hear about changes. Suppose unemployment is reported at 6.2 percent one month and 6.4 percent the next. If the margin of error on each estimate is larger than that difference, the change could simply be noise - random wobble from sampling - rather than a real shift. Statisticians call a change statistically significant when it is large enough, relative to the uncertainty, that chance alone is an unlikely explanation.
Official releases sometimes note when a change is “not statistically significant.” That phrase is worth listening for. It does not mean nothing happened; it means the data cannot tell a real change apart from random variation.
Suppose one survey finds that 48 percent of small businesses expect to hire this year, and a second survey a month later finds 51 percent. A headline might say "hiring optimism rises." But if each survey has a margin of error of plus or minus 4 percentage points, the first result could really be anywhere from 44 to 52 percent, and the second anywhere from 47 to 55 percent. Those ranges overlap heavily, so the data cannot confidently show any change at all. A careful listener would hear this as "roughly half of small businesses expect to hire, about the same as last month."
Margins for smaller groups
Margins of error grow when a survey reports on a subgroup, such as one state, one age group, or one industry. The subgroup’s sample is smaller, so its estimate is less precise. A national figure might be quite reliable while a figure for a single small region from the same survey is much less so.
A common mistake is hearing a small rise or fall in a survey result and treating it as meaningful. If the change is smaller than the margin of error, it may simply be sampling noise. Wait for a consistent pattern over several releases before concluding that something has really changed.
- Survey results are estimates, and the margin of error shows how uncertain they are.
- Hear every survey figure as a range, not a single exact number.
- A random sample of about 1,000 typically has a margin of roughly plus or minus 3 points.
- Changes smaller than the margin of error may be noise, not real shifts.
- Estimates for small subgroups are less precise than national figures.
No recording for this one yet - EconReader can read it aloud for you.