Natural Experiments

How economists find situations where chance or policy quirks create something like a randomised experiment, and the famous studies that used them.

Many important questions cannot be tested with a planned experiment. No one can randomly assign a country a new minimum wage or a wave of immigration. Instead, economists look for natural experiments: real events that happen to create a fair comparison between groups, as if chance had assigned them.

The 2021 Nobel prize

In 2021, the Nobel prize in economics was shared by David Card, for his empirical work on labour economics, and by Joshua Angrist and Guido Imbens, for their contributions to analysing causal relationships. Much of their work used natural experiments.

Two famous studies

The minimum wage. In 1992, New Jersey raised its minimum wage while neighbouring Pennsylvania did not. David Card and Alan Krueger surveyed fast-food restaurants on both sides of the border before and after the change. Because the two areas were similar, Pennsylvania restaurants showed what would likely have happened in New Jersey without the increase. They found no evidence that the higher minimum wage reduced employment. The study sparked a long debate and many follow-up studies.

The Mariel boatlift. In 1980, around 125,000 Cubans arrived in the United States within a few months after Cuba temporarily allowed emigration, and many settled in Miami, suddenly increasing its labour force by about seven percent. Card compared Miami’s labour market with similar cities. He found little effect on the wages or unemployment of existing workers. Later researchers have reanalysed the data and debated the findings.

A lottery for school places

When a popular school has more applicants than places, it may admit pupils by lottery. The lottery acts like a randomised experiment: winners and losers are similar on average. Comparing their later results shows the effect of attending that school, separate from the fact that motivated families tend to apply.

What makes a good natural experiment

The key question is whether the groups being compared were really similar before the event, so that the difference in outcomes can be attributed to the event itself. Economists describe the ideal as treatment that is as-if random. Critics of any natural experiment usually focus on this point.

Treating any before-and-after comparison as a natural experiment

A natural experiment needs a credible comparison group that was affected differently by the event for reasons unrelated to the outcome. Simply comparing a place before and after a policy change is not enough, because many other things may have changed at the same time.

Key takeaways

Natural experiments use real events that create fair comparisons, as if by chance.
Card, Angrist and Imbens shared the 2021 Nobel prize for work using and analysing such designs.
Card and Krueger's New Jersey minimum wage study and Card's Mariel boatlift study are famous examples.
A good natural experiment needs groups that were similar before the event.