How Economists Know Things: Evidence & Experiments
Difference-in-Differences, Explained in Words
A widely used method that compares changes over time between an affected group and an unaffected one, and the assumption it depends on.
One of the most widely used tools in applied economics has an awkward name: difference-in-differences. The idea behind it is simple, and it was the method behind the famous New Jersey minimum wage study.
Two differences
Suppose a city introduces a new policy and we want to know its effect on employment. We could compare employment before and after the policy. But employment might have changed anyway because of the national economy. So we find a similar city without the policy, called the comparison group.
The method takes two differences:
- The change in the city with the policy, before versus after.
- The change in the comparison city over the same period.
The difference between those two changes is the estimated effect of the policy. Subtracting the comparison city’s change removes the influence of things that affected both cities, like a national recession.
Before a new job programme, City A has 60 percent of adults in work and City B has 55 percent. A year later, City A has 64 percent and City B has 57 percent. City A rose by 4 points and City B by 2 points. The difference-in-differences estimate is 4 minus 2, or 2 percentage points. That 2 points is the estimated effect of the programme, after removing the general rise both cities experienced.
The key assumption
Difference-in-differences depends on the parallel trends assumption: without the policy, both groups would have changed in the same way. The two cities do not need to start at the same level, but their trends must be similar.
Researchers check this by looking at trends before the policy. If both cities were moving in parallel for years beforehand, that makes the assumption more believable. If the treated city was already improving faster, the method will wrongly credit the policy with that improvement.
Where it is used
The method is used to study minimum wages, tax changes, health reforms, school policies and many other questions where a policy affects some places or groups but not others. Economists have recently developed improved versions for cases where different places adopt a policy at different times.
The whole method rests on the comparison group being a good stand-in for what would have happened. Choosing a comparison group only because data is available, without checking that its trend matched beforehand, can produce badly misleading results.
- Difference-in-differences compares the change in an affected group with the change in an unaffected group.
- Subtracting the comparison group's change removes shared influences like national trends.
- It depends on the parallel trends assumption.
- Checking trends before the policy helps judge whether the assumption is believable.
No recording for this one yet - EconReader can read it aloud for you.