The role of natural experiments in determining the outcomes of economic policies
23-Nov-2021 •Puja Mehra
How do you tell if an economic policy being rolled out will work effectively or not? Economic policies cannot be tested in laboratories the way medicines or other sciences can be. Economists cannot conduct rigidly controlled clinical trials. Nor can they conjure up floods or famines in order to examine policy outcomes and effectiveness. Therefore, one of the things economists, especially labour economists, do to overcome the constraint is that they rely on what are called 'natural experiments'. For this, economists analyse data that is naturally observable to infer the real-world outcomes of economic policies by establishing cause and effect.
This year's Nobel for economics recognises the contribution of such analyses in making the subject more evidence-based. The award has been won by three economists, David Card, Joshua Angrist and Guido Imbens. Their work since the 1990s was a turning point and has completely reshaped the field of economics. The methods pioneered by these economists are now routinely used in many areas of economic research.
Card, who works at the University of California, Berkeley, conducted ground-breaking experiments, often in partnership with the late economist Alan Krueger, in the early 1990s in New Jersey in the fast-food sector that went on to show that an increase in the minimum wage did not always lead to job losses. These were counter-intuitive findings and this research ultimately went on to become empirical evidence used as a basis for the Biden administration's legislation for a $15 minimum wage in the United States.
Another of his studies focused on the impact on wages for Miami residents after Fidel Castro in 1980 allowed all Cubans who wished to leave the country to do so. Remember the Hollywood blockbuster 'Scarface' that is considered the greatest gangster film ever made? In the film, Cuban refugee and ex-convict Tony Montana, played by the star Al Pacino, arrives penniless in Miami, initially becomes a dishwasher at an eatery, but is not satisfied, and declares that he is cut-out for bigger things. He goes on to become a powerful drug lord.
In off-reel life too, Castro's decision led to high levels of migration to Miami. Traditional economics held that such influx of immigrants ought to lower native-born workers' wages. But Card found that the incoming migrants from Cuba had no negative wage or labour effects for Miami residents with low levels of education and skills. This finding completely upended conventional wisdom about the impact of immigration. Plus, these findings also revolutionised thinking about how economic studies should be conducted.
Before this, economists used to approach the topic of minimum wage and its impact mostly in theoretical terms. Card and Krueger, however, were taken in by research methods used in medicine that involved randomised experiments. What researchers in the medical field do is that they randomly divide people into statistically identical groups. They then administer a pill or treatment being tested to only one of the groups and compare the outcomes of both groups to produce evidence about the nature of the effects of medicine. Card and Krueger wanted to do the same for testing the impact of policies on people, which is arguably a much tougher proposition, involving formidable logistical challenges. So, they formulated an alternative method. They saw the opportunity for a 'natural experiment' when the state of New Jersey raised the minimum wage. For this, while fast-food restaurants in New Jersey became the treatment group, those across the state border, in Eastern Pennsylvania, were the control group. Looking at the changes in employment in each state, they found that a minor increase in the minimum wage had not killed jobs. The economists employed similar natural experiments to study numerous other policy matters.
Angrist, a dual US/Israeli citizen, works at Massachusetts Institute of Technology (MIT). Imbens is at Stanford University. They further built on Card's work to design new methods of conducting 'natural experiments' in economics. These studies help us better understand cause and effect in complex social situations. Changes in government policy for schools and healthcare, etc., can create situations in which natural experiments can be conducted for understanding the effects of policies. The Nobel citation records the impact in the field of a paper that Angrist and Imbens co-wrote in 1994. The basic idea of the paper is that the average effect of something - a new government policy, a hike in wage rates, acquiring work experience or a new skill, or earning an educational qualification - is best measured by its impact on people who normally would never have experienced it.
Angrist and Krueger, in a paper published in 1991 on the relationship between education and lifetime earnings, studied decades of data from the 1920s onwards and found that an additional year of education was worth about 7.5 per cent more in annual earnings. The co-authors used varying state laws about the age at which students could drop out of school for the study. The paper has since appeared in many textbooks as a classic example of a natural experiment. A more recent paper Angrist co-authored in 2020 turned the empirical lens on economics itself. It found that the portion of papers based on empirical research, in contrast with those based on theory or methodology, has increased by about 20 percentage points since 1990. In another recent project, Angrist and his fellow researchers found that the reason graduates of elite public schools tend to do well has more to do with selective admissions than with the education they provide.
Imbens and Angrist tested the impact of a policy of guaranteed income on the labour force in a unique way. They looked at a Massachusetts lottery, in which winners received their pay-outs over a number of years in a manner that resembled a guaranteed income. Then they compared lottery winners with those who did not win. They found that the 'guaranteed income' had had some impact on the labour supply but workers did not drastically reduce the amount they worked.
Puja Mehra is a Delhi-based journalist and author of 'The Lost Decade (2008-18): How India's Growth Story Devolved Into Growth Without A Story'