At this year’s meeting of the American Economic Association (AEA), David Card — the Class of 1950 Professor of Economics at the University of California, Berkeley, an Amazon Scholar, and the AEA’s incoming president — chaired three sessions, one of which was a tribute to the 2019 Nobel Prize winners in economics, Esther Duflo and Abhijit Bannerjee of MIT and Michael Kremer of Harvard University.
Duflo, Bannerjee, and Kremer are proponents of what’s sometimes called empirical economics: the attempt to put microeconomics on a more secure empirical footing through both field experiments and natural experiments. In natural experiments, circumstances conspire to enable the comparison of a population that receives an economic intervention with one that doesn’t.
One famous example of a natural experiment is a study that Card and his colleague Alan Krueger conducted in the early 1990s, in which they investigated fast-food markets in two adjacent, demographically similar regions in Pennsylvania and New Jersey. One region had recently seen a minimum-wage hike, and the other hadn’t, but employment growth remained similar in both.
The rise of empirical economics is frequently traced to a group of economists at Princeton University in the 1980s, which included Card, Krueger, and a graduate student who later became one of Duflo’s graduate advisors at MIT, Josh Angrist.
“The textbooks that I was taught out of had very elaborate models with very little — almost no — evidence,” Card says. “Oftentimes in economics, there are too many perfectly good theoretical models, and no one knows ‘Is that one right or this one right?’ So then it became, ‘Let's try and understand.’
“It became increasingly recognized in the 1970s and ’80s that there were some really big failures in our knowledge. My field is labor economics, and probably the worst failure of economists until very recently was to oversimplify how wages are determined. How do ketchup prices get determined? The store sets the price, right? And ketchup prices aren't exactly the same in different stores, and they change from month to month. But economists for many, many years were working with a model where any given worker got the same wage, no matter what employer they worked at. It took a lot of empirical work to show that the idea that employers hire workers, but they don't set wages, just isn't completely valid.”
Research design
Initially, the empirical work of Card and his colleagues met skepticism from many in the field. As Edward Leamer, an economics professor now at the University of California, Los Angeles, put it in a 1983 essay, “Hardly anyone takes data analysis seriously. Or perhaps more accurately, hardly anyone takes anyone else’s data analysis seriously.”
Empirical economists, the skeptics claimed, simply cherry-picked data to fit their hypotheses; the phenomena under investigation were so complex that correlations in the data couldn’t be taken to imply causation.
Card and his colleagues knew that, to make their analyses of natural experiments persuasive, they needed to control for other possible explanations of data correlations. Research design thus became one of the empirical economists’ chief concerns.
At Amazon, economists’ expertise in research design has two distinct applications. The first is its original application in empirical economics: making sense of large natural experiments.
The other is the interpretation and generalization of more conventional randomized experiments. To determine what types of information, presented in what format, customers find most useful, Amazon does frequent A/B testing, in which one group of customers sees one version of a web page, and another group sees another version.
Such tests can suffer from various confounders, however, such as spillover effects. Suppose that, in an A/B test, the presentation of information to group A — the treatment group — leads to increased purchase of a particular product. That product’s rise in popularity means that it’s recommended more frequently to shoppers — including those in group B, the control group.
“That's the kind of thing that economists can obsess over forever,” Card says. “One reason we’re useful is that we'll say, ‘Wait a minute, if that happens, the control group is no longer a control. The test results are unreliable.’
“That's the way economists think about things. The way I explain it to physicists — a lot of people who become economists train as physicists — is the particles are all fighting back. All the time. It's not that they're randomly going and filling the room. It's that actually, you try and get them in the room, and they're sneaking out the door. So that’s an order-of-magnitude harder problem.”
