Economists often express incredulity toward people who target their criticisms at an amorphous entity called ‘economics’ (perhaps prefixed with ‘neoclassical’ or ‘mainstream’), instead of targeting specific areas of the discipline. They point out that, contrary to the popular view of economists as a group who are excessively concerned with theory, a majority of economic papers are empirical. Sometimes, even the discipline’s most vehement defenders are happy to disown the theoretical areas-such as macroeconomics-which attract the most criticism, whilst still insisting that, broadly speaking, economists are a scientifically minded bunch.
Perhaps surprisingly, I agree somewhat with this perspective. I think there is a disconnect within economics: between the core theories (neoclassical economics, or marginalism) and econometrics.* I believe the former to be logically, empirically and methodologically unsound. However, I believe the latter – though not without its problems – has all the hallmarks of a much better way to do ‘science’. There are several reasons to believe this:
First, econometrics has a far more careful approach to assumptions than marginalism. To start with, you are simply made more aware of the assumptions you use, whereas I find many are implicit in marginalist theory. Furthermore, there is extensive discussion of each individual assumption’s impact, of what happens when each assumption is relaxed, and of what we can do about it. For example: if your time-series data are not weakly stationary (loosely speaking, this means the data oscillates around the same average, with the size of the oscillations also staying, on average, roughly the same, like this) you simply cannot use Ordinary Least Squares (OLS) regression. There is no suggestion that, even though the assumption is false, we can use it as an approximation, or to highlight a key aspect of the problem, or other such hand waving. The method is simply invalidated, and we must use another method, or different data. Such an approach is refreshing and completely at odds with marginalist theory, whose proponents insist on clinging to models – and even applying them broadly – despite a wealth of absurdly unrealistic assumptions.
Second, econometrics has dealt with criticisms far better and more fundamentally than its theoretical counterpart. The most broad and pertinent criticism of econometrics was delivered by Edward Leamer in his classic paper ‘Let’s Take the Con Out of Econometrics’. Leamer highlighted the ‘identification problem’ inevitably faced by econometricians. Since econometricians try to isolate causal links, but can rarely do controlled experiments, they must pick and choose which variables they want to include in their model. Yet there are so many variables in the real world that we cannot discern, a priori, which ones are really the ‘key culprits’ in our purported causal chain, so inevitably this choice is something of a judgment call.
The result is that two different econometricians can use econometrics to paint two very different pictures, based on their choice of model. For example, David Hendry famously showed that the link between inflation and rainfall – whichever way it ran – was quite robust. Unfortunately, such absurdity can be much harder to detect in the murky waters of economic data, making purported causal links highly suspect. Leamer chastised his colleagues (and himself) for basing their choice of included variables and key assumptions on “whimsy”, making inference results highly subject to change based on the biases of the author, and which direction they (consciously or unconsciously) have pointed the data in. He pointed out that data on what exactly impacts murder rates could give wildly disparate results based on a few key decisions made by the practitioner.
However, the discipline has, in my opinion, taken the challenge seriously. In 2010, Joshua Angrist & Jörn-Steffen Pischke responded to Leamer, summing up some key changes in the way econometricians use and interpret data. I’ll briefly highlight a few of them:
(1) An increase in the use of data from quasi-randomised trials, whether intended or by ‘natural experiment’. Econometricians have increased the use of the former where they can, but real experiments are hard to come by in social sciences, so they are generally stuck with the latter. One such example of a natural experiment is the ‘differences-in-differences’ approach, which uses natural boundaries such as nation states to estimate whether certain variables are key causal factors. If the murder rate follows roughly the same trend in both the US and Canada, then the trend is surely not attributable to changes in policy. Such quasi-experiments eliminate the problem even more fundamentally than Leamer imagined it could be, by vastly improving the raw data.
(2) More common, careful use of methods intended to isolate causality, such as the use of Instrumental Variables (IV). The basic idea here is this: if we have an independent variable x, and a dependent variable y, correlation between them does not imply causation from x to y. So one way we can support the hypothesis of a causal link is by using another variable z, which influences x directly, but doesn’t influence y. In other words, z should only affect y through its influence on x, and if we find a correlation, this is consistent with the idea of a causal link.
To borrow an example from Wikipedia, consider smoking and health outcomes. We may find a correlation between smoking rates and worse health outcomes, and intuitively suppose that the causation runs from smoking. But ultimately, intuition isn’t enough. So we could use tobacco taxes – which surely affect health outcomes only because they influence smoking rates – as an instrument, and see if they are correlated with worse health outcomes. If they are, then this supports our initial hypothesis; if not, it may be an issue of reverse causation, or some third cause which impacts both smoking and health outcomes. IV and other methods like it are not exhaustive, but they certainly bring us closer to the truth, which is surely what science is about.
(3) More transparency in, and discussion of, research designs, so that results can be verified and others can (try to) replicate them. It is worth noting that, though Reinhart and Rogoff’s 90% threshold was junk science, they were exposed relatively soon after their data were made available.
The result of all these efforts is that econometrics is much more credible than it was when Leamer wrote his article in 1983 (at which time everyone seemed to agree it was fairly worthless). Hopefully it will continue to improve on this front.
A final, albeit less fundamental, reason I prefer econometrics and econometricians is that the nature of the field, with its numerous uncertainties, naturally demands a more modest interpretation of results. The rigid and hard-to-master framework of neoclassical theory often seems to give those who’ve mastered it the idea that they have been burdened with secret truths about the economy, which they are all too happy to parade on the op-ed pages of widely read papers. In contrast, you are unlikely to find Card & Krueger blithely asserting that the minimum wage has a positive effects on employment, and that anyone who disagrees with them just doesn’t understand econometrics. Perhaps this is just due to differences in the types of people that do theory versus those who do evidence, but I’d be willing to bet it is symptomatic of the generally more measured approach taken by econometricians.
The way forward?
I believe it would be a positive step for economists to opt for theoretical methods more resembling the econometric approach, preferring observed empirical regularities and basic statistical relationships to ‘rigorous’ theory. In fact, I have previously seen Steve Keen’s model referred to as ‘econometrics’, and perhaps this is broadly right in a sense. But it’s more of a compliment than an insult: ditching the straitjacket of marginalism, with its various restrictive assumptions (coupled with insistence that we simply can’t do it any other way), and heading for simple stock-flow relationships between various economic entities could well be a step forwards. It will of course seem like a step backwards to most economists, but then, highly complex models are not correct just because they are highly complex.
As for the Lucas Critique, well, statistical regularities that may collapse upon exploitation can be taken on a case-by-case basis: it’s actually not that difficult to foresee, and even the ‘Bastard-Keynesians’ saw it in the Phillips Curve (as did Keynes). Ironically, it seems economists themselves, blindly believing that they have ‘solved’ this problem, are least aware of it, having only a shallow interpretation of its implications (seemingly, as a gun that fires left). A more dynamic awareness of the relationship between policy and the economy would be a more progressive approach than being shackled by microfoundations.
I am half expecting my regular readers to point out 26723 problems with econometrics that I have not considered. To be sure, econometrics has problems: inferring causality will forever be an issue, as will the cumulative effects of the inevitable judgments calls involved in dealing with data. No doubt, econometrics is prone to misuse. However, it seems to me that most of the problems with econometrics are simply those experienced in all areas of statistics. This is at least a start: I would love, one day, to be able to say that the problems with economic theory were merely those experienced by all social sciences.
*Indeed, this blog would be more accurately titled ‘Unlearning Marginalism’, but obviously that wouldn’t be as catchy or