Posts Tagged Econometrics
Something has been bothering me about the way evidence is (sometimes) used in economics and econometrics: theories are assumed throughout interpretation of the data. The result is that it’s hard to end up questioning the model being used.
Let me give some examples. The delightful fellas at econjobrumours once disputed my argument that supply curves are flat or slope downward by noting that, yes, Virginia, in conditions where firms have market power (high demand, drought pricing) prices tend to go up. Apparently this “simple, empirical point” suffices to refute the idea that supply curves do anything but slope upward. But this is not true. After all, “supply curves slope downward/upward/wiggle around all over the place” is not an empirical statement. It is an interpretation of empirical evidence that also hinges on the relevance of the theoretical concept of the supply curve itself. In fact, the evidence, taken as whole, actually suggests that the demand-supply framework is at best incomplete.
This is because we have two major pieces of evidence on this matter: higher demand/more market power increases price, and firms face constant or increasing returns to scale. These are contradictory when interpreted within the demand-supply framework, as they imply that the supply curve slopes in different directions. However, if we used a different model – say, added a third term for ‘market power’, or a Kaleckian cost plus model, where the mark up was a function of the “degree of monopoly”, that would no longer be the case. The rising supply curve rests on the idea that increasing prices reflect increasing costs, and therefore cannot incorporate these possibilities.
Similarly, many empirical econometric papers use the neoclassical production function, (recent one here) which states that output is derived from the labour and capital, plus a few parameters attached to the variables, as a way to interpret the data. However, this again requires that we assume capital and labour, and the parameters attached to them, are meaningful, and that the data reflect their properties rather than something else. For example, the volume of labour employed moving a certain way only implies something about the ‘elasticity of substitution’ (the rate at which firms substitute between labour and capital) if you assume that there is an elasticity of substitution. However, the real-world ‘lumpiness‘ of production may mean this is not the case, at least not in the smooth, differentiable way assumed by neoclassical theory.
Assuming such concepts when looking at data means that economics can become a game of ‘label the residual‘, despite the various problems associated with the variables, concepts and parameters used. Indeed, Anwar Shaikh once pointed out that the seeming consistency between the Cobb-Douglas production function and the data was essentially tautological, and so using the function to interpret any data, even the word “humbug” on a graph, would seem to confirm the propositions of the theory, simply because they follow directly from the way it is set up.
Joan Robinson made this basic point, albeit more strongly, concerning utility functions: we assume people are optimising utility, then fit whatever behaviour we observe into said utility function. In other words, we risk making the entire exercise “impregnably circular” (unless we extract some falsifiable propositions from it, that is). Frances Wooley’s admittedly self-indulgent playing around with utility functions and the concept of paternalism seems to demonstrate this point nicely.
Now, this problem is, to a certain extent, observed in all sciences – we must assume ‘mass’ is a meaningful concept to use Newton’s Laws, and so forth. However, in economics, properties are much harder to pin down, and so it seems to me that we must be more careful when making statements about them. Plus, in the murky world of statistics, we can lose sight of the fact that we are merely making tautological statements or running into problems of causality.
The economist might now ask how we would even begin to interpret the medley of data at our disposal without theory. Well, to make another tired science analogy, the advancement of science has often not resulted from superior ‘predictions’, but on identifying a closer representation of how the world works: the go-to example of this is Ptolemy, which made superior predictions to its rival but was still wrong. My answer is therefore the same as it has always been: economists need to make better use of case studies and experiments. If we find out what’s actually going on underneath the data, we can use this to establish causal connections before interpreting it. This way, we can avoid problems of circularity, tautologies, and of trapping ourselves within a particular model.
Economists often express incredulity toward people who target their criticisms at an amorphous entity called ‘economics’ (perhaps prefixed with ‘neoclassical’ or ‘mainstream’), instead of targeting specific areas of the discipline. They point out that, contrary to the popular view of economists as a group who are excessively concerned with theory, a majority of economic papers are empirical. Sometimes, even the discipline’s most vehement defenders are happy to disown the theoretical areas-such as macroeconomics-which attract the most criticism, whilst still insisting that, broadly speaking, economists are a scientifically minded bunch.
Perhaps surprisingly, I agree somewhat with this perspective. I think there is a disconnect within economics: between the core theories (neoclassical economics, or marginalism) and econometrics.* I believe the former to be logically, empirically and methodologically unsound. However, I believe the latter – though not without its problems – has all the hallmarks of a much better way to do ‘science’. There are several reasons to believe this:
First, econometrics has a far more careful approach to assumptions than marginalism. To start with, you are simply made more aware of the assumptions you use, whereas I find many are implicit in marginalist theory. Furthermore, there is extensive discussion of each individual assumption’s impact, of what happens when each assumption is relaxed, and of what we can do about it. For example: if your time-series data are not weakly stationary (loosely speaking, this means the data oscillates around the same average, with the size of the oscillations also staying, on average, roughly the same, like this) you simply cannot use Ordinary Least Squares (OLS) regression. There is no suggestion that, even though the assumption is false, we can use it as an approximation, or to highlight a key aspect of the problem, or other such hand waving. The method is simply invalidated, and we must use another method, or different data. Such an approach is refreshing and completely at odds with marginalist theory, whose proponents insist on clinging to models – and even applying them broadly – despite a wealth of absurdly unrealistic assumptions.
Second, econometrics has dealt with criticisms far better and more fundamentally than its theoretical counterpart. The most broad and pertinent criticism of econometrics was delivered by Edward Leamer in his classic paper ‘Let’s Take the Con Out of Econometrics’. Leamer highlighted the ‘identification problem’ inevitably faced by econometricians. Since econometricians try to isolate causal links, but can rarely do controlled experiments, they must pick and choose which variables they want to include in their model. Yet there are so many variables in the real world that we cannot discern, a priori, which ones are really the ‘key culprits’ in our purported causal chain, so inevitably this choice is something of a judgment call.
The result is that two different econometricians can use econometrics to paint two very different pictures, based on their choice of model. For example, David Hendry famously showed that the link between inflation and rainfall – whichever way it ran – was quite robust. Unfortunately, such absurdity can be much harder to detect in the murky waters of economic data, making purported causal links highly suspect. Leamer chastised his colleagues (and himself) for basing their choice of included variables and key assumptions on “whimsy”, making inference results highly subject to change based on the biases of the author, and which direction they (consciously or unconsciously) have pointed the data in. He pointed out that data on what exactly impacts murder rates could give wildly disparate results based on a few key decisions made by the practitioner.
However, the discipline has, in my opinion, taken the challenge seriously. In 2010, Joshua Angrist & Jörn-Steffen Pischke responded to Leamer, summing up some key changes in the way econometricians use and interpret data. I’ll briefly highlight a few of them:
(1) An increase in the use of data from quasi-randomised trials, whether intended or by ‘natural experiment’. Econometricians have increased the use of the former where they can, but real experiments are hard to come by in social sciences, so they are generally stuck with the latter. One such example of a natural experiment is the ‘differences-in-differences’ approach, which uses natural boundaries such as nation states to estimate whether certain variables are key causal factors. If the murder rate follows roughly the same trend in both the US and Canada, then the trend is surely not attributable to changes in policy. Such quasi-experiments eliminate the problem even more fundamentally than Leamer imagined it could be, by vastly improving the raw data.
(2) More common, careful use of methods intended to isolate causality, such as the use of Instrumental Variables (IV). The basic idea here is this: if we have an independent variable x, and a dependent variable y, correlation between them does not imply causation from x to y. So one way we can support the hypothesis of a causal link is by using another variable z, which influences x directly, but doesn’t influence y. In other words, z should only affect y through its influence on x, and if we find a correlation, this is consistent with the idea of a causal link.
To borrow an example from Wikipedia, consider smoking and health outcomes. We may find a correlation between smoking rates and worse health outcomes, and intuitively suppose that the causation runs from smoking. But ultimately, intuition isn’t enough. So we could use tobacco taxes – which surely affect health outcomes only because they influence smoking rates – as an instrument, and see if they are correlated with worse health outcomes. If they are, then this supports our initial hypothesis; if not, it may be an issue of reverse causation, or some third cause which impacts both smoking and health outcomes. IV and other methods like it are not exhaustive, but they certainly bring us closer to the truth, which is surely what science is about.
(3) More transparency in, and discussion of, research designs, so that results can be verified and others can (try to) replicate them. It is worth noting that, though Reinhart and Rogoff’s 90% threshold was junk science, they were exposed relatively soon after their data were made available.
The result of all these efforts is that econometrics is much more credible than it was when Leamer wrote his article in 1983 (at which time everyone seemed to agree it was fairly worthless). Hopefully it will continue to improve on this front.
A final, albeit less fundamental, reason I prefer econometrics and econometricians is that the nature of the field, with its numerous uncertainties, naturally demands a more modest interpretation of results. The rigid and hard-to-master framework of neoclassical theory often seems to give those who’ve mastered it the idea that they have been burdened with secret truths about the economy, which they are all too happy to parade on the op-ed pages of widely read papers. In contrast, you are unlikely to find Card & Krueger blithely asserting that the minimum wage has a positive effects on employment, and that anyone who disagrees with them just doesn’t understand econometrics. Perhaps this is just due to differences in the types of people that do theory versus those who do evidence, but I’d be willing to bet it is symptomatic of the generally more measured approach taken by econometricians.
The way forward?
I believe it would be a positive step for economists to opt for theoretical methods more resembling the econometric approach, preferring observed empirical regularities and basic statistical relationships to ‘rigorous’ theory. In fact, I have previously seen Steve Keen’s model referred to as ‘econometrics’, and perhaps this is broadly right in a sense. But it’s more of a compliment than an insult: ditching the straitjacket of marginalism, with its various restrictive assumptions (coupled with insistence that we simply can’t do it any other way), and heading for simple stock-flow relationships between various economic entities could well be a step forwards. It will of course seem like a step backwards to most economists, but then, highly complex models are not correct just because they are highly complex.
As for the Lucas Critique, well, statistical regularities that may collapse upon exploitation can be taken on a case-by-case basis: it’s actually not that difficult to foresee, and even the ‘Bastard-Keynesians’ saw it in the Phillips Curve (as did Keynes). Ironically, it seems economists themselves, blindly believing that they have ‘solved’ this problem, are least aware of it, having only a shallow interpretation of its implications (seemingly, as a gun that fires left). A more dynamic awareness of the relationship between policy and the economy would be a more progressive approach than being shackled by microfoundations.
I am half expecting my regular readers to point out 26723 problems with econometrics that I have not considered. To be sure, econometrics has problems: inferring causality will forever be an issue, as will the cumulative effects of the inevitable judgments calls involved in dealing with data. No doubt, econometrics is prone to misuse. However, it seems to me that most of the problems with econometrics are simply those experienced in all areas of statistics. This is at least a start: I would love, one day, to be able to say that the problems with economic theory were merely those experienced by all social sciences.
*Indeed, this blog would be more accurately titled ‘Unlearning Marginalism’, but obviously that wouldn’t be as catchy or