Climate capers at Cato

NOTE: The code and data used to produce all of the figures in this post can be found here.

Having forsworn blogging activity for several months in favour of actual dissertation work, I thought I'd mark a return to Stickman's Corral in time for the holidays. Our topic for discussion today is a poster (study?) by Cato Institute researchers, Patrick Michaels and "Chip" Knappenberger.

Michaels & Knappenberger (M&K) argue that climate models predicted more warming than we have observed in the global temperature data. This is not a particularly new claim and I'll have more to say about it generally in a future post. However, M&K go further in trying to quantify the mismatch in a regression framework. In so doing, they argue that it is incumbent upon the scientific community to reject current climate models in favour of less "alarmist" ones. (Shots fired!) Let's take closer look at their analysis, shall we?

In essence, M&K have implemented a simple linear regression of temperature on a time trend,
\begin{equation}
Temp_t = \alpha_0 + \beta_1 Trend + \epsilon_t.
\end{equation}
This is done recursively, starting from 2014 and incrementing backwards one year at a time until the sample extends until the middle of the 20th century. The key figure in their study is the one below, which compares the estimated trend coefficient, $\hat{\beta_1}$, from a bunch of climate models (the CMIP5 ensemble) with that obtained from observed climate data (global temperatures as measured by the Hadley Centre's HadCRUT4 series).

Since the observed warming trend consistently falls below that predicted by the suite of climate models, M&K conclude:  "[A]t the global scale, this suite of climate models has failed. Treating them as mathematical hypotheses, which they are, means that it is the duty of scientists to reject their predictions in lieu of those with a lower climate sensitivity."

Bold words. However, not so bold on substance. M&K's analysis is incomplete and their claims begin to unravel under further scrutiny. I discuss some of these shortcomings below the fold.

Obligatory comment on the 2013 Nobelists

Seeing as it is very de jour to comment on this sort of thing in the econ blogosphere, here is a quick personal take:

I know that this year's laureates have raised eyebrows -- not least of all because people think that Fama and Shiller are at complete odds with one another. This doesn't strike me as especially correct. (Hansen is really the odd one out in this triumvirate, but we'll get to him in a second). For starters, and as pointed out many times over the last two days, Fama was one of the first people to publish results that ran counter to EMH predictions. Mark Thoma is exactly right in pointing out the EMH remains a really useful benchmark/framework for thinking about markets in an empirical sense. I've used it a fair bit when looking at energy and commodity markets for my own research and also when asked to to advise/comment on market trends. 

Shiller has played less of a formal role for me personally, though his housing index and his "dividend returns" data have been extremely handy tools in the blogosphere. The former is better known, but the latter is especially useful when, say, debating your average goldbug. (E.g. When dividends are taken into account, U.S. stocks have enjoyed inflation-adjusted returns of +/-1,000% since 1974. Gold, on the other hand, has yielded a rather more modest 130% over the same time period...)

The 2013 Nobelist who has had the most relevance for me, however, is Lars Peter Hansen. I suspect that this is true for many people working in economic research today, simply because the tools that he bequeathed us are so widely used in modern empirical work. Alex Tabarrok has one of the best "layman" explanations of GMM that I've seen here. Guan Yang has a more wonkish (but still accessible to anyone who is familiar with basic econometrics) exposition here.

The more you read about ABCT... the more you read about ABCT

Chris responded last week to my previous post on the empirical (ir?)relevance of ABCT. I've been too busy to reply properly until now. (To be honest, I think that my original points remain intact.) I should also say that neither of us can afford to keep this dialogue going on for much longer. Still, here are some excerpts from his latest post, followed by my comments.

First, on the challenge of trying to distinguish between business processes that are fundamentally short-term in nature versus those of the longer-term:

My dad’s business, for instance, does multiple short-term contracting projects within long-term property development projects. In the normal production structure distribution his irrigation installations would be classified near the consumer level as it sits very close to final consumption, but he prices projects at the outset of long-term investment projects when the developer begins to plan and commence his project. My dad’s business therefore adjusts prices early in the business cycle at the same time that projects more remote of the consumer do, and will continue to price for projects throughout the period of the long-term project.

Unlike Chris' initial post, where he was bemoaning the use of statistical indices, I regard this as a more interesting observation. Yes, it is true that firms with short-term production horizons will in some sense be dependent on the activity of (other) firms with longer-term production horizons. However, I still don't regard this as a decisive barrier to an empirical investigation into ABCT.[*] First note, however, that Chris' objection could be seen as theoretical critique of ABCT as much as an empirical one. For if his remarks hold true, then it is extremely difficult even in principle to distinguish the way in which, say, products closer to the end consumer are made less attractive by a fall in interest rates. The mechanics of the classic (naive?) Hayekian triangle begin to unravel, since the underlying distortions -- the switch into capital goods at the expense of consumption goods during an initial period of credit expansion -- may not even occur in a qualitative sense. Indeed, if processes all along the chain of production benefit from credit expansion then we are closer to a theory of economic growth than of business cycles.

Nevertheless, what really matters in this case is the change in relative prices. If you buy the insights provided by ABCT, then it seems extremely implausible that conditions inherently favourable to long-term production processes could benefit short-term processes to a near (or even greater) extent, merely through the creation of auxiliary demand. This is particularly true if the economy is operating at anywhere near full capacity, as is typically emphasised as the starting point for their analysis by Hayek and Mises... i.e. Any increase in capital goods production must increasingly come at the expense of consumer goods.[**] The focus of Lester and Wolff (2013) was the changing nature of such relative prices. It therefore seems a perfectly valid approach from my perspective and, moreover, the failure of the data to conform to the theory's broad predictions, or show signs of economic/statistical significance, is indeed cause for scepticism of ABCT's relevance. A final point on this matter is that L&W trace the evolution of these relative prices over time, which further accounts for the dynamic shifts between sequential processes in the economy.

Chris also made a few other remarks that I thought were worthy of comment, so here are some brief(ish) observations on other parts of his post:

Of course we have only have had around 5/6 business cycles since 1972 that to my mind can’t produce any statistically significant results either.

Okay, and how many monetary policy interventions have we had in that time? Again, I would think that this reflects rather poorly on a theory that places central bank interventions at the (inevitable) heart of all swings in the business cycle.

ABCT does not claim to be a theory that can explain all observed economic phenomena,which is what Grant thinks it claims to do.

Strawman. I have been very clear -- directly following the paper by L&W -- that this was entirely a question of how relevant ABCT is for explaining observed business cycles in the macroeconomy. Nothing more, nothing less. (Although, one wonders about the usefulness of a theory on business cycles if it seemingly fails to achieve that primary goal.)

On the subject of cycles, here is a beautiful example of circularity:

Let me emphasize that the relevance of the Austrian theory can only increase the more one engages and learns about[...] Austrian theory.

I love this sentence and have re-worked the title of my post in its honour.

On theory versus data:

So to Grant’s point, it is more than just a tendency of Austrians to dismiss empirical ‘evidence’ that runs counter to ABCT and related concepts, because their theories are not built on empirical data but on rigorous logical deduction.

Firstly, I challenge anyone to show me that ABCT follows solely and directly from the action axiom alone. The list of subsidiary axioms and assumptions becomes enormous once we reach the full scope of the theory. This idea of an immaculately conceived business cycle theory, of pure logical cogency and free of any auxiliary pillars is, to be frank, so fanciful that not even the most zealous praxeologist could believe it. More importantly, the "choice" between theory and empirics is a false dichotomy. The above paragraph betrays a misunderstanding of how theory in mainstream economics (or elsewhere) is developed and exactly why it is mutually reinforcing to empirical observation. All economic theory is essentially deductive in nature. You start with some primary axioms or propositions and work through to the implications and consequences. Yet, how do we arbitrate between competing theories or measure their importance? Well, the same way that we do for any scientific field; we test them using data from the real world. Rejection of empirical scrutiny, validation and testing means that we are no longer debating economics or any kind of science for that matter. We are now in the realm of religion.

Chris ends his post in decidedly Churchillian mode:

But Grant should know, in our professions as economists and in the practice of economic forecasting, we are continuously, nay, every week, refining and enhancing our forecasting methods and theories based on what’s available and recent experience. Economic theory and economic forecasting are, of course, very different things.

Typing up that final paragraph must have been difficult whilst holding a bowler hat over his breast and staring defiantly into the distance. Just kidding, bud. I agree with the sentiments here. I ask only that theory shape our forecasting efforts and that we avail ourselves of the opportunity to reconsider these theories when the facts do not match the predictions.

___
[*] As a technical point, there is also some confusion about data classification in the above paragraph. The PPI stage-of-process data is classified by commodities, not firms. Chris' dad's business -- hi Len! -- could therefore have goods classified in various stage-of-process categories, depending on where and who the end consumer was.

[**] This is analogous to an argument made by Tyler Cowen on the co-movement of investment and consumption over the business cycle. See pp. 8-9 of Daniel Kuehn's paper on the Hayekian version of ABCT, which I also mentioned in my previous post.

Empirical evidence and the relevance of ABCT

A new study by Lester and Wolff (2013), hereafter L&W,  is set to cause a bit of a stir in Austrian circles. [HT: Daniel Kuehn]

The paper, which was published in the Review of Austrian Economics no less, finds that Austrian Business Cycle Theory (ABCT ) is not particularly relevant from an empirical standpoint. In short: The unique predictions made by ABCT, concerning the relative price and output changes of goods in different stages of production, are not borne out by the data. It is therefore very difficult to argue that such dynamics are driving the business cycle of the macroeconomy.

I've read through the paper and think that it is a very thorough and technically sound piece of analysis. More importantly, it fills a gap in the literature by using good data to ask the right questions. The conclusion closely matches my own view on ABCT, which is that it constitutes an internally consistent framework for the most part, yet has limited relevance as an overarching macro theory. (That said, L&W also acknowledge that Austrians emphasise a number of concepts, from the coordinating role of market prices and the inter-temporal allocation of resources, that are very valuable to broader economics. Mainstream macro is certainly richer for incorporating these insights.)

Arguing with Austrian-types is something of a side hobby for yours truly and I sent a copy of the paper to Chris Becker, my friend since school days and staunch proponent of all things ABCT. He has written a thoughtful blog post on what he sees are the "flaws and shortcomings" of the study. However, I am not persuaded by his arguments.

Chris starts out by calling into question the various data and metrics used by L&W. For instance, he says that PPI is "only a proxy" for actual economic activity and market prices as "no statistical measure is 100% accurate". Wait a minute, that is simply a tautology. Statistics are by definition imperfect representations of the true state of nature based on probabilistic laws and frequency distributions. To claim that this invalidates their use in scientific research is to a) betray a misunderstanding of how statistics actually works and b) discard a great majority of scientific discoveries and technological advancements since before even the Enlightenment. All that really matters in this case is that these indexes constitute accurate representations of the underlying variables and populations that they refer to. I see no reason to think that they are biased in a manner that systematically renders them uninformative (or misleading) -- particularly if the proposed dynamics were truly the main drivers of large swings in economic activity. I should also say that Chris' objections here would strike me as more convincing if I didn't see Austrians constantly referring to PPI, money supply data, etc. in support of their own arguments.

Next, Chris walks through the various monetary policy variables used in the study and what he perceives as their shortcomings. For the record, L&W use the Federal Funds Rate (FRR) as their main monetary policy variable, while a number of other metrics (M0, M1, M2, etc ) are utilized for robustness checks. In each case, these various monetary policy variables return the same broad set of results that ultimately fail to find vindication for ABCT. (That's the point of running robustness checks after all; they should produce results that are consistent with each other.) Chris does seem to agree with L&W in regarding the FFR as the most appropriate variable to proxy for changes in monetary policy. He even writes: "It is instructive that distortions of the FFR provide the most significant response in favour of ABCT, as it is the divergence between this interest rate and the natural rate of interest that sets in motion the business cycle, according to the Mises-Hayek theory." Except it isn't really instructive at all, because even if some of the coefficients have the same sign as predicted by the theory, they are almost uniformly insignificant from an economic and statistical perspective! L&W are very clear about this and make the point several times throughout their paper. For example (and with emphasis added):

It is critical to note that the results lack statistical significance. In each IRF [Impulse Response Function], the 80 % confidence interval bands suggest that none of the four IRFs demonstrate impact or dynamic responses which differ significantly from zero for more than a few months. This point is particularly relevant when ABCT would otherwise rely on the large shifts in capital to drive business cycle dynamics. 

Once again, we are trying to discern whether ABCT is a plausible candidate for explaining the business cycle at large. According to this evidence, that doesn't appear to be the case.

Chris continues his discussion on monetary policy variables by describing ways in which they may or may not be directly relevant to ABCT, and how the theory can ostensibly accommodate findings that run counter to predictions made by the standard ABCT model. I won't go into too deeply into these issues except to say that I think he runs dangerously close to describing ABCT in pseudoscientific terms. As Popper correctly pointed out many years ago, a theory which claims its strength is to account for any possible outcome is no real scientific theory at all. On the flipside, to say that there are other factors that mitigate how the dynamics of ABCT play out in the economy, is tantamount to admitting that it has limited relevance for explaining observed economic phenomena! (Further, given that the dataset runs from 1972 to 2011 and the empirical analysis tracks variables over a 60-month period following a policy shock, I personally don't think that appealing to "credit injection points" and "historical contingencies" holds much water.)

The post ends with a helpful (ahem) reading list. I have taken the liberty of noting down the respective page numbers for each of the books that Chris recommends: "To really understand ABCT, one should read Ludwig von Mises’ “Human Action” [924 pages] , Friedrich von Hayek’s “Prices and Production” [594 pages], Murray Rothbard’s “Man, Economy, and State” [1,441 pages], and Jesus Huerta de Soto’s “Money, Bank Credit, and Economic Cycles.” [777 pages]". Now, I know that people like to make fun of some Austrians for inevitably referring them to incredibly lengthy treatises during internet debates (often in lieu of making actual arguments). I don't usually think of my friend as falling into that category, but come on... 3,736 pages! If that's what it takes to truly understand ABCT, then I sincerely doubt that anyone has a coherent grip on it.

Allow me to conclude by making two general observations:

1) I may be wrong, but I can't quite escape the feeling that econometrics is seen by many as the preserve of academics and government. That couldn't be further from the truth. Econometrics and statistical analysis has been fundamental to virtually every private company and industry that I have ever worked in, with or am aware of... from the energy sector to finance to consulting to media. If empirical methods were truly misleading, then surely the evolutionary dynamics of the market would have brought about their demise long ago?

2) As with any scientific field or theory, no single study -- no matter how well done -- is enough to invalidate an entire research programme. Similarly, I am hardly claiming that econometrics and empirical studies are infallible. (In addition to discussing the vexing problems of identification many times on this blog, I have also argued that theory and data are mutually reinforcing so that one acts as a check on the other.) However, I do wonder what evidence would be sufficient for Austrians to reconsider their theories. I detect a remarkable tendency to dismiss any empirical evidence that runs counter to ABCT and its related concepts. It should be said that all major schools of economic thought have had to face up to the challenges presented by the data... And are better for it. Keynesians made significant adjustments to their theories in the face of 1970's stagflation, as well as the intellectual challenges of the Lucas Critique and microfoundations movement. For their part, recent events have forced Monetarists to confront the limitations of Friedman's quantity of money supply rule and the potential ineffectiveness of monetary policy at the zero lower bound. Theory cannot advance if it is impervious to data.

___

PS - An ungated version of the L&W paper can be downloaded here.

PPS - Those interested in this subject should also read Daniel's excellent overview of Hayek's version of ABCT, which I believe is forthcoming in Critical Review.

Is gold highly correlated with money supply?

Amidst all this talk about US Republicans eyeing a return to the gold standard, something on my twitter feed earlier this week caught the eye: A link to an old Zero Hedge post together with a claim that "there is a 93% correlation between M2 [money supply] and gold". A similar post here is more specific in saying "[t]he correlation between the total U.S. M2 and gold has exceeded 0.90 since November 2004".

Now presumably, this tweet was aimed at countering the inconvenient fact that the correlation between price inflation (i.e. CPI) and gold is virtually zero. And, if predictions of imminent hyperinflation have yet to materialise, well then at least "hard money" types can point to way in which monetary inflation has manifesting itself in the surging gold price of the last decade. (There's a lesson to be learned here about the velocity of money, kids, but that will have to wait until another time...)

Anyway here's a graph of the gold price and U.S. M2 since 2004, taken from the FRED website. Both are shown in terms of moving monthly averages and, sure enough, the correlation looks very high indeed.



Unfortunately, there are two things wrong with this picture. The first is that the time-frame really does matter. The second has to do with the statistical properties of these series. Let's take these two issues in turn.

Consider what happens when we look at the period from 1981 (which is first date for which FRED has data on both series) until 2004.



Woah! That positive relationship isn't looking too good all of a sudden. Now, of course, I can already hear angry golden-tinged voices accusing me of dueling a strawman. The correlation coefficient was specifically cited for the 2004-post period, so who really cares about what happened 20 or 30 years ago? Okay, perhaps something special happened around the mid-2000s that explains why the two variables have since become so intertwined. Fine, but then don't try to tell me that it's anything specifically to do with money supply. The noticeable kink in the M2 series leading up to that moment occurs around 1995 (after a period of mild tapering), which is close on a decade before the supposed special relationship with gold prices begins.

The broader point here is that if you are going posit a structural theory for why two variables are related, then that relationship needs to have enduring qualities. If not, how can you be sure that gold and M2, rather than one causing the other, aren't both being driven by some other factor? (For one thing, the money supply is supposed to be endogenous to what is happening in the broader economy...) I'm inclined to argue that focusing on the period since 2004 is just a form of data-mining and, as we'll see next, not a particularly good example of that anyway.

Okay. So, ignore the fact that the (weak) longer-term correlation between M2 and gold prices matters. Surely, eight years of data showing a 90%+ correlation can't be denied? Surely, we can say with extreme confidence that recent gold prices have been greatly influenced by money supply? Right?

RIGHT??

Sadly, no. Whenever someone points excitedly to very high correlations between trending time-series, your spidey-sense should be going off like Peter Parker on methylamphetamine. The reason lies with one of the fundamental concepts in time-series econometrics: Nonstationarity.

Wait a minute. Are those series... nonstationary?

Without getting too bogged down by statistical concepts, nonstationary series are characterized by a mean and variance that are changing over time. It's not that they can't be growing or declining over time, but rather that they should consistently return to some kind of mean trend. The most important thing from our perspective is failing to account for this issue will generally lead to spurious (i.e. "nonsense") regression results; an idea that goes all the way back to a classic paper by Yule  in 1926.

Aaaaaaaand.... as you might have guessed by now, the above series are nonstationary. In technical parlance, they are referred to as random walks with drift. Now, there is a famous exception to this rule that occurs when two series are said to be "cointegrated". Again, I'd rather avoid delving too deeply into the murky waters of statistics in a blog post, but suffice to say that that cointegration does not hold here.

To avoid the problems of bullshit spurious regressions, we must therefore adopt a tried and tested approach: Take the first differences of the series and only then test for correlation. Doing so produces a correlation coefficient of exactly <drum roll>... 0.32. Moreover, if we actually regress gold on M2 we get a pretty unimpressive R-squared statistic of 0.1. In other words, only ten percent of the gold's movements are explained by what is happening to money supply.  [UPDATE: Based on the comments, let me again emphasise that these numbers are specifically for the "highly" correlated post-2004 period.]

THOUGHT FOR THE DAY: Simply regressing gold prices on any nonstationary series -- whether that be iPhone sales or the number of Crocs™ wearers in Bangladesh -- would likely produce equally impressive, but obviously bogus, results. Now gold fans might be inclined to protest loudly at this point: "Duh", but there's no theoretical basis for linking those goods to gold. We have a theory that predicts the price of gold will rise with (monetary) inflation!" Except that we've just tested that theory and found it, if not entirely wanting, then at least highly oversold. Perhaps a bit like gold then... [See comments.]

PS - For anyone interested in checking all of this for themselves, here is some Stata code that I used to test the series. The code will always call the most recently available FRED data, so the exact figures you get may differ from those presented here depending on when you run it.

NOTE: In writing this post, I see that others have effectively made the same point before.

Another post on empirics (and a priorism)

I was away last week in Amsterdam taking part in The Econometric Game, a competition involving 30 universities from around the world. Despite the... hmmm... dour reputation of econometric get-togethers, this ended being a very enjoyable and social event. I can certainly recommend Amsterdam to any would-be travellers too. It's relatively small, but built around tourism and there's a great deal more to the city than stoner coffeehouses and sex shops.[*]

Unfortunately, my team didn't make the finals and I think that some inexperience caught us out here, since our university decided to send an all new side this year after reaching the finals in two of the last three events.[**] The time pressure of putting together a complete academic paper in one day -- using some technical routines that you aren't necessarily familiar with -- is something that's hard to prepare for. I do think that our analysis was pretty respectable and we certainly ticked all the boxes highlighted by the case makers after the submission. However, we probably let ourselves down by not "selling" our results well enough in the conclusion and discussion parts of the paper. Nonetheless, definitely a good experience overall and congratulations to the top three teams: 1) University of Copenhagen, 2) Aarhus University, and 3) Harvard University. Danes ruling the roost!

The case topic itself was to investigate "the effect that maternal smoking during pregnancy has on birthweight". Of course, maternal smoking is associated with a range of afflictions in addition to low birthweight and premature birth; from clefts to intrauterine hypoxia. However, the long-term economic implications of low birthweight (and, thus, the causal impact of smoking) are far more important than many people realise. All other things being equal, low birthweight babies will on average suffer higher mortality rates, be more likely to have cognition and attention problems, and be more prone to unemployment and lower wage earnings in later life. (For example, see Black, Devereux and Salvanes; 2005.) 

Some of you may remember that I have actually discussed the issue of maternal smoking on this blog before. Of course, that post had very little to do with empirics and was instead aimed at exploring the philosophical ramifications of allowing pregnant mothers to smoke. In essence, it was a thought-experiment on whether it would be morally permissible to ban mothers from smoking (if this were somehow enforceable). The ensuing comments thread became quite excitable, so take a look if you want to see some divided opinions.

___

[*] Mind you, these are freely on display as well. Like any good boy from Cape Town, I am strongly in favour of dope legalization. Holland's drug policy is more complex than simple soundbites, but far superior to what one finds elsewhere.

[**] The coffeeshops had nothing to with it ;)

...

Having returned from Amsterdam, I saw that my mate Russell had left a comment under an older post concerning the paleo diet's strong popularity within libertarian circles. As a follower of the praxeological method, Russ suggests that I am "asserting a false choice" by claiming that that there is a marked inconsistency in the way that (some) libertarians invoke the scientific method and empirical evidence in finding support for their preferred worldview. You can see my reply here, which concludes: My broader point is that the praxeological fixation among its proponents has created a hermetic seal; a complete aversion to empirical methods that far surpasses the limits of what praxeology could (conceivably) claim to hold sovereignty over.

The language is perhaps a bit dramatic, but still accurate I think. A more specific point that I wanted to make is that if you are going to claim that econometrics and other empirical methods in economics hold no validity because of reasons X, Y and Z... Then it it behoves you to be equally dismissive of their applications to medical studies of the type that Gary Taubes (go-to-guy for the paleo crowd) advocates. For these too seek to identify causal effects in a world characterised by complex interactions between people and their changing environments.