Manufactured controversy and the "hockey stick": A football analogy

Even if you're only vaguely aware of the climate change debate, then you will probably have heard of the "hockey stick". You know, this bad boy:

Source: Mann et al. (1999).

This famous depiction of global temperatures going back into time has generated a lot of controversy. It doesn't seem to matter much to sceptics that the initial hockey stick(s) -- i.e. those produced by Mann, Bradley and Hughes (1998, 1999) -- have since been replicated by multiple studies using different lines of evidence and computational procedures. No, we are invariably told that the hockey stick is a fraud and has been debunked by the likes of Steve McIntyre and Ross McKitrick.

The problem with these debates is that they are necessarily technical and involve concepts that are very unfamiliar to most people. Whenever I tried to explain things to my friends and family, I could see their eyes glazing over as soon as I mentioned the words "principal component analysis". So here is a sports analogy that captures the essence of what critics like McIntyre and McKitrick got wrong.

Imagine that you are football (i.e. soccer) player who has discovered a great way of scoring goals when you are within the opposition's penalty area. In particular, you also happen to be a physics geek, and have calibrated the optimal direction and force with which to strike the ball so that it lands in the back of the net every time. You demonstrate this impressive feat by placing the ball at the right-side corner of the penalty box and repeatedly slot a very high percentage of goals. Well done, FC Barcelona and Bayern Munich scouts are already fighting for your signature!

Hold on. Some rival player now arrives on the scene and says that you are talking complete nonsense. He points out -- not unreasonably -- that most shots at goal during a game are taken from further in front of the goals. In other words, more to the centre of the penalty area and not from the right-hand corner where you were showing off your abilities. He then "proves" how bad your system is by taking the ball to the centre of the box and, using your exact same kicking force and direction, proceeds to drill his shots wide of the goals. Worse, he then moves on to the left-corner of the box, where his shots are so off-target that they are threatening to hit spectators in the stands...

By now, you should of course realise what is wrong with this story. The rival player has completely failed to recalculate his kicks based on the new position of ball! The underlying physics remains completely intact; you would use exactly the same principles in determining what force and direction to use in scoring goals, even if those factors would vary according to where you were shooting from.

You forgot to recalculate, Wayne!

This, in simplified form and among other things, is the same mistake that those early critics of the hockey stick made. Except now the relevant decision variables involve selection rules for the number of principal components (PC) and where your data are centered along the time-series. In short, McIntyre and McKitrick (MM) argued that Mann et al. had used an unusual centering procedure in their work and, when a more standard convention was used, the hockey stick vanished. The problem with this argument is that when you re-center your analysis, you also need to re-determine the optimal number of PC series, which are the things that determine the key patterns in your data. Just like the footballer in our example, you need to adjust your shot depending on where you are kicking from. MM failed to do this and just ran the same number of PC series as Mann et al. had (i.e. two), but now under their preferred centering convention. Correcting for this mistake and including the right number of PC series under the new centering convention (i.e. five) means that -- hey, presto! -- the hockey stick magically reappears.

If you are interested in reading further, here are two excellent posts dealing with this manufactured controversy over the hockey stick. I should also say that Michael Mann's book gives a very accessible overview of whole palaver. He doesn't use a sporting analogy, though, so I still have the upper hand on that score.