Levelling the spirit - pt 1

I listened to an episode of Radio 4's Analysis programme about the ubiquitous book 'The Spirit Level' today (it has also been covered on More and Less). I resisted reading the book for a long time because I both (a) sympathised with the authors aims and broad policy recommendations and (b) suspected that the actual data and methods used would make me cringe and then rant.

I was right.

So in order to fulfil the 'rant' part of that reaction I'm going to share my thoughts on the evidence behind the Spirit Level in two parts trying not to associate myself with some of the attacks on the book from the right (but note that criticism hasn't come only from the right). I'll start with a discussion of the scatter plots that form the core of the book.

Bivariate scatter plots - playing dot-to-dot with the data

Now I'm just not a fan of 'ecological' analyses at the level of countries - even if you do multivariate analyses trying to control for confounding variables you're still taking a dozen heterogeneous societies and drawing straight lines through the data in a simplistic fashion. But in this book the authors, Richard Wilkinson and Kate Pickett, aren't even doing that, they're just plotting bivariate scattergrams, which doesn't bring the level of the analysis much above the level of a blog post (and I should know, I've written just such posts).

What the authors do is basically print a large number of scatter plots which show a relationship between 'inequality' (largely represented as income inequality) and something bad (e.g. murder rates, overall mortality, subway sandwich bars per capita, etc*) - they promise that the scientific literature shows that any potential confounding factors don't matter.

So just how robust are the correlations in the Spirit Level? Well lets look at one of them - the book makes considerable hay with the relationship between (income) inequality and worse life expectancy so that is what I will focus on (see Figure 1 below). It seems that the worse the income inequality in a country the lower the life expectancy.

Figure 1: Income inequality versus life expectancy (from the Spirit Level)

Well first let's sense check the data** and look at the relationship between income inequality (here represented by the mid-2000s Gini coefficient) and life expectancy at birth (see Figure 2 below). There's a big relationship with a correlation of -.41 (that is, the higher the GINI coefficient, and the more unequal the society, the lower the life expectancy) just as we would expect from the Spirit Level data.***

Figure 2: Income inequality versus life expectancy (OECD figures)

So what is causing this relationship between income inequality and life expectancy? Wilkinson & Pickett would say that it is psychosocial factors (such as chronic stress caused by the status anxiety of an unequal society - more on this in part 2) but we can't really reach the conclusion that it is inequality per se causing the lower life expectancy without considering some alternative explanations - that is, we need to ask what other factors are correlated with both inequality and life expectancy that might actually causally mediate the relationship and show that they don't, in fact, do this. 

Confounding variables - assessing alternative explanations

I've previously talked about the relationship between health expenditure and health outcomes including life expectancy (in these five posts here) and this would represent a good first proxy for those material factors that could underlie this relationship. Wilkinson & Pickett (W&P) say that health expenditure cannot be the causal factor, and produce this little graph to illustrate why (see Figure 3 below).

Figure 3: Health expenditure versus life expectancy (from the Spirit Level)

In this scatterplot they find no relationship between the expenditure on health in these developed countries and life expectancy. Could this really be true? The US could reduce its expenditure from nearly $6000 per person down to the Portugese level of nearly $2000 and have no effect on life expectancy? That is quite a bold claim to be making with really very far reaching potential consequences if it is true.

So let's sense check this data, we'll get our data from the OECD again, looking at total expenditure on healthcare in dollar purchasing power parity equivalents (see Figure 4 below).*4

Figure 4: Health expenditure versus life expectancy (OECD figures)

Strangely I seem to get quite a strong relationship between health expenditure and life expectancy, the correlation is .56 (that is, the more money spent the longer the life expectancy). In addition the correlation between Gini and expenditure is -.30 - so it might be plausible that rather than being mediated via psychosocial stresses maybe more unequal societies just don't spend as much on healthcare which leads to material deficits in the healthcare for that country - is it the case? 

Well there are statistical methods to think about (although maybe not resolve) this question - a partial correlation looks to control for a third variable using regression analysis and then to then look at the relationship is between the two other variables assuming that third variable is the same across the sample. If we look at the relationship between Gini and life expectancy where we partial out the effect of health expenditure there is a correlation of .3 between Gini and life expectancy - that is, if we assume that expenditure is equal between all countries then increasing inequality actually predicts longer life expectancy (but this is not actually statistically significant).*5

Choosing your sample - or picking those cherries?

So why does the graph from the Spirit Level show no relationship between health expenditure when my scatterplot shows a strong relationship? Well we can see from the two charts that in the bottom left hand corner of mine there is a little bunch of countries that I have included and W&P have not - these include Hungary, Poland, the Czech Republic, Slovenia, Turkey and Mexico - I also don't have Singapore in my chart since it isn't in the OECD (while the others are).*6 So why aren't these countries in the Spirit Level? The book says:

"All the data come from the most reputable sources - from the World Bank, the World Health Organization, the United Nations and the Organization for Economic Cooperation and Development (OECD), and others."

Which isn't particularly enlightening. But in their 'response to critics' W&P say:

"In The Spirit Level analysis the authors took countries among the 50 richest in the world with populations of more than 3 million, for which there was comparable income distribution data. They did this because they wanted to look at the countries where life expectancy and other outcomes have ceased to be related to economic growth. Peter Saunders adds in Chile, Argentina, Mexico, Venezuela, Turkey, Trinidad & Tobago, Malaysia, Russia, Estonia, Lithuania, Latvia, Poland, S. Korea, Romania, Slovenia, Hungary, Croatia, Czech Republic, Slovakia. In Figure 1.1 (in The Spirit Level ) it can be seen that all these countries are on the rising part of the curve indicating that for them, unlike the richest countries, economic growth remains beneficial. Saunders' later demonstration that economic growth remains beneficial is entirely a result of including these poorer countries."

That figure they refer to is this one (Figure 5 below):

Figure 5: National income (per person) versus life expectancy (from the Spirit Level)

What they are arguing (and indeed argue in the book) is that for poorer countries there is a relationship between life expectancy and economic development - the richer the country the longer the life expectancy - but this relationship then disappears when you get to a certain threshold of wealth. We can see from their graph that there is certainly a steep portion of the scatterplot (below $10,000) and the region they want to concentrate on is the flatter part (above $25,000) but I think it is pretty misleading to claim that we can see no relationship between wealth and life expectancy once we move beyond the steep portion of the curve - certainly to my eye there is a less steep but still linear relationship between wealth and life expectancy in the region above $10-20,000.*7

Even by W&P's standards if we look at countries by PPP adjusted GDP or by nominal GDP per capita they should have included countries like Hungary, the Czech Republic, Slovakia, Poland or Croatia which all come in the top 50 and have populations over 3 million. I took my data (without trying to decide on countries a priori) by just looking up what OECD data was available, and using that directly including where the numbers were indicated to be estimates (since we're interested in trends not exact numbers here). I'm somewhat concerned as to what exact grounds W&P have used to justify excluding those countries which fall at the bottom left of the scatterplot and thus will contribute most to the correlation between wealth (or health expenditure, or whatever measure of material difference we're using) and life expectancy.

At this point I think we need to talk about range restriction. If there is a correlation between two variables (say, for example, height and weight) then, despite the scatter (since not every heavy person is tall, and not every tall person is heavy, it is a relationship that holds on average) there will be a correlation and you can draw a nice regression line showing the relationship - but what happens if you just look at one part of the range (say only those above average in height)? well the relationship (as measured by the correlation) gets smaller, because the scatter noise now begins to mask the relationship. If you keep restricting the range eventually you'll end up with no relationship at all (say looking only at those in the top 10% of heights) even though we know that there is a relationship when we consider the whole range (have a look here for a visual example).

What this digression means is that you have to be careful about insisting on only looking at rich countries because you may not be showing that there is no relationship between expenditure and health in those countries, you may simply have restricted the range so much the relationship gets lost in the noise. And a good sense check for that is to see what happens when you add in a few countries at the bottom of the range - if there really is not relationship it shouldn't make much difference, and if it does you might want to think about just why you're excluding them - a point that is think is highly relevant if you're thinking about keeping in, say, Portugal (population 11m, nominal GDP per capita $22k) but excluding, say, the Czech Republic (population 10m, nominal GDP per capita $18k). Looking at W&P's argument above you also have to be very careful you don't begin some circular reasoning ("these countries are on the rising part of the curve...economic growth remains beneficial") where the very fact that a country has low health expenditure and low life expectancy, so that if it is included in the analysis there will be a relationship, it therefore must be excluded from the analysis - i.e. if there is a relationship we'll eliminate these countries until there is no longer a relationship! (see part 1.5 for further discussion of just how important the specific selection of countries by W&P is for their claims).

So, I think my take home message from part 1 is that the Spirit Level doesn't deal with potentially confounding factors in a satisfactory way, ignoring or dismissing material differences between high and low equality countries that could be the actual causal mechanism for the relationship between inequality and life expectancy (or other measures).*8 It also contains some fairly arbitrary and suspect looking decisions to exclude various countries that upset the arguments W&P are making.

Nothing new under the sun

It is probably worth noting that these shortcomings are not accidental - Wilkinson has been working in this field, and making these sorts of arguments, for many years and others have been questioning the assumptions he makes for just as long.

Take a look at some figures from literature published in the BMJ in 2000-2001 on this topic of wealth and inequality:

Figure 6: GNP per capita versus life expectancy (from Marmot & Wilkinson 2001)

The authors of this figure (Figure 6 above), one of whom is Richard Wilkinson, report that:

The figure shows the relation between life expectancy and gross national product per capita at purchasing power parities for the 25 richest countries for which the World Health Organization holds 1998 data. There is a slight negative relation between the two (r=−0.107). For the 30 richest countries, the correlation is 0.064. It is only when poorer countries are included that the association with mean income emerges.

 This article is in response to one which produces the figure below:

Figure 7: GDP per capita versus life expectancy (from Lynch et al 2000)

The authors of this graph (Figure 7 above) are criticising Wilkinson's previous claims that there is no relationship between wealth and life expectancy:

Wilkinson's demonstration that absolute income was unrelated (r=0.08) to health among developed countries has been important in staking a claim for this psychosocial theory of health inequalities. Figures...show the association between gross domestic product per person and life expectancy...for the 33 countries where gross domestic product was greater than $10000—the cut-off used by Wilkinson. Our results, however, include data for all the countries above $10000, not a selection of some countries in the Organisation for Economic Cooperation and Development as used by Wilkinson. The correlation between life expectancy and gross domestic product per person in the complete sample is r=0.51 (P=0.003). Thus the association between absolute income and life expectancy among wealthier countries depends on which countries are included.

This is an argument that started a long time ago, the points being made now by the critics of W&P are not new, while I'm sure we're not going to resolve this debate any time soon it would be grossly misleading to pretend that these criticisms are purely politically motivated. Many researchers in this field were unimpressed when Wilkinson first made these arguments and those objections still stand largely unanswered.

I think that's enough for today - in part 2 we'll discuss the evidence for those psychosocial mechanisms W&P postulate to provide the causal connection between inequality and health (and various other 'bad things'). If yiou just can't wait until then 'Levelling the spirit - pt 1.5' should tide you over.

* I may have made up one or more of these examples. 


** I'm going to use OECD data on OECD countries as my sample - the Spirit Level uses a lot of data from them and they largely represent what people would consider developed high income countries, I'm using 2007 which is the most complete set of data I have available. I've included all OECD countries I could get data for directly from the OECD so I don't include Chile because there was no Gini data or Luxembourg and Portugal because there was no health expenditure data, the US and Korea lack infant mortality data but are included in the life expectancy analysis.  

*** All correlations are statistically significant at least to p<.05 unless I specifically note otherwise. 

*4 This means that instead of adjusting expenditure between countries by the exchange rate between their currencies you look at what that expenditure would actually buy within the country concerned (so in a poor country your dollar is likely to go a lot further than in the US, and the purchasing power parity equivalent conversion reflects this). This seems to be the same thing that they do in the Spirit Level (see, I don't just make this stuff up).


*5 Note that if you partial out Gini from the relationship between expenditure and life expectancy you still see a partial correlation of .50 - more expenditure means longer life expectancy.

*6 This is a similar finding to that made in this person's blog post except, for some reason (perhaps selection of source for Gini coefficients) they don't find a significant relationship between Gini and life expectancy (whereas I do) but they do find the reversal of this relationship if you include covariates (in their analysis per capita GDP).

*7 An interesting comparison to W&P's claim that income (or expenditure) has no effect on life expectancy (or other health outcomes) can be found in the Spirit Level itself - in a figure from the book (Figure 8 below) W&P compare the relationship between the mortality rates in individual US counties and median household income in those counties:

Figure 8: Median household income by US county versus standardised mortality rate (from the Spirit Level)

What W&P want to do is compare the steepness of the slope indicating the relationship between income and mortality between those counties from the 25 most equal states and the 25 least equal states. That more equal states have a less steep relationship between income and mortality doesn't concern us here, what is interesting is that there is a relationship between income and mortality at the county level within the US at all - this is contrary to what W&P are arguing at the level of individual nations where they dismiss a relationship between national income (or health expenditure) and life expectancy (or other health outcomes) and you have to wonder why this relationship would hold within a rich country but not between them.

*8 Looking at infant mortality, another measure often used to estimate a country's level of health, we find that there is a correlation between the Gini coefficient and infant mortality of  .77.  Health expenditure correlates with infant mortality with a coefficient of -.64. However, unlike with the relationship between Gini and life expectancy, the partial correlation between Gini and infant mortality, controlling for health expenditure, the coefficient is attenuated to .62 - that is the relationship between income inequality and infant mortality is not likey to be mediated purely through the relationship between health spending and infant mortality - so it is more complicated than just claiming that the relationship between inequality and any health measure is simply mediated by health spending.