Wednesday 13 October 2010

Levelling the spirit - pt 1

I listened to an episode of Radio 4's Analysis programme about the ubiquitous book 'The Spirit Level' today (it has also been covered on More and Less). I resisted reading the book for a long time because I both (a) sympathised with the authors aims and broad policy recommendations and (b) suspected that the actual data and methods used would make me cringe and then rant.

I was right.

So in order to fulfil the 'rant' part of that reaction I'm going to share my thoughts on the evidence behind the Spirit Level in two parts trying not to associate myself with some of the attacks on the book from the right (but note that criticism hasn't come only from the right). I'll start with a discussion of the scatter plots that form the core of the book.

Bivariate scatter plots - playing dot-to-dot with the data

Now I'm just not a fan of 'ecological' analyses at the level of countries - even if you do multivariate analyses trying to control for confounding variables you're still taking a dozen heterogeneous societies and drawing straight lines through the data in a simplistic fashion. But in this book the authors, Richard Wilkinson and Kate Pickett, aren't even doing that, they're just plotting bivariate scattergrams, which doesn't bring the level of the analysis much above the level of a blog post (and I should know, I've written just such posts).

What the authors do is basically print a large number of scatter plots which show a relationship between 'inequality' (largely represented as income inequality) and something bad (e.g. murder rates, overall mortality, subway sandwich bars per capita, etc*) - they promise that the scientific literature shows that any potential confounding factors don't matter.

So just how robust are the correlations in the Spirit Level? Well lets look at one of them - the book makes considerable hay with the relationship between (income) inequality and worse life expectancy so that is what I will focus on (see Figure 1 below). It seems that the worse the income inequality in a country the lower the life expectancy.

Figure 1: Income inequality versus life expectancy (from the Spirit Level)
Well first let's sense check the data** and look at the relationship between income inequality (here represented by the mid-2000s Gini coefficient) and life expectancy at birth (see Figure 2 below). There's a big relationship with a correlation of -.41 (that is, the higher the GINI coefficient, and the more unequal the society, the lower the life expectancy) just as we would expect from the Spirit Level data.***

Figure 2: Income inequality versus life expectancy (OECD figures)

So what is causing this relationship between income inequality and life expectancy? Wilkinson & Pickett would say that it is psychosocial factors (such as chronic stress caused by the status anxiety of an unequal society - more on this in part 2) but we can't really reach the conclusion that it is inequality per se causing the lower life expectancy without considering some alternative explanations - that is, we need to ask what other factors are correlated with both inequality and life expectancy that might actually causally mediate the relationship and show that they don't, in fact, do this. 

Confounding variables - assessing alternative explanations

I've previously talked about the relationship between health expenditure and health outcomes including life expectancy (in these five posts here) and this would represent a good first proxy for those material factors that could underlie this relationship. Wilkinson & Pickett (W&P) say that health expenditure cannot be the causal factor, and produce this little graph to illustrate why (see Figure 3 below).

Figure 3: Health expenditure versus life expectancy (from the Spirit Level)

In this scatterplot they find no relationship between the expenditure on health in these developed countries and life expectancy. Could this really be true? The US could reduce its expenditure from nearly $6000 per person down to the Portugese level of nearly $2000 and have no effect on life expectancy? That is quite a bold claim to be making with really very far reaching potential consequences if it is true.

So let's sense check this data, we'll get our data from the OECD again, looking at total expenditure on healthcare in dollar purchasing power parity equivalents (see Figure 4 below).*4

Figure 4: Health expenditure versus life expectancy (OECD figures)
Strangely I seem to get quite a strong relationship between health expenditure and life expectancy, the correlation is .56 (that is, the more money spent the longer the life expectancy). In addition the correlation between Gini and expenditure is -.30 - so it might be plausible that rather than being mediated via psychosocial stresses maybe more unequal societies just don't spend as much on healthcare which leads to material deficits in the healthcare for that country - is it the case? 

Well there are statistical methods to think about (although maybe not resolve) this question - a partial correlation looks to control for a third variable using regression analysis and then to then look at the relationship is between the two other variables assuming that third variable is the same across the sample. If we look at the relationship between Gini and life expectancy where we partial out the effect of health expenditure there is a correlation of .3 between Gini and life expectancy - that is, if we assume that expenditure is equal between all countries then increasing inequality actually predicts longer life expectancy (but this is not actually statistically significant).*5

Choosing your sample - or picking those cherries?

So why does the graph from the Spirit Level show no relationship between health expenditure when my scatterplot shows a strong relationship? Well we can see from the two charts that in the bottom left hand corner of mine there is a little bunch of countries that I have included and W&P have not - these include Hungary, Poland, the Czech Republic, Slovenia, Turkey and Mexico - I also don't have Singapore in my chart since it isn't in the OECD (while the others are).*6 So why aren't these countries in the Spirit Level? The book says:
"All the data come from the most reputable sources - from the World Bank, the World Health Organization, the United Nations and the Organization for Economic Cooperation and Development (OECD), and others."
Which isn't particularly enlightening. But in their 'response to critics' W&P say:
"In The Spirit Level analysis the authors took countries among the 50 richest in the world with populations of more than 3 million, for which there was comparable income distribution data. They did this because they wanted to look at the countries where life expectancy and other outcomes have ceased to be related to economic growth. Peter Saunders adds in Chile, Argentina, Mexico, Venezuela, Turkey, Trinidad & Tobago, Malaysia, Russia, Estonia, Lithuania, Latvia, Poland, S. Korea, Romania, Slovenia, Hungary, Croatia, Czech Republic, Slovakia. In Figure 1.1 (in The Spirit Level ) it can be seen that all these countries are on the rising part of the curve indicating that for them, unlike the richest countries, economic growth remains beneficial. Saunders' later demonstration that economic growth remains beneficial is entirely a result of including these poorer countries."
That figure they refer to is this one (Figure 5 below):

Figure 5: National income (per person) versus life expectancy (from the Spirit Level)
What they are arguing (and indeed argue in the book) is that for poorer countries there is a relationship between life expectancy and economic development - the richer the country the longer the life expectancy - but this relationship then disappears when you get to a certain threshold of wealth. We can see from their graph that there is certainly a steep portion of the scatterplot (below $10,000) and the region they want to concentrate on is the flatter part (above $25,000) but I think it is pretty misleading to claim that we can see no relationship between wealth and life expectancy once we move beyond the steep portion of the curve - certainly to my eye there is a less steep but still linear relationship between wealth and life expectancy in the region above $10-20,000.*7

Even by W&P's standards if we look at countries by PPP adjusted GDP or by nominal GDP per capita they should have included countries like Hungary, the Czech Republic, Slovakia, Poland or Croatia which all come in the top 50 and have populations over 3 million. I took my data (without trying to decide on countries a priori) by just looking up what OECD data was available, and using that directly including where the numbers were indicated to be estimates (since we're interested in trends not exact numbers here). I'm somewhat concerned as to what exact grounds W&P have used to justify excluding those countries which fall at the bottom left of the scatterplot and thus will contribute most to the correlation between wealth (or health expenditure, or whatever measure of material difference we're using) and life expectancy.

At this point I think we need to talk about range restriction. If there is a correlation between two variables (say, for example, height and weight) then, despite the scatter (since not every heavy person is tall, and not every tall person is heavy, it is a relationship that holds on average) there will be a correlation and you can draw a nice regression line showing the relationship - but what happens if you just look at one part of the range (say only those above average in height)? well the relationship (as measured by the correlation) gets smaller, because the scatter noise now begins to mask the relationship. If you keep restricting the range eventually you'll end up with no relationship at all (say looking only at those in the top 10% of heights) even though we know that there is a relationship when we consider the whole range (have a look here for a visual example).

What this digression means is that you have to be careful about insisting on only looking at rich countries because you may not be showing that there is no relationship between expenditure and health in those countries, you may simply have restricted the range so much the relationship gets lost in the noise. And a good sense check for that is to see what happens when you add in a few countries at the bottom of the range - if there really is not relationship it shouldn't make much difference, and if it does you might want to think about just why you're excluding them - a point that is think is highly relevant if you're thinking about keeping in, say, Portugal (population 11m, nominal GDP per capita $22k) but excluding, say, the Czech Republic (population 10m, nominal GDP per capita $18k). Looking at W&P's argument above you also have to be very careful you don't begin some circular reasoning ("these countries are on the rising part of the curve...economic growth remains beneficial") where the very fact that a country has low health expenditure and low life expectancy, so that if it is included in the analysis there will be a relationship, it therefore must be excluded from the analysis - i.e. if there is a relationship we'll eliminate these countries until there is no longer a relationship! (see part 1.5 for further discussion of just how important the specific selection of countries by W&P is for their claims).

So, I think my take home message from part 1 is that the Spirit Level doesn't deal with potentially confounding factors in a satisfactory way, ignoring or dismissing material differences between high and low equality countries that could be the actual causal mechanism for the relationship between inequality and life expectancy (or other measures).*8 It also contains some fairly arbitrary and suspect looking decisions to exclude various countries that upset the arguments W&P are making.

Nothing new under the sun

It is probably worth noting that these shortcomings are not accidental - Wilkinson has been working in this field, and making these sorts of arguments, for many years and others have been questioning the assumptions he makes for just as long.

Take a look at some figures from literature published in the BMJ in 2000-2001 on this topic of wealth and inequality:

Figure 6: GNP per capita versus life expectancy (from Marmot & Wilkinson 2001)
The authors of this figure (Figure 6 above), one of whom is Richard Wilkinson, report that:
The figure shows the relation between life expectancy and gross national product per capita at purchasing power parities for the 25 richest countries for which the World Health Organization holds 1998 data. There is a slight negative relation between the two (r=−0.107). For the 30 richest countries, the correlation is 0.064. It is only when poorer countries are included that the association with mean income emerges.
 This article is in response to one which produces the figure below:

Figure 7: GDP per capita versus life expectancy (from Lynch et al 2000)
The authors of this graph (Figure 7 above) are criticising Wilkinson's previous claims that there is no relationship between wealth and life expectancy:
Wilkinson's demonstration that absolute income was unrelated (r=0.08) to health among developed countries has been important in staking a claim for this psychosocial theory of health inequalities. Figures...show the association between gross domestic product per person and life expectancy...for the 33 countries where gross domestic product was greater than $10000—the cut-off used by Wilkinson. Our results, however, include data for all the countries above $10000, not a selection of some countries in the Organisation for Economic Cooperation and Development as used by Wilkinson. The correlation between life expectancy and gross domestic product per person in the complete sample is r=0.51 (P=0.003). Thus the association between absolute income and life expectancy among wealthier countries depends on which countries are included.

This is an argument that started a long time ago, the points being made now by the critics of W&P are not new, while I'm sure we're not going to resolve this debate any time soon it would be grossly misleading to pretend that these criticisms are purely politically motivated. Many researchers in this field were unimpressed when Wilkinson first made these arguments and those objections still stand largely unanswered.

I think that's enough for today - in part 2 we'll discuss the evidence for those psychosocial mechanisms W&P postulate to provide the causal connection between inequality and health (and various other 'bad things'). If yiou just can't wait until then 'Levelling the spirit - pt 1.5' should tide you over.

* I may have made up one or more of these examples. 


** I'm going to use OECD data on OECD countries as my sample - the Spirit Level uses a lot of data from them and they largely represent what people would consider developed high income countries, I'm using 2007 which is the most complete set of data I have available. I've included all OECD countries I could get data for directly from the OECD so I don't include Chile because there was no Gini data or Luxembourg and Portugal because there was no health expenditure data, the US and Korea lack infant mortality data but are included in the life expectancy analysis.  

*** All correlations are statistically significant at least to p<.05 unless I specifically note otherwise. 

*4 This means that instead of adjusting expenditure between countries by the exchange rate between their currencies you look at what that expenditure would actually buy within the country concerned (so in a poor country your dollar is likely to go a lot further than in the US, and the purchasing power parity equivalent conversion reflects this). This seems to be the same thing that they do in the Spirit Level (see, I don't just make this stuff up).


*5 Note that if you partial out Gini from the relationship between expenditure and life expectancy you still see a partial correlation of .50 - more expenditure means longer life expectancy.

*6 This is a similar finding to that made in this person's blog post except, for some reason (perhaps selection of source for Gini coefficients) they don't find a significant relationship between Gini and life expectancy (whereas I do) but they do find the reversal of this relationship if you include covariates (in their analysis per capita GDP).

*7 An interesting comparison to W&P's claim that income (or expenditure) has no effect on life expectancy (or other health outcomes) can be found in the Spirit Level itself - in a figure from the book (Figure 8 below) W&P compare the relationship between the mortality rates in individual US counties and median household income in those counties:

Figure 8: Median household income by US county versus standardised mortality rate (from the Spirit Level)
What W&P want to do is compare the steepness of the slope indicating the relationship between income and mortality between those counties from the 25 most equal states and the 25 least equal states. That more equal states have a less steep relationship between income and mortality doesn't concern us here, what is interesting is that there is a relationship between income and mortality at the county level within the US at all - this is contrary to what W&P are arguing at the level of individual nations where they dismiss a relationship between national income (or health expenditure) and life expectancy (or other health outcomes) and you have to wonder why this relationship would hold within a rich country but not between them.

*8 Looking at infant mortality, another measure often used to estimate a country's level of health, we find that there is a correlation between the Gini coefficient and infant mortality of  .77.  Health expenditure correlates with infant mortality with a coefficient of -.64. However, unlike with the relationship between Gini and life expectancy, the partial correlation between Gini and infant mortality, controlling for health expenditure, the coefficient is attenuated to .62 - that is the relationship between income inequality and infant mortality is not likey to be mediated purely through the relationship between health spending and infant mortality - so it is more complicated than just claiming that the relationship between inequality and any health measure is simply mediated by health spending.

6 comments:

Neuroskeptic said...

Good post. Some random thoughts on TSL which you may or may not find useful for future posts:

* Singapore is an obvious outlier on many of their graphs, because it has very high income inequality, but low crime etc. To be fair, it is a very small country, little more than a city-state, but still, it seems to disprove the idea that inequality is always bad. Interestingly, they do not include Singapore in many of their graphs, and they don't (iirc) explain why, because it is part of their dataset.

* Pretty much all of the "bad" things correlate with each other: crime, life expectancy, etc. Income inequality does too. But the skeptical view would be that it's just one more bad thing that correlates with all the others - not a causal factor. IIRC the evidence for the correlations is pretty strong but the argument for causality is weaker and relies on a story about how inequality causes stress which causes illness etc.

* Suicide rates do not fit the above pattern: Japan and Scandinavia have notoriously high rates while the UK is relatively low. I can't remember what the authors make of this, but in my opinion, it is a serious spanner in the works because a society in which the suicide rate is as high as Japan's, must have something wrong with it albeit maybe something hard to quantify.

pj said...

Cheers. Singapore is a country I would choose to exclude form any analyses actually - this is because it imports quite a lot of its labour (hundreds of thousands of people) commuting from Malaysia, and also many of its foreign workers return to their country of origin (particularly Malaysia) on retirement - which means I'm not sure Singapore's data accurately characterises the benefits and costs of the Singapore model to all those concerned. But that may just be my prejudice.

In terms of including Singapore in figures I think W&P would say that they couldn't get comparable data for whatever measure they are reporting - unfortunately it all seems a bit opaque which ones make it in or out, and I would be concerned that their apparent a priori selection of this or that source was not entirely prejudice free - in that they probably already know which sources or selection methods will give positive or negative results.

I will be dealing with the psychosocial 'stress' explanation in part 2, I agree that it is likely that these bad things all correlate, and there has to be a causal mechanism in there somewhere to explain why they all correlate, I'm just not convinced that the psychological impact of income inequality in necessarily that causal explanation.

Suicide rate is an interesting one - W&P actually make the claim that homicide rate and suicide rate are sort of inversely related such that existential rage is either channeled via homicide or suicide depending on the society - which is quite a bold and maybe counterintuitive claim although I don't think they necessarily establish that explanation. I wonder whether there are good explanations for suicide rate, i know Durkheim thought it was due to religion - certainly I think suicide stats can be a bit dubious and societal tolerance of suicide may skew reporting rates - theres also an association with alcohol abuse so i don't know whether Japan or Scandinavian countries have more risky drinking behaviour. I think I'm somewhat sympathetic to W&P in that suicide does seem to be an odd measure that doesn't fit the classical pattern of inequality and health/social outcomes so it doesn't necessarily disprove their argument, just complicate it a little, and at least they mention it.

Neuroskeptic said...

All good points.

Re: the "they're only looking at a restricted range" argument, I think they may a good case that the "flat" part of the graph is interesting and important in itself. Especially because more and more countries are likely to move into this bit as time goes on. What they're saying is "beyond this point, inequality is more important than GDP" which seems a fair argument. Of course if you included poorer countries you would weaken the results, but that actually supports their argument, that there is an important difference.

pj said...

My concern regarding the 'flat' part of the graph is that when you actually look at the graph it appears to be a steep bit and a less steep bit. The only way to make the less steep bit flat is to arbitrarily decide to only look at the far right handside - but similarly, if I only look at the far left hand side of that graph I don't seem to get any relationship either - does that mean there is no relationship between wealth and life expectancy in poor countries? No.

Interestingly, W&P actually talk about how increased inequality in the former Soviet block countries after the fall of the Berlin Wall caused their decline in life expectancy. Well there's some serious cake having and eating going on there. We can't consider these countries when looking at the relationship between wealth or expenditure because they are somehow too poor to be part of this brave new world of rich countries where ill-health is caused solely by inequality, but they are not too poor to illustrate the ill effects of inequality.

It is worth noting that W&P say that if you use countries on the curvy bit of their wealth graph you have to control for the effect of wealth - well that is exactly what my partial correlation analysis does, indeed it is the whole point of that analysis (except I looked at health expendture rather than GDP per capita) and that analysis shows that controlling in such a way eliminates the effect of inequality on life expectancy. If there is no relationship between expenditure and life expectancy in the rich countries then why does it eliminate the effect of inequality when we control for it? Inclusion of the poorer countries in the analysis shouldn't cause this because the whole point of correcting for expenditure is to prevent their poorness disrupting the analysis. This is why I think arguing that "beyond this point, inequality is more important than GDP" is actually not proven by the graph.

I think the best line for that wealth/life expectancy graph is going to be reaching something of an asymptote or log-graph (as ceiling effects are reached and diminishing returns come from attempts to lengthen life) which is intuitively plausible - but what that does not mean is that higher up the line you go (nearer the asymptote) the relationship between life expectancy and expenditure (or wealth) disappears - no it just becomes flatter, which is a very different claim to that being made by W&P which demands very different action. In my explanation you need to spend more on health, in theirs you can pretty much cut back spending and have no effect - a conclusion that I don't think W&P have quite grasped the repercussions of yet.

pj said...

Neuro: in response to your suggestion that the countries in the 'flat' bit might somehow be special, rather than it simply being range restriction I had another look at my data.

This time I only included those OECD countries that W&P included and looked at the Gini relationship with life expectancy. Well the correlation is not remotely statistically significant (p around .5). Now I don't have data on all the same countries (I've excluded Singapore and Israel because they aren't OECD) so I went to the UN Human Development report 2007 and took some Gini and life expectancy data from there and added them in - still no significant relationship, the correlation is -.11 and p=.61. So it seems that there is definitely a sample size/range restriction issue going on because even the inequality-life expectancy relationship doesn't hold with my data if we cut the number of countries down so much.

An alternative sense check is to say, fine, lets just look at the richer countries, and I'm going to define that non-arbitrarily by saying that in the W&P health expenditure graph the lowest spender is Singapore at just above $1000, so I'll use that round figure as my cut-off. Well we still get a statistically significant relationship between life expectancy and expenditure (r=.42).

So what about selecting on national income? I'll pick countries on my OECD list by national income according to that graph from W&P (the curved one we're talking about), the lowest country W&P include is Portugal so no country below that gets included (I still get to keep Hungary, the Czech Republic and Korea). Well there is still no stat sig relationship between Gini and life expectancy and an almost flat trend line, so no inequality health relationship here. What about health expenditure and life expectancy? Well we get r=.57 (stat sig) for all OECD excluding Portugal (no OECD figures on health expenditure for them - if I estimate from the UN HD report I still get r=.56). The multivariate analysis shows that partialling out health expenditure gives a Gini-life expectancy correlation of r=.46 (stat sig)! Now I'm not really claiming that inequality causes increased life expectancy, but I'd probably be on a firmer statistical footing than W&P if I was!

pj said...

I did a short 'part 1.5' post with some graphs on this topic.