Tuesday, 26 February 2008

Misrepresenting science

Further to the discussion below on the PLoS Medicine paper on antidepressants versus placebo, I've noted what I think is a strange way of reporting the paper that ultimately traces back to the authors themselves and the PLoS summary.

From the authors:

"the increased benefit for extremely depressed patients seems attributable to a
decrease in responsiveness to placebo, rather than an increase in responsiveness
to medication."

And the summary:

"The findings also show that the effect for these patients seems to be due to
decreased responsiveness to placebo, rather than increased responsiveness to

Now there are two ways to interpret this (see Figure 3 repeated here). One way would be to say that if we assume (for simplicity) that drug responses are due to placebo + 'true' drug effect then the 'true' drug effect in severe depression is greater than for milder depression. The decreased placebo effect at more severe depression suggests that there is less spontaneous remission (or less response to non-specific interventions) with more severe depression.

But a lot of people have been saying things like:

"The only exception is in the most severely depressed patients...But that is probably because the placebo stopped working so well, they say, rather than the drugs having worked better."


"The researchers said that the drug was more effective than a placebo in severely depressed patients but that this was because of a decreased placebo effect."


"...with slightly more benefit in severe depression but only because of less
response to placebo"


"People with severe symptoms appeared to gain more clear-cut benefit - but this
might be more down to the fact that they were less likely to respond to the
placebo pill, rather than to respond positively to the drugs. "

Now there is a sense in which this could be true, and I really hope it is the interpretation that the authors intended, and that is consistent with the view that a large placebo effect can mask the response to a drug. So if the drugs cause a constant improvement in symptoms at any level of severity (see the figure) then at lower severity where the placebo effect is greater those people who got better with the drug would have got better anyway so there is no net difference between the groups. At more severe levels of depression the drug works just as well but those people who get better on the drug wouldn't have got better with the placebo, that is the decreasing placebo response unmasks the 'real' drug effect. Technically what we are saying here is that the drug effect is not additive, that the response to the drug is more than simply adding the response to placebo plus adding the 'true' drug effect. But note here that we are saying that the drugs do in fact have an appreciable 'true' drug effect that is simply masked by high placebo response rates.

However, the way the stories are written, and I admit this may just be me over interpreting the subtext, suggests to me that the implication intended is that the falling placebo effect with increasing severity is somehow causing the greater apparent effect of the antidepressants over placebo and this is thus not a 'real' effect. That would be a fundamental misunderstanding.

[Language Log has a nice discussion of the reporting of this paper]


Le Canard Noir said...

The whole thing is very interesting. Simplistic responses, such as 'the drugs don't work', do not look right.

In one sense, this result is pretty much what you might expect for a placebo responsive condition. For mild cases, a placebo may well be sufficient. The placebo + treatment is not additive because there is no where to go once a certain amount of benefit has been achieved. For more severe cases of depression then the placebo effect would expected to be diminished as 'belief systems' become more and more impaired. The effect of the drug is likely to be clearer.

What I think might well be harmful is if the results are taken to mean that me should not give these drugs to milder sorts of depression. In the absence of other therapies then the patient is doubly denied any small effect from the drug and any placebo response.

Interesting stuff.

phil said...

Good post!

Wouldn't it be great if you could test the effect of just the drug. Simply get a friend of the test person to slip the pill into his morning cuppa while he's not looking. You would then have three sets of results: just drug, just placebo and placebo+drug. You could then find out whether the relation is purely additive, whether one masks the other, or whether they somehow wonderfullly combine to together achieve more than they would individually.

Of course such a test would be unethical, and might spoil a good cup of tea ;-)

In the absence of the third test, we have to make do with the comparison between the drug and a placebo. The only relevant questions are "Does the drug help the patient?" and "Does the drug help the patient better than the placebo?". If both answers are yes, then the drug works and the effect is real.

pj said...

phil - none of this bad science crap, I wanna be an evil scientist!

black duck - it is actually quite worrying that based on a single study of patients with mild depression and extrapolating a regression line from more severe depression the message has been sent to the public (and GPs) that antidepressants are useless.