Wifi rots your brain!

The latest mobile/wifi scare story:

"Radiation from mobile phones damages sleep and causes headaches, according to a study by telephone makers."

is covered by gimpy:

"This is interesting as it implies the authors assessed a wide range of variables and possibly rushed the strongest results out in this paper. I can’t help but wonder if they fell victim to selection bias and discarded a whole series of measurements showing no difference. So this paper has enough weaknesses and oversights to throw doubt on its conclusions."

Not much to add to gimpy's analysis, if you read the paper it is clear that they measured a cornucopia of things (self-reported symptoms, cognitive symptoms, stress hormones, cognitive and memory tests, and subsequent sleep and EEG), yet they only report data for latency to deep sleep and amount of stage 4 sleep, so there were clearly a wealth of other comparisons performed, for example, latency to stage 4 sleep and amount of deep sleep, but presumably many many others which are not mentioned (and are therefore likely to all be negative). I don't suppose I need to even mention the problems of multiple comparisons. They also report a random effects logistic regression for headache (finding an increased incidence with exposure) and that 'electrosensitive' types were no better at judging exposure than normal people.

I don't want to be too hard on them, there is always a trade-off between doing lots of experiments to find things that might be affected by your experiment in order to generate hypotheses, and limiting the number of comparisons in order to maximise statistical power and falsify those hypotheses. It also sounds like this paper was derived from a conference presentation and is thus likely incomplete, preliminary, and ever so slightly dodgy (I hate to think how many 'groundbreaking' conference abstracts I've seen that have subsequently failed to be published as full peer reviewed papers, or have been published showing the opposite result).

UPDATE
The NHS Knowledge Service's "Behind the headlines" covers this story:

"This experiment has several important limitations and does not provide sufficient evidence to suggest that mobile use at night is a risk to health. The study only had 71 participants and 38 of them reported suffering from problems that they attributed to mobile use before they entered the study. The small group size and high proportion of people who reported sensitivity to mobile use are unlikely to be representative of the population."

Their emphasis differs from mine, in particular I disagree that the sample size was very small, it isn't bad for lab based studies of this kind. They don't mention the issue of multiple comparisons or cherry picking data.

I don't think their objection that this was an artificial situation is particularly relevant - sure the exposure was quite high, and sure they were trying to get to sleep in a lab, but the whole point of the experiment was to see if there is any tangible biological effect from microwave exposure. I'm also not sure that I agree that the use of allegedly radiosensitive individuals, who are thus unrepresentative, is a problem - because the experiment demonstrates that they actually can't detect when the signal is on, and because current evidence suggests that they aren't actually radiosensitive (i.e. it is a physical response to the belief that there is a microwave signal) - if it turned out that they were in fact radiosensitive then this again would demonstrate what the study sought to discover, that mobile phone signals have a detectable physiological effect.