Skip to main content

Bad science writer, bad! A major mea culpa

The authors found that increased neck pain is 25% more likely with SMT than if you did nothing or stuck to safe and neutral treatments. No. Wrong. Bzzz! Thank you for playing. This morning I received a note from Lisa Carlesso, PT, MSc, first author of the paper, letting me know that I got it wrong: although her data showed that 25% number, it was not a statistically significant number. And that’s significant. If there were any noteworthy increases in neck pain following this kind of neck treatment, presumably clearer data would have emerged. Thus the paper concluded that there is “strong evidence that neck manipulation or mobilization does not result in an increase in neck pain.” I’m not sure if I quite agree that a statistically insignificant number constitutes “strong evidence” so much as just generally low confidence in the results (and Carlesso acknowledges this in the paper as well, practically in the next sentence: I basically saw in the data what I wanted to see. Funny how that works. “However, the limitations of the Strunk study and the low GRADE rating remain, affecting confidence in the estimate.”) But I am guilty of doing something I’ve accused others of doing: emphasizing a statistically insignificant number to make my point. When the numbers lean your way but fail to reach statistical significance, it’s called a “trend.” A trend in favour of a therapy is often trotted out as if it were supportive evidence. I did the opposite, and so I am doing the walk of shame now. Bad science writer, bad! I erroneously thought the number was statistically significant, probably because I’m a debunker by nature, and I basically saw in the data what I wanted to see. Funny how that works. (Gosh, I wonder what system of knowledge-seeking could possibly compensate for that aspect of human nature? You get a gold star if you guessed “science.”) What does it mean? Not much, really. “Here be statistical dragons.” There are so many ways that all this stuff about treatment harms can be wrong that I can’t really walk away from this feeling like I’ve learned anything terribly important one way or the other. The hard statistical bottom line is that statistically insignificant means that no conclusion can actually be drawn — the data was no more than suggestive. Interestingly, Lisa Carlesso pointed out that she has written another paper about how difficult it is to study adverse effects.

Popular posts from this blog

PF-ROM Exercises

"Pain-free range of motion’ or early mobilization exercises can help you heal" PF-ROM is physical therapy talk for “pain free range of motion.” When a therapist evaluates an injury, he or she will be interested to see how far you can move affected joints without hurting. Sometimes, of course, you can’t move at all without pain. But in most injuries, even many serious ones, you will have at least some painless movement. And whatever you’ve got, you should use. When you are hurt, the pain-free range is your new best friend: that’s the range you’ll be exercising in for a while. Pain free range of motion exercises are also known as “early mobilization.” Use it or lose it “Use it or lose it,” they say. And it’s true. While many seemingly simple medical questions are controversial, this one appears to be straightforward: plenty of recent research demonstrates that early mobilization is A Very Good Thing. A 2006 study of people with surgically repaired achilles tendon ruptures sh...
Read more

What is dementia?

Dementia is a syndrome (a group of related symptoms) associated with an ongoing decline of brain functioning. This may include problems with: memory loss thinking speed mental sharpness and quickness language understanding judgement mood movement difficulties carrying out daily activities There are many different causes of dementia. People often get confused about the difference between Alzheimer's disease and dementia. Alzheimer's disease is a type of dementia and, together with vascular dementia, makes up the vast majority of cases. People with dementia can become apathetic or uninterested in their usual activities, or may have problems controlling their emotions. They may also find social situations challenging and lose interest in socialising. Aspects of their personality may change. A person with dementia may lose empathy (understanding and compassion), they may see or hear things that other people do not (hallucinations). Because people with dementia may lose t...
Read more

Stress at work

People with high-stress jobs have twice the risk of developing serious depression or anxiety compared with others in less stressful occupations, The Independent reported. The link between rising rates of depression and rates of work stress means that “one in 20 cases of depression or anxiety annually is attributable to high stress at work”, it stated on August 2 2007. Highly stressed jobs include head chefs and construction workers, the paper reports, and least stressful jobs include looking after children at home, “where there are no deadlines to meet, greater flexibility and no fear of public failure”. The researchers explain that “time pressure is the single most important cause of stress and of the illness to which it leads,” the newspaper said. The research supports the intuitive link between stress at work and the mental health of young working adults. Where did the story come from? The research was carried out by Maria Melchior and colleagues at the Institute of Psychiatry, K...
Read more