Skip to main content

Bad science writer, bad! A major mea culpa

The authors found that increased neck pain is 25% more likely with SMT than if you did nothing or stuck to safe and neutral treatments. No. Wrong. Bzzz! Thank you for playing. This morning I received a note from Lisa Carlesso, PT, MSc, first author of the paper, letting me know that I got it wrong: although her data showed that 25% number, it was not a statistically significant number. And that’s significant. If there were any noteworthy increases in neck pain following this kind of neck treatment, presumably clearer data would have emerged. Thus the paper concluded that there is “strong evidence that neck manipulation or mobilization does not result in an increase in neck pain.” I’m not sure if I quite agree that a statistically insignificant number constitutes “strong evidence” so much as just generally low confidence in the results (and Carlesso acknowledges this in the paper as well, practically in the next sentence: I basically saw in the data what I wanted to see. Funny how that works. “However, the limitations of the Strunk study and the low GRADE rating remain, affecting confidence in the estimate.”) But I am guilty of doing something I’ve accused others of doing: emphasizing a statistically insignificant number to make my point. When the numbers lean your way but fail to reach statistical significance, it’s called a “trend.” A trend in favour of a therapy is often trotted out as if it were supportive evidence. I did the opposite, and so I am doing the walk of shame now. Bad science writer, bad! I erroneously thought the number was statistically significant, probably because I’m a debunker by nature, and I basically saw in the data what I wanted to see. Funny how that works. (Gosh, I wonder what system of knowledge-seeking could possibly compensate for that aspect of human nature? You get a gold star if you guessed “science.”) What does it mean? Not much, really. “Here be statistical dragons.” There are so many ways that all this stuff about treatment harms can be wrong that I can’t really walk away from this feeling like I’ve learned anything terribly important one way or the other. The hard statistical bottom line is that statistically insignificant means that no conclusion can actually be drawn — the data was no more than suggestive. Interestingly, Lisa Carlesso pointed out that she has written another paper about how difficult it is to study adverse effects.

Popular posts from this blog

Being overweight

"Four million people died in 2015 as a result of being too tubby, struck by cancer, heart disease, diabetes and other killer conditions," reports The Sun. This is based on a global study that looked at how the proportion of people who are overweight and obese has changed over time. This was determined by recording body mass index (BMI), where a BMI of 25-29.9 means being overweight and 30 or above is being obese. Researchers then assessed the link between having an unhealthy BMI and health outcomes including cardiovascular disease, diabetes and cancer. It found that, despite public health efforts, obesity is on the rise in almost every country and in both adults and children. Prevalence has doubled in most countries over the past 30 years. Researchers also estimated that having a high BMI accounted for 4 million deaths globally, 40% of which occurred in people who were overweight but not yet obese. This demonstrates that being overweight may almost be as risky to health a...
Read more

What is dementia?

Dementia is a syndrome (a group of related symptoms) associated with an ongoing decline of brain functioning. This may include problems with: memory loss thinking speed mental sharpness and quickness language understanding judgement mood movement difficulties carrying out daily activities There are many different causes of dementia. People often get confused about the difference between Alzheimer's disease and dementia. Alzheimer's disease is a type of dementia and, together with vascular dementia, makes up the vast majority of cases. People with dementia can become apathetic or uninterested in their usual activities, or may have problems controlling their emotions. They may also find social situations challenging and lose interest in socialising. Aspects of their personality may change. A person with dementia may lose empathy (understanding and compassion), they may see or hear things that other people do not (hallucinations). Because people with dementia may lose t...
Read more

Exercise

Two sets of US guidelines have revised the public health advice on physical activity to clarify that gentle exercise is not enough to improve health, The Guardian reported. Current guidelines suggest that 30 minutes of exercise a day is enough to offer health benefits, however the newspaper said that adults “need to add jogging and twice-weekly weight-training sessions if they want to cut their risk of heart disease and obesity.” The newspaper reports that “the lightest of activities such as dusting and the stroll to the car are being counted as exercise” and quotes the researchers as saying that people have “not accepted, and others have misinterpreted the original recommendation.” The expert panel process and the recommendations offered have clarified some of the more unclear recommendations that were published in 1995. The group of experts met in 2000 to prepare this update, and this was supplemented by panel members’ own searches of the literature. It is not possible to validate ...
Read more