Skip to main content

Bad science writer, bad! A major mea culpa

The authors found that increased neck pain is 25% more likely with SMT than if you did nothing or stuck to safe and neutral treatments. No. Wrong. Bzzz! Thank you for playing. This morning I received a note from Lisa Carlesso, PT, MSc, first author of the paper, letting me know that I got it wrong: although her data showed that 25% number, it was not a statistically significant number. And that’s significant. If there were any noteworthy increases in neck pain following this kind of neck treatment, presumably clearer data would have emerged. Thus the paper concluded that there is “strong evidence that neck manipulation or mobilization does not result in an increase in neck pain.” I’m not sure if I quite agree that a statistically insignificant number constitutes “strong evidence” so much as just generally low confidence in the results (and Carlesso acknowledges this in the paper as well, practically in the next sentence: I basically saw in the data what I wanted to see. Funny how that works. “However, the limitations of the Strunk study and the low GRADE rating remain, affecting confidence in the estimate.”) But I am guilty of doing something I’ve accused others of doing: emphasizing a statistically insignificant number to make my point. When the numbers lean your way but fail to reach statistical significance, it’s called a “trend.” A trend in favour of a therapy is often trotted out as if it were supportive evidence. I did the opposite, and so I am doing the walk of shame now. Bad science writer, bad! I erroneously thought the number was statistically significant, probably because I’m a debunker by nature, and I basically saw in the data what I wanted to see. Funny how that works. (Gosh, I wonder what system of knowledge-seeking could possibly compensate for that aspect of human nature? You get a gold star if you guessed “science.”) What does it mean? Not much, really. “Here be statistical dragons.” There are so many ways that all this stuff about treatment harms can be wrong that I can’t really walk away from this feeling like I’ve learned anything terribly important one way or the other. The hard statistical bottom line is that statistically insignificant means that no conclusion can actually be drawn — the data was no more than suggestive. Interestingly, Lisa Carlesso pointed out that she has written another paper about how difficult it is to study adverse effects.

Popular posts from this blog

Being overweight

"Four million people died in 2015 as a result of being too tubby, struck by cancer, heart disease, diabetes and other killer conditions," reports The Sun. This is based on a global study that looked at how the proportion of people who are overweight and obese has changed over time. This was determined by recording body mass index (BMI), where a BMI of 25-29.9 means being overweight and 30 or above is being obese. Researchers then assessed the link between having an unhealthy BMI and health outcomes including cardiovascular disease, diabetes and cancer. It found that, despite public health efforts, obesity is on the rise in almost every country and in both adults and children. Prevalence has doubled in most countries over the past 30 years. Researchers also estimated that having a high BMI accounted for 4 million deaths globally, 40% of which occurred in people who were overweight but not yet obese. This demonstrates that being overweight may almost be as risky to health a...
Read more

Exercise

Two sets of US guidelines have revised the public health advice on physical activity to clarify that gentle exercise is not enough to improve health, The Guardian reported. Current guidelines suggest that 30 minutes of exercise a day is enough to offer health benefits, however the newspaper said that adults “need to add jogging and twice-weekly weight-training sessions if they want to cut their risk of heart disease and obesity.” The newspaper reports that “the lightest of activities such as dusting and the stroll to the car are being counted as exercise” and quotes the researchers as saying that people have “not accepted, and others have misinterpreted the original recommendation.” The expert panel process and the recommendations offered have clarified some of the more unclear recommendations that were published in 1995. The group of experts met in 2000 to prepare this update, and this was supplemented by panel members’ own searches of the literature. It is not possible to validate ...
Read more

Back to basics

Simple painkillers are better than spinal manipulation or anti-inflammatory drugs to cure a bad back, reported The Daily Telegraph and other newspapers. “Visiting a chiropractor for a bad back may be a waste of time and money,” the newspaper said. The Daily Mail adds, “Patients get better just as quickly if they simply stay active and take a mild painkiller such as paracetamol.” The story is based on a well-conducted study in people who had persistent, acute, mild back pain. The researchers compared stronger painkillers and spinal manipulation with continued conservative treatment (paracetamol and advice about avoiding bed rest and remaining active and so on). The study suggests that neither spinal manipulation nor strong painkillers are more effective than conservative treatment including paracetamol for acute back pain.
Read more