Skip to main content

Bad science writer, bad! A major mea culpa

The authors found that increased neck pain is 25% more likely with SMT than if you did nothing or stuck to safe and neutral treatments. No. Wrong. Bzzz! Thank you for playing. This morning I received a note from Lisa Carlesso, PT, MSc, first author of the paper, letting me know that I got it wrong: although her data showed that 25% number, it was not a statistically significant number. And that’s significant. If there were any noteworthy increases in neck pain following this kind of neck treatment, presumably clearer data would have emerged. Thus the paper concluded that there is “strong evidence that neck manipulation or mobilization does not result in an increase in neck pain.” I’m not sure if I quite agree that a statistically insignificant number constitutes “strong evidence” so much as just generally low confidence in the results (and Carlesso acknowledges this in the paper as well, practically in the next sentence: I basically saw in the data what I wanted to see. Funny how that works. “However, the limitations of the Strunk study and the low GRADE rating remain, affecting confidence in the estimate.”) But I am guilty of doing something I’ve accused others of doing: emphasizing a statistically insignificant number to make my point. When the numbers lean your way but fail to reach statistical significance, it’s called a “trend.” A trend in favour of a therapy is often trotted out as if it were supportive evidence. I did the opposite, and so I am doing the walk of shame now. Bad science writer, bad! I erroneously thought the number was statistically significant, probably because I’m a debunker by nature, and I basically saw in the data what I wanted to see. Funny how that works. (Gosh, I wonder what system of knowledge-seeking could possibly compensate for that aspect of human nature? You get a gold star if you guessed “science.”) What does it mean? Not much, really. “Here be statistical dragons.” There are so many ways that all this stuff about treatment harms can be wrong that I can’t really walk away from this feeling like I’ve learned anything terribly important one way or the other. The hard statistical bottom line is that statistically insignificant means that no conclusion can actually be drawn — the data was no more than suggestive. Interestingly, Lisa Carlesso pointed out that she has written another paper about how difficult it is to study adverse effects.

Popular posts from this blog

BMI categories

Being overweight may not be as unhealthy as it was 40 years ago," BBC News reports. New research has found a body mass index (BMI) of 27 is linked to the lowest rate of death – but someone with a BMI of 27 is currently classed as being overweight. BMI is a score calculated by dividing your weight (usually in kilograms) by the square of your height (usually in metres and centimetres). Currently, a BMI of 25 to 29.9 is classified as being overweight. Researchers looked at 120,528 people from Copenhagen, recruited from 1976 to 2013, and separately compared those recruited during the 1970s, 1990s and 2000s. They were followed up until they died, emigrated, or the study finished. The BMI linked to the lowest risk of having died from any cause was 23.7 in the 1970s group, 24.6 in the 1990s group, and had further risen to 27 in the 2003-13 group. It may be the case that the suggested upward shift in optimal BMI is the result of improvements in preventative treatments for weight-rela...
Read more

The Anatomy of Vitality

You are a biological entity. A living thing. An animal. An organism. You are a great galaxy of cells, the smallest units of life, a community of ten trillion. They all breathe together. You are ten trillion life forms, the sum of which is entirely different from the parts. Do the stars added together have a mind? Is the cosmos awake? Your community is alive with communication. Your cells talk to each other, intricately, intimately. You are filled with the whispering of microscopic lives. What language do I speak to myself? The language of nerves and glands, at least, but probably even more subtle dialects exist. We do not know the secrets of tumours yet, nor the detailed gossip of undifferentiated cells. No one on planet earth can say with the slightest certainty how consciousness arises from the ruckus of ionic currents in our skulls, or even if it does; the brain may be but an instrument, a lens through which we study ourselves. You would likely die without the quiet energies tha...
Read more

Meditation

There is now hard evidence that meditation can cut stress, newspapers reported October 10 2007. The Daily Mail said that “five short sessions of meditation could be enough to help us achieve piece of mind”. The Daily Telegraph reported that “after meditation training of 20 minutes once a day for only five days, people, had measurably less anxiety and lower levels of the stress hormone cortisol”. The papers said that levels of anxiety, depression, anger, and fatigue had also gone down. The stories are based on a study comparing meditative practice (using integrative body-mind training) with relaxation training, in 80 Chinese students. The newspapers have reported accurately the positive outcomes of the research. The study is a small but well-conducted trial. Whether the findings can be generalised to the practice of individual meditation (as opposed to guided, group practice as is used here) and across cultures remains to be seen.
Read more