Group items tagged

Statistical validation of results, as Shaffer described it, simply involves testing the null hypothesis: that the pattern you detect in your data occurs at random. If you can reject the null hypothesis—and science and medicine have settled on rejecting it when there's only a five percent or less chance that it occurred at random—then you accept that your actual finding is significant. The problem now is that we're rapidly expanding our ability to do tests. Various speakers pointed to data sources as diverse as gene expression chips and the Sloan Digital Sky Survey, which provide tens of thousands of individual data points to analyze. At the same time, the growth of computing power has meant that we can ask many questions of these large data sets at once, and each one of these tests increases the prospects than an error will occur in a study; as Shaffer put it, "every decision increases your error prospects." She pointed out that dividing data into subgroups, which can often identify susceptible subpopulations, is also a decision, and increases the chances of a spurious error. Smaller populations are also more prone to random associations. In the end, Young noted, by the time you reach 61 tests, there's a 95 percent chance that you'll get a significant result at random. And, let's face it—researchers want to see a significant result, so there's a strong, unintentional bias towards trying different tests until something pops out. Young went on to describe a study, published in JAMA, that was a multiple testing train wreck: exposures to 275 chemicals were considered, 32 health outcomes were tracked, and 10 demographic variables were used as controls. That was about 8,800 different tests, and as many as 9 million ways of looking at the data once the demographics were considered.

The study authors, led by Rachel Manber, PhD, professor of psychiatry and behavioral sciences, said they hope the results will raise awareness of the problem of depression during pregnancy and provide patients and physicians an alternative to antidepressants. "This standardized acupuncture protocol could be a viable treatment option for depression during pregnancy," they wrote in a paper that will appear in the March issue of Obstetrics & Gynecology. Up to 14 percent of pregnant women may have major depressive disorder, a condition characterized by feelings of dread, gloom and hopelessness, and a loss of interest in normally pleasurable activities. Some women suffer from depression before becoming pregnant, stop taking their medication and then experience a relapse; in other women, pregnancy itself may cause depression.

For this study, the researchers recruited 150 women whose pregnancies were between 12 and 30 weeks gestation and who met the criteria for major depressive disorder. The women were randomized to receive one of three treatments: acupuncture specific for depression; control acupuncture, during which needles were inserted in points not known to help alleviate depressive symptoms; or massage. All of the women received eight weeks of therapy and were assessed for depression at the four- and eight-week marks by an interviewer who was unaware of the treatment each woman received. The researchers found that women who received the depression-specific acupuncture experienced a bigger reduction in depression symptoms than the women in the other groups. The response rate — defined as having a 50 percent or greater reduction in symptoms — was 63 percent for the women receiving depression-specific acupuncture, compared with 44 percent for the women in the other two treatment groups combined. The researchers weren't surprised by what they found — a pilot study yielded similar results, and other studies have shown acupuncture is an effective treatment for depression in the general public — but they were pleased with the results. "I don't think that one-size-fits-all treatments are appropriate for everyone, but acupuncture should be considered as an option," said Lyell. "I hope that people will respect the rigorous methodology used in this blinded, randomized, controlled trial and accept the result: Traditional acupuncture was associated with a significant improvement in depression."