Statistical validation of results, as Shaffer described it, simply involves testing the null hypothesis: that the pattern you detect in your data occurs at random. If you can reject the null hypothesis—and science and medicine have settled on rejecting it when there's only a five percent or less chance that it occurred at random—then you accept that your actual finding is significant.
The problem now is that we're rapidly expanding our ability to do tests. Various speakers pointed to data sources as diverse as gene expression chips and the Sloan Digital Sky Survey, which provide tens of thousands of individual data points to analyze. At the same time, the growth of computing power has meant that we can ask many questions of these large data sets at once, and each one of these tests increases the prospects than an error will occur in a study; as Shaffer put it, "every decision increases your error prospects." She pointed out that dividing data into subgroups, which can often identify susceptible subpopulations, is also a decision, and increases the chances of a spurious error. Smaller populations are also more prone to random associations.
In the end, Young noted, by the time you reach 61 tests, there's a 95 percent chance that you'll get a significant result at random. And, let's face it—researchers want to see a significant result, so there's a strong, unintentional bias towards trying different tests until something pops out.
Young went on to describe a study, published in JAMA, that was a multiple testing train wreck: exposures to 275 chemicals were considered, 32 health outcomes were tracked, and 10 demographic variables were used as controls. That was about 8,800 different tests, and as many as 9 million ways of looking at the data once the demographics were considered.
Group items tagged
Cairo to Madison: Hope and Solidarity - 0 views
-
Egyptian engineer Muhammad Saladin Nusair, the one whose photo supporting Wisconsin workers went viral, now has thousands of new American Facebook friends. He wrote in his blog that many of his new friends were surprised by his gesture of solidarity, but he was taught that "we live in ONE world and under the same sky." "If a human being doesn't feel the pain of his fellow human beings, then everything we've created and established since the very beginning of existence is in great danger," Muhammad wrote. "We shouldn't let borders and differences separate us. We were made different to complete each other, to integrate and live together. One world, one pain, one humanity, one hope."
Chris Hedges: The Sky Really Is Falling - Chris Hedges' Columns - Truthdig - 0 views
-
The rapid and terrifying acceleration of global warming, which is disfiguring the ecosystem at a swifter pace than even the gloomiest scientific studies predicted a few years ago, has been confronted by the power elite with two kinds of self-delusion. There are those, many of whom hold elected office, who dismiss the science and empirical evidence as false. There are others who accept the science surrounding global warming but insist that the human species can adapt. Our only salvation-the rapid dismantling of the fossil fuel industry-is ignored by both groups. And we will be led, unless we build popular resistance movements and carry out sustained acts of civil disobedience, toward collective self-annihilation by dimwitted pied pipers and fools.
Robert Reich (Why This is Exactly the Time to Rebuild America's Infrastructure) - 0 views
-
Seems like only yesterday conservative nabobs of negativity predicted America's ballooning budget deficit would generate soaring inflation and crippling costs of additional federal borrowing. Remember Standard & Poor's downgrade of the United States? Recall the intense worry about investors' confidence in government bonds - America's IOUs? Hmmm. Last week ten-year yields on U.S. Treasuries closed at 1.83 percent. In other words, they were wrong. In fact, it's cheaper than ever for the United States to borrow. That's because global investors desperately want the safety of dollars. Almost everywhere else on the globe is riskier. Europe is in a debt crisis, many developing nations are gripped by fears the contagion will spread to them, Japan remains in critical condition, China's growth is slowing. Put this together with two other facts: Unemployment in America remains sky-high. 14 million Americans are out of work and 25 million are looking for full-time jobs. The nation's infrastructure is crumbling. Our roads, bridges, water and sewer systems, subways, gas pipelines, ports, airports, and school buildings are desperately in need of repair. Deferred maintenance is taking a huge toll. Now connect the dots. Anyone with half a brain will see this is the ideal time to borrow money from the rest of the world to put Americans to work rebuilding the nation's infrastructure. Problem is, too many in Washington have less than half a brain.
Hey President Obama ... | Adbusters Culturejammer Headquarters - 0 views
-
If thousands of us hang in there day after day, week after week, we may be able to create a spectacular revolutionary experience that fires up the public imagination and eventually maneuvers Obama into doing something that he has so far not had the guts to do: agree to a bold, decisive stroke against the financial corruption of America. Now that would get the American people behind us and cheering us on from coast to coast. If we can achieve that, the sky will be the limit … further demands will follow and a new America will be born.
InfiniteTolerance.com» Stephen Murphy Presents Infinite Tolerance » Infinite ... - 0 views
-
"All the leaves are brown and the sky is gray, California streaming on such a winter's day". Seems like yesterday the entire country was sweltering in wicked summer heat with LA being the most Icelandic spot in the nation.
We're so good at medical studies that most of them are wrong - 0 views
-
It's possible to get the mental equivalent of whiplash from the latest medical findings, as risk factors are identified one year and exonerated the next. According to a panel at the American Association for the Advancement of Science, this isn't a failure of medical research; it's a failure of statistics, and one that is becoming more common in fields ranging from genomics to astronomy. The problem is that our statistical tools for evaluating the probability of error haven't kept pace with our own successes, in the form of our ability to obtain massive data sets and perform multiple tests on them. Even given a low tolerance for error, the sheer number of tests performed ensures that some of them will produce erroneous results at random.
1 - 7 of 7
Showing 20▼ items per page