Statistical validation of results, as Shaffer described it, simply involves testing the null hypothesis: that the pattern you detect in your data occurs at random. If you can reject the null hypothesis—and science and medicine have settled on rejecting it when there's only a five percent or less chance that it occurred at random—then you accept that your actual finding is significant.
The problem now is that we're rapidly expanding our ability to do tests. Various speakers pointed to data sources as diverse as gene expression chips and the Sloan Digital Sky Survey, which provide tens of thousands of individual data points to analyze. At the same time, the growth of computing power has meant that we can ask many questions of these large data sets at once, and each one of these tests increases the prospects than an error will occur in a study; as Shaffer put it, "every decision increases your error prospects." She pointed out that dividing data into subgroups, which can often identify susceptible subpopulations, is also a decision, and increases the chances of a spurious error. Smaller populations are also more prone to random associations.
In the end, Young noted, by the time you reach 61 tests, there's a 95 percent chance that you'll get a significant result at random. And, let's face it—researchers want to see a significant result, so there's a strong, unintentional bias towards trying different tests until something pops out.
Young went on to describe a study, published in JAMA, that was a multiple testing train wreck: exposures to 275 chemicals were considered, 32 health outcomes were tracked, and 10 demographic variables were used as controls. That was about 8,800 different tests, and as many as 9 million ways of looking at the data once the demographics were considered.
Group items tagged
Vulnerability Indexes, Homelessness, and Disability - 0 views
-
Looks at services being devised/piloted to assist the long-term homeless (approximately 1/3 of all homeless folks). Individuals in this population tend to have physical and/or mental disabilities. These issues demonstrate the inextricability of disability issues from broader socioeconomic concerns like urban poverty and homelessness.
We're so good at medical studies that most of them are wrong - 0 views
-
It's possible to get the mental equivalent of whiplash from the latest medical findings, as risk factors are identified one year and exonerated the next. According to a panel at the American Association for the Advancement of Science, this isn't a failure of medical research; it's a failure of statistics, and one that is becoming more common in fields ranging from genomics to astronomy. The problem is that our statistical tools for evaluating the probability of error haven't kept pace with our own successes, in the form of our ability to obtain massive data sets and perform multiple tests on them. Even given a low tolerance for error, the sheer number of tests performed ensures that some of them will produce erroneous results at random.
Bernie Sanders Puts Barack Obama to Shame | Rolling Stone Politics - 0 views
-
Not long ago I was sitting at home writing something for publication - I won't say what, except that it was a passage about a certain politician on the Hill. Out of habit I launched into a description that was full of nasty and personal language, and I was about to press on to the next part of the piece when suddenly I hit a mental speed bump. A voice in my head whispered - this really happened - "If you write that shit and Bernie Sanders sees it, he's going to be disappointed in you." So I went back and removed the gratuitous body blows from the article.
The reflexive call for fewer liberties - Glenn Greenwald - Salon.com - 0 views
-
Demands that it be easier to involuntarily commit the mentally ill are knee-jerk and irrational
The Politics of Reproductive Healthcare |The Political Pragmatic - 0 views
-
Since the 2010 elections, according to the Guttmacher Institute, Republican legislators have introduced more than 2,000 bills related to women's reproductive healthcare. The assault on women's health and freedom denies millions of women access to affordable contraception, life-saving cancer screenings, testing for H.I.V. and other sexually transmitted diseases. Some laws call for measures as drastic as imprisonment. In 2013, National Advocates for Pregnant Women, published the results of a study that examined cases in which a woman's pregnancy led to arrests and incarceration, detentions in hospitals, mental institutions, and forced medical interventions. The summary of some cases follows:
1 - 7 of 7
Showing 20▼ items per page