Group items matching

in title, tags, annotations or url

"People tend to conform their factual beliefs to ones that are consistent with their cultural outlook, their world view," Braman says. The Cultural Cognition Project has conducted several experiments to back that up. Participants in these experiments are asked to describe their cultural beliefs. Some embrace new technology, authority and free enterprise. They are labeled the "individualistic" group. Others are suspicious of authority or of commerce and industry. Braman calls them "communitarians." In one experiment, Braman queried these subjects about something unfamiliar to them: nanotechnology — new research into tiny, molecule-sized objects that could lead to novel products. "These two groups start to polarize as soon as you start to describe some of the potential benefits and harms," Braman says. The individualists tended to like nanotechnology. The communitarians generally viewed it as dangerous. Both groups made their decisions based on the same information. "It doesn't matter whether you show them negative or positive information, they reject the information that is contrary to what they would like to believe, and they glom onto the positive information," Braman says.

"Basically the reason that people react in a close-minded way to information is that the implications of it threaten their values," says Dan Kahan, a law professor at Yale University and a member of The Cultural Cognition Project. Kahan says people test new information against their preexisting view of how the world should work. "If the implication, the outcome, can affirm your values, you think about it in a much more open-minded way," he says. And if the information doesn't, you tend to reject it. In another experiment, people read a United Nations study about the dangers of global warming. Then the researchers told the participants that the solution to global warming is to regulate industrial pollution. Many in the individualistic group then rejected the climate science. But when more nuclear power was offered as the solution, says Braman, "they said, you know, it turns out global warming is a serious problem."And for the communitarians, climate danger seemed less serious if the only solution was more nuclear power.

Statistical validation of results, as Shaffer described it, simply involves testing the null hypothesis: that the pattern you detect in your data occurs at random. If you can reject the null hypothesis—and science and medicine have settled on rejecting it when there's only a five percent or less chance that it occurred at random—then you accept that your actual finding is significant. The problem now is that we're rapidly expanding our ability to do tests. Various speakers pointed to data sources as diverse as gene expression chips and the Sloan Digital Sky Survey, which provide tens of thousands of individual data points to analyze. At the same time, the growth of computing power has meant that we can ask many questions of these large data sets at once, and each one of these tests increases the prospects than an error will occur in a study; as Shaffer put it, "every decision increases your error prospects." She pointed out that dividing data into subgroups, which can often identify susceptible subpopulations, is also a decision, and increases the chances of a spurious error. Smaller populations are also more prone to random associations. In the end, Young noted, by the time you reach 61 tests, there's a 95 percent chance that you'll get a significant result at random. And, let's face it—researchers want to see a significant result, so there's a strong, unintentional bias towards trying different tests until something pops out. Young went on to describe a study, published in JAMA, that was a multiple testing train wreck: exposures to 275 chemicals were considered, 32 health outcomes were tracked, and 10 demographic variables were used as controls. That was about 8,800 different tests, and as many as 9 million ways of looking at the data once the demographics were considered.