Statistical validation of results, as Shaffer described it, simply involves testing the null hypothesis: that the pattern you detect in your data occurs at random. If you can reject the null hypothesis—and science and medicine have settled on rejecting it when there's only a five percent or less chance that it occurred at random—then you accept that your actual finding is significant.
The problem now is that we're rapidly expanding our ability to do tests. Various speakers pointed to data sources as diverse as gene expression chips and the Sloan Digital Sky Survey, which provide tens of thousands of individual data points to analyze. At the same time, the growth of computing power has meant that we can ask many questions of these large data sets at once, and each one of these tests increases the prospects than an error will occur in a study; as Shaffer put it, "every decision increases your error prospects." She pointed out that dividing data into subgroups, which can often identify susceptible subpopulations, is also a decision, and increases the chances of a spurious error. Smaller populations are also more prone to random associations.
In the end, Young noted, by the time you reach 61 tests, there's a 95 percent chance that you'll get a significant result at random. And, let's face it—researchers want to see a significant result, so there's a strong, unintentional bias towards trying different tests until something pops out.
Young went on to describe a study, published in JAMA, that was a multiple testing train wreck: exposures to 275 chemicals were considered, 32 health outcomes were tracked, and 10 demographic variables were used as controls. That was about 8,800 different tests, and as many as 9 million ways of looking at the data once the demographics were considered.
S C on 20 Jul 09You are overweight for the most simple of reasons -- because you're eating the wrong foods, the wrong types of calories per meal, and you're also eating meals in the wrong patterns each day.Think closely about what we're about to tell you, since it's going to change the way you think about dieting...FOOD is more powerful than any prescription weight loss pills, because the FOOD that you eat can either make you THIN or FAT. You don't get fat because of a lack of exercising, that's a myth. You get fat because you don't eat the right foods at the right intervals each day.Also, the pattern that you choose to eat your meals each day is more powerful than any prescription weight loss pills. This is true because your body is like an 'engine' and it only needs certain foods at certain intervals each day, and if you don't eat the right foods at the right times then it won't burn those calories -- and you'll wind up storing those calories as fat tissue. (Hint: You need to eat more than 3 times per day to lose weight, but we'll show you the details later).Don't want to wait for details later. Go now and see this amazing 'diet'!Click Here
