Group items tagged

Statistical validation of results, as Shaffer described it, simply involves testing the null hypothesis: that the pattern you detect in your data occurs at random. If you can reject the null hypothesis—and science and medicine have settled on rejecting it when there's only a five percent or less chance that it occurred at random—then you accept that your actual finding is significant. The problem now is that we're rapidly expanding our ability to do tests. Various speakers pointed to data sources as diverse as gene expression chips and the Sloan Digital Sky Survey, which provide tens of thousands of individual data points to analyze. At the same time, the growth of computing power has meant that we can ask many questions of these large data sets at once, and each one of these tests increases the prospects than an error will occur in a study; as Shaffer put it, "every decision increases your error prospects." She pointed out that dividing data into subgroups, which can often identify susceptible subpopulations, is also a decision, and increases the chances of a spurious error. Smaller populations are also more prone to random associations. In the end, Young noted, by the time you reach 61 tests, there's a 95 percent chance that you'll get a significant result at random. And, let's face it—researchers want to see a significant result, so there's a strong, unintentional bias towards trying different tests until something pops out. Young went on to describe a study, published in JAMA, that was a multiple testing train wreck: exposures to 275 chemicals were considered, 32 health outcomes were tracked, and 10 demographic variables were used as controls. That was about 8,800 different tests, and as many as 9 million ways of looking at the data once the demographics were considered.

Proponents of comprehensive regulatory reform hope for sweeping measures to protect consumers from predatory lending, rein in high-stakes Wall Street trading in arcane derivatives, boost capital requirements for banks that want to bet big with depositors' money and spread some regulatory sunshine on the dark pools of the “shadow banking system” that caught regulators flat-footed when the market spiraled into the abyss in the fall of 2008. “We cannot afford to let the status quo continue,” Sheila Bair, head of the Federal Deposit Insurance Corp., told a meeting of business economists in Washington. The final law is still in doubt. Sen. Christopher Dodd, D-Conn., has pressed for reform during a year of intensely partisan bickering. On Friday, Dodd — a lame duck who announced his retirement after disclosures that he accepted favorable terms from subprime lender Countrywide Financial — claimed that the Senate Banking Committee he chairs was “days away” from wrapping up a bill. Any resolution faces a major political hurdle that has drawn the most public attention: a proposal to create a new agency to protect consumers from predatory lending and other abusive financial practices. While the "systemic risks" to the financial system may represent a bigger threat in dollar terms, voters might be more focused on the consumer impact.Dodd said that’s not hard to understand.“The subject matter of derivatives and swaps and the issue of systemic risk and too-big-to- fail seem somewhat removed from the general public,” he told CNBC after the Senate compromise was reached. “Watching my credit card go to 32 percent rates and huge fees, watching prepayment penalties on mortgages, these are things that millions of people understand.”