Group items tagged

Statistical validation of results, as Shaffer described it, simply involves testing the null hypothesis: that the pattern you detect in your data occurs at random. If you can reject the null hypothesis—and science and medicine have settled on rejecting it when there's only a five percent or less chance that it occurred at random—then you accept that your actual finding is significant. The problem now is that we're rapidly expanding our ability to do tests. Various speakers pointed to data sources as diverse as gene expression chips and the Sloan Digital Sky Survey, which provide tens of thousands of individual data points to analyze. At the same time, the growth of computing power has meant that we can ask many questions of these large data sets at once, and each one of these tests increases the prospects than an error will occur in a study; as Shaffer put it, "every decision increases your error prospects." She pointed out that dividing data into subgroups, which can often identify susceptible subpopulations, is also a decision, and increases the chances of a spurious error. Smaller populations are also more prone to random associations. In the end, Young noted, by the time you reach 61 tests, there's a 95 percent chance that you'll get a significant result at random. And, let's face it—researchers want to see a significant result, so there's a strong, unintentional bias towards trying different tests until something pops out. Young went on to describe a study, published in JAMA, that was a multiple testing train wreck: exposures to 275 chemicals were considered, 32 health outcomes were tracked, and 10 demographic variables were used as controls. That was about 8,800 different tests, and as many as 9 million ways of looking at the data once the demographics were considered.

So in coming to this meeting of some of the very best in the field -- from Elizabeth Warren to George Soros -- I was keen to hear just what the strategy was to restore us to some sort of financial sanity. How could we avoid it again? Yet through the course of the morning, I was struck by two very different and very depressing points. The first is that things are actually much worse than anyone ever talks about. The pivot points of our financial system -- the infrastructure that lets free markets produce real wealth -- have become profoundly corrupted. Balance sheets are "fictions," as Professor Frank Partnoy put it. Trillions of dollars in liability hide behind these fictions. And as expert after expert demonstrated, practically every one of the design flaws that led to the collapse of the past few years remains essentially unchanged within our financial system still. That bubble burst, but we can already see the soaring profits of the same firms that sucked billions in taxpayer funds. The cycle has started again. But the second point was even worse. Expert after expert spoke as if the problems we faced were simple math errors. As if regulators had just miscalculated, like a pilot who accidentally overshoots the run way, or an engineer who mis-estimates the weight of cargo on a plane. And so, because these were mere errors, people spoke as if these errors could be corrected by a bunch of good ideas. The morning was filled with good ideas. An angry earnestness was the tone of the day.

There were exceptions. The increasingly prominent folk-hero for the middle class, Elizabeth Warren, tied the endless list of problems to the endless power of "the banking lobby." But that framing was rare. Again and again, we were led back to a frame of bad policies that smart souls could correct. At least if "the people" could be educated enough to demand that politicians do something sensible. This is a profound denial. The gambling on Wall Street was not caused by the equivalent of errors in arithmetic. It was caused by a corruption of the system by which we regulate those markets. No true theorist of free markets -- and certainly none of the heroes of even the libertarian right -- believe that infrastructure markets like financial systems can be left free of any regulation, including the regulation of rules against fraud. Yet that ignorant anarchy was the precise rule that governed a large part of our financial system. And not by accident: An enormous amount of political influence was brought to bear on the regulators of these core institutions of a free market to get them to turn a blind eye to Wall Street's "innovations." People who should have known better yielded to this political pressure. Smart people did stupid things because "the politics" of doing right was impossible. Why? Why was their no political return from sensible policy? The answer is so obvious that one feels stupid to even remark it. Politicians are addicts. Their dependency is campaign cash. And in their obsessive search for campaign funds, they let these funders convince them that for the first time in capitalism's history, markets didn't need the basic array of trust-producing regulation. They believed this insanity because it made it easier for them -- in good faith -- to accept the money and steer financial policy over the cliff. Not a single presentation the whole morning focused this part of the problem. There wasn't even speculation about how we could build an alternative to this campaign funding system of pathological dependency, so that policy makers could afford to hear sense rather than obsessively seek campaign dollars. The assembled experts were even willing to brainstorm about how to educate ordinary Americans about the intricacies of financial regulation. But the idea of changing the pathological economy of influence that governs how Washington governs wasn't even a hint. We need to admit our (democracy's) problem. We need to get beyond this stage of denial. We need to recognize that until we release our leaders from a system that forces them to ignore good sense when there is an opportunity for large campaign cash, we won't have policy that makes sense. Wall Street continues unchanged because the Congress that would change it is already shuttling to Wall Street fundraisers. Both parties are already pandering to this power, so they can find the fix to fund the next cycle of campaigns. Throughout the morning, expert after expert celebrated the brilliance in Franklin Roosevelt's response to the Nation's last truly great financial collapse. They yearned for a modern version of his system of regulation. But we won't get to Franklin Roosevelt's brilliance till we accept Teddy Roosevelt's insight -- that privately funded public elections tend inevitably towards this kind of corruption. And until we solve that (eminently solvable) problem, we won't make any progress in making America's finances safe again.