Group items tagged

Statistical validation of results, as Shaffer described it, simply involves testing the null hypothesis: that the pattern you detect in your data occurs at random. If you can reject the null hypothesis—and science and medicine have settled on rejecting it when there's only a five percent or less chance that it occurred at random—then you accept that your actual finding is significant. The problem now is that we're rapidly expanding our ability to do tests. Various speakers pointed to data sources as diverse as gene expression chips and the Sloan Digital Sky Survey, which provide tens of thousands of individual data points to analyze. At the same time, the growth of computing power has meant that we can ask many questions of these large data sets at once, and each one of these tests increases the prospects than an error will occur in a study; as Shaffer put it, "every decision increases your error prospects." She pointed out that dividing data into subgroups, which can often identify susceptible subpopulations, is also a decision, and increases the chances of a spurious error. Smaller populations are also more prone to random associations. In the end, Young noted, by the time you reach 61 tests, there's a 95 percent chance that you'll get a significant result at random. And, let's face it—researchers want to see a significant result, so there's a strong, unintentional bias towards trying different tests until something pops out. Young went on to describe a study, published in JAMA, that was a multiple testing train wreck: exposures to 275 chemicals were considered, 32 health outcomes were tracked, and 10 demographic variables were used as controls. That was about 8,800 different tests, and as many as 9 million ways of looking at the data once the demographics were considered.

So in coming to this meeting of some of the very best in the field -- from Elizabeth Warren to George Soros -- I was keen to hear just what the strategy was to restore us to some sort of financial sanity. How could we avoid it again? Yet through the course of the morning, I was struck by two very different and very depressing points. The first is that things are actually much worse than anyone ever talks about. The pivot points of our financial system -- the infrastructure that lets free markets produce real wealth -- have become profoundly corrupted. Balance sheets are "fictions," as Professor Frank Partnoy put it. Trillions of dollars in liability hide behind these fictions. And as expert after expert demonstrated, practically every one of the design flaws that led to the collapse of the past few years remains essentially unchanged within our financial system still. That bubble burst, but we can already see the soaring profits of the same firms that sucked billions in taxpayer funds. The cycle has started again. But the second point was even worse. Expert after expert spoke as if the problems we faced were simple math errors. As if regulators had just miscalculated, like a pilot who accidentally overshoots the run way, or an engineer who mis-estimates the weight of cargo on a plane. And so, because these were mere errors, people spoke as if these errors could be corrected by a bunch of good ideas. The morning was filled with good ideas. An angry earnestness was the tone of the day.

There were exceptions. The increasingly prominent folk-hero for the middle class, Elizabeth Warren, tied the endless list of problems to the endless power of "the banking lobby." But that framing was rare. Again and again, we were led back to a frame of bad policies that smart souls could correct. At least if "the people" could be educated enough to demand that politicians do something sensible. This is a profound denial. The gambling on Wall Street was not caused by the equivalent of errors in arithmetic. It was caused by a corruption of the system by which we regulate those markets. No true theorist of free markets -- and certainly none of the heroes of even the libertarian right -- believe that infrastructure markets like financial systems can be left free of any regulation, including the regulation of rules against fraud. Yet that ignorant anarchy was the precise rule that governed a large part of our financial system. And not by accident: An enormous amount of political influence was brought to bear on the regulators of these core institutions of a free market to get them to turn a blind eye to Wall Street's "innovations." People who should have known better yielded to this political pressure. Smart people did stupid things because "the politics" of doing right was impossible. Why? Why was their no political return from sensible policy? The answer is so obvious that one feels stupid to even remark it. Politicians are addicts. Their dependency is campaign cash. And in their obsessive search for campaign funds, they let these funders convince them that for the first time in capitalism's history, markets didn't need the basic array of trust-producing regulation. They believed this insanity because it made it easier for them -- in good faith -- to accept the money and steer financial policy over the cliff. Not a single presentation the whole morning focused this part of the problem. There wasn't even speculation about how we could build an alternative to this campaign funding system of pathological dependency, so that policy makers could afford to hear sense rather than obsessively seek campaign dollars. The assembled experts were even willing to brainstorm about how to educate ordinary Americans about the intricacies of financial regulation. But the idea of changing the pathological economy of influence that governs how Washington governs wasn't even a hint. We need to admit our (democracy's) problem. We need to get beyond this stage of denial. We need to recognize that until we release our leaders from a system that forces them to ignore good sense when there is an opportunity for large campaign cash, we won't have policy that makes sense. Wall Street continues unchanged because the Congress that would change it is already shuttling to Wall Street fundraisers. Both parties are already pandering to this power, so they can find the fix to fund the next cycle of campaigns. Throughout the morning, expert after expert celebrated the brilliance in Franklin Roosevelt's response to the Nation's last truly great financial collapse. They yearned for a modern version of his system of regulation. But we won't get to Franklin Roosevelt's brilliance till we accept Teddy Roosevelt's insight -- that privately funded public elections tend inevitably towards this kind of corruption. And until we solve that (eminently solvable) problem, we won't make any progress in making America's finances safe again.

No silver bullets. This is the simple premise of Diane Ravitch’s new book, “The Death and Life of the Great American School System,” which is being brought out this week by Basic Books. Written by one of our nation’s most respected scholars, it has been eagerly awaited. But it has also been, at least in some quarters, anticipated with a certain foreboding, because it was likely to debunk much of the conventional — and some not so conventional — wisdom surrounding education reform. Click Image to Enlarge

What of the once-great comprehensive high schools, institutions with history and in some cases a track record of success going back generations? As time moves on, it is fast becoming clear that the new small schools, many with inane themes (how about the School of Peace and Diversity?), can never substitute for a good neighborhood high school, which can become a center of communal life and pride. Ms. Ravitch’s report underscores the fact that the trick is to fix the neighborhood schools beset with problems, not destroy them.

It is not only the foundations that Ms. Ravitch blames for the current crisis: government has also failed in the attempt to reform the schools from above, lacking a clear perspective of how schools work on a day-to-day basis. Thus, the major federal initiative, No Child Left Behind, well intentioned as it may have been, ended up damaging the quality of education, not improving it. While the federal government declares schools as “failing” and prescribes sanctions for schools not meeting its goal of “annual yearly progress,” it is the states that are allowed to write and administer the tests. This has led to a culture of ever easier tests and more test preparation rather than real instruction. More ominously, it led to such scandals as the New York State Education Department lowering the “cut scores” that define the line between passing and failing. Ms. Ravitch suggests that the proper roles of the states and federal government have been reversed under NCLB. Maybe the standards for achievement should be set in Washington, which, after all, administers the National Assessment of Educational Progress , and the solutions found at the local level, using the accurate data provided by Washington. Instead of moving in a different direction from the failed NCLB model of the Bush Administration, the Obama administration has adopted and expanded on them.

"Despite the fact that the legal admissibility of scanned paper documents has been established for nearly 20 years and is nailed down in legislation and standards around the world, there is still this suspicion among users that they may need to produce the original paper copy at some stage," Mancini said. The survey also found that many documents are "born digital," then printed out to be signed and later scanned into document systems.