Group items tagged

Just this week, Florida announced its new district grading system. These systems have been popping up all over the nation, and given the fact that designing one is a requirement of states applying for No Child Left Behind waivers, we are sure to see more. I acknowledge that the designers of these schemes have the difficult job of balancing accessibility and accuracy. Moreover, the latter requirement - accuracy - cannot be directly tested, since we cannot know "true" school quality. As a result, to whatever degree it can be partially approximated using test scores, disagreements over what specific measures to include and how to include them are inevitable (see these brief analyses of Ohio and California). As I've discussed before, there are two general types of test-based measures that typically comprise these systems: absolute performance and growth. Each has its strengths and weaknesses. Florida's attempt to balance these components is a near total failure, and it shows in the results.

"In a direct challenge to New York's most high-profile education initiative, school superintendents from across the region are beginning an independent review of the accuracy of state-generated teacher ratings that are based on student test scores. The Lower Hudson Council of School Superintendents has hired a research center at the University of Wisconsin to study the state's first round of teacher scores, released last summer, and to evaluate the strengths and weaknesses of New York's approach. At least 80 school districts in the Lower Hudson Valley and Long Island are turning over data on thousands of students and teachers - all anonymously - so that researchers can run the numbers."

Would you rather have an actual person score your carefully crafted essay, or an automated software program designed for that purpose? I'd still take the flawed human being any day-assuming, of course, the proper expertise and that he or she is operating on a good night's sleep-but a new study suggests there is little, if any, difference in the reliability and accuracy of the computer approach.

Last week the New York Times provided valuable, disturbing information by reporting recent research on the growing educational achievement gap between rich and poor students, which has grown substantially over the past few decades, even while the achievement gap between black and white students has narrowed. As the author of one study put it, "family income appears more determinative of educational success than race." Yet, as is often true of the Times, what it gives with one hand, it takes with the other. For example, as the media watchdog group, Fairness and Accuracy in Reporting has long documented, while the paper of record frequently provides factual information about events, its interpretation of the facts buttresses against drawing the "wrong" conclusions about political-economic power relationships.

The numbers fell so drastically because, as announced last summer, state officials toughened the standards, paying more attention to grammar and spelling as well as to the factual accuracy of supporting details in essays. But they did not change the scoring system, resulting in a public relations disaster. What to do? They could live with the results - that after 15 years of education reform, three-fourths of Florida children could not write. Or they could scale the results upward after the fact, an embarrassment, but people probably would not be so angry if they had good scores.

In short, there are different ways to measure the gender gap, and their "accuracy" is not about the statistics as much as how they're interpreted. The gap is 75-80 cents on the male dollar if you're making no claims that the difference is attributable solely to discrimination. When you account for the underlying factors - and you must do so to interpret the data in this manner - you get a somewhat different picture of the extent of the problem (problem though it still is). Now, think about how easily this all applies to test data in education. We are inundated every day with average scores and rates - for schools, districts, states, subgroups of students, etc. These data are frequently compared between groups and institutions in much the same way as wages are compared between men and women.

Cohen's argument on the importance of infrastructure does not necessarily mean that we should abandon the testing of new evaluation systems, only that we should be very careful about how we interpret their results and the policy conclusions we draw from them (which is good advice at all times). Unfortunately, however, it seems that caution is in short supply.

With NBC airing a second "Education Nation" special that resembles an infomercial for charter schools and online learning, the media watchdog group FAIR(Fairness & Accuracy in reporting) held an event on Sept. 27, 2011 to clear the air. The goal according to FAIR was to offer a more reasonable conversation about public education than the corporate-interest perspective featured in "Education Nation." The panelists at the MisEducation Nation forum in New York City said the coverage offered by NBC was, at best, misguided-a noble but seriously uninformed effort, said Leonie Haimson, a New York City public school parent and leader of Class Size Matters, which advocates for reducing the number of students per teacher.

As policymakers and school leaders seek new ways to measure and improve teacher effectiveness, it's important for journalists and others to understand what is known about the topic so far, and what remains unsettled or unknown. This research brief does not synthesize all the studies in this highly technical field. But it does aim to improve the accuracy and clarity of reporting by exploring what the research says about timely questions surrounding the complex topic of teacher effectiveness.