Group items tagged

Unfortunately the mainstream media continue to repeat without dispute Suransky's claim that the progress reports were much more "stable" this year, even though 60% of schools changed grades.     Not one reporter, to my knowledge anyway, has bothered to point out how experts have shown that 32-80% of the annual gains or losses in scores at the school level are essentially random - and yet 60% of the school grade is based upon these annual gains or losses. 

The roughly 65,000 students who have entered eighth grade in New York City public schools will face a formidable task in the coming months. In addition to completing homework assignments and taking tests, preparing for the dreaded state exams and meeting the city's multiple promotion requirements, all eighth-grade students who wish to attend public (non-charter) high schools in New York city must also submit applications in which they rank up to 12 programs or schools from among nearly 700 possibilities citywide.

This week, the Daily News published yet another editorial taking an unjustly negative view of district schools in comparison to the charter sector - in this case, arguing that the relatively high proficiency levels in upper grades at schools in the Harlem Success and Harlem Village charter chains are primarily due to those schools' extended days and school years. However, the latest available official data indicates that the schools in these two chains are also characterized by lower proportions of high-needs students than local district schools, and by extremely high rates of student attrition over time - in one case, a 68% drop in cohort size between 5th and 8th grades.

An organization called "The Third Way" released a report last week, in which they present descriptive data on what they call "middle class schools." The primary conclusion of their analysis is that "middle class schools" aren't "making the grade," and that they are "falling short on their most basic 21st century mission: To prepare kids to get a college degree." They also argue that "middle class schools" are largely ignored in our debate and policymaking, and we need a "second phase of school reform" in order to address this deficit. The Wall Street Journal swallowed the report whole, running a story presenting Third Way's findings under the headline "Middle class schools fail to make the grade."

The No Child Left Behind (NCLB) Act compelled states to design school accountability systems based on annual student assessments. The effect of this federal legislation on the distribution of student achievement is a highly controversial but centrally important question. This study presents evidence on whether NCLB has influenced student achievement based on an analysis of state-level panel data on student test scores from the National Assessment of Educational Progress (NAEP). The impact of NCLB is identified using a comparative interrupted time series analysis that relies on comparisons of the test-score changes across states that already had school accountability policies in place prior to NCLB and those that did not. Our results indicate that NCLB generated statistically significant increases in the average math performance of fourth graders (effect size 5 0.23 by 2007) as well as improvements at the lower and top percentiles. There is also evidence of improvements in eighth-grade math achievement, particularly among traditionally low-achieving groups and at the lower percentiles. However, we find no evidence that NCLB increased fourth-grade reading achievement.

The pre-K-3 movement, which refers to the years spanning prekindergarten to 3rd grade, wants to revolutionize early education through an ambitious list of connected initiatives, including universal access to free public preschool, mandatory full-day kindergarten, and curriculum that is seamlessly connected from preschool to 3rd grade. Increasing parent involvement is also a major focus.

Daniel Pink was recently interviewed on a local Washington, D.C. television show along with a local university official. You watch it all here, but I thought the few minutes he spent discussing the role of grades, autonomy and inquiry in education to be particularly thought-provoking.

While his research is limited, because E.T.S. is the only organization that has permitted him to test its product, he says the automated reader can be easily gamed, is vulnerable to test prep, sets a very limited and rigid standard for what good writing is, and will pressure teachers to dumb down writing instruction. The e-Rater's biggest problem, he says, is that it can't identify truth. He tells students not to waste time worrying about whether their facts are accurate, since pretty much any fact will do as long as it is incorporated into a well-structured sentence. "E-Rater doesn't care if you say the War of 1812 started in 1945," he said. Mr. Perelman found that e-Rater prefers long essays. A 716-word essay he wrote that was padded with more than a dozen nonsensical sentences received a top score of 6; a well-argued, well-written essay of 567 words was scored a 5. An automated reader can count, he said, so it can set parameters for the number of words in a good sentence and the number of sentences in a good paragraph. "Once you understand e-Rater's biases," he said, "it's not hard to raise your test score."

A while back, I noted that states and districts should exercise caution in assigning weights (importance) to the components of their teacher evaluation systems before they know what the other components will be. For example, most states that have mandated new evaluation systems have specified that growth model estimates count for a certain proportion (usually 40-50 percent) of teachers' final scores (at least those in tested grades/subjects), but it's critical to note that the actual importance of these components will depend in no small part on what else is included in the total evaluation, and how it's incorporated into the system. In slightly technical terms, this distinction is between nominal weights (the percentage assigned) and effective weights (the percentage that actually ends up being the case). Consider an extreme hypothetical example - let's say a district implements an evaluation system in which half the final score is value-added and half is observations. But let's also say that every teacher gets the same observation score. In this case, even though the assigned (nominal) weight for value-added is 50 percent, the actual importance (effective weight) will be 100 percent, since every teacher receives the same observation score, and so all the variation between teachers' final scores will be determined by the value-added component.

"I have reviewed, albeit superficially, the test-based components of several states' school rating systems (e.g., OH, FL, NYC, LA, CO), with a particular focus on the degree to which they are actually measuring student performance (how highly students score), rather than school effectiveness per se (whether students are making progress). Both types of measures have a role to play in accountability systems, even if they are often confused or conflated, resulting in widespread misinterpretation of what the final ratings actually mean, and many state systems' failure to tailor interventions to the indicators being used. One aspect of these systems that I rarely discuss is the possibility that the ratings systems are an end in themselves."

Just this week, Florida announced its new district grading system. These systems have been popping up all over the nation, and given the fact that designing one is a requirement of states applying for No Child Left Behind waivers, we are sure to see more. I acknowledge that the designers of these schemes have the difficult job of balancing accessibility and accuracy. Moreover, the latter requirement - accuracy - cannot be directly tested, since we cannot know "true" school quality. As a result, to whatever degree it can be partially approximated using test scores, disagreements over what specific measures to include and how to include them are inevitable (see these brief analyses of Ohio and California). As I've discussed before, there are two general types of test-based measures that typically comprise these systems: absolute performance and growth. Each has its strengths and weaknesses. Florida's attempt to balance these components is a near total failure, and it shows in the results.

Good grades do not a good school make. School Chancellor Dennis Walcott said the city is moving to massively overhaul seven schools it rated this fall with As or Bs because their high marks don't paint the full picture.

High schools serving the most challenging students were much likelier to get saddled with a bad grade from the Department of Education than schools that serve few high-needs kids, a Post analysis found.

"A while back, I argued that Florida's school grading system, due mostly to its choice of measures, does a poor job of gauging school performance per se. The short version is that the ratings are, to a degree unsurpassed by most other states' systems, driven by absolute performance measures (how highly students score), rather than growth (whether students make progress). Since more advantaged students tend to score more highly on tests when they enter the school system, schools are largely being judged not on the quality of instruction they provide, but rather on the characteristics of the students they serve."

"General students from kindergarten to third grade are retained by the privately operated schools at a slightly higher rate than district schools, according to the study report by the Independent Budget Office released Thursday."

"New York state is weighing whether to prohibit high-school teachers from grading their own students' Regents exams. The Board of Regents, which sets education policy for the state, is set to vote next week on the ban. "

"In the out-of-the-mouths-of-babes category, here's a post about the flaws of modern teacher evaluation that are evidence to an 11-year-old but, apparently, not to school reformers."

"New York City has just released the new round of results from its school rating system (they're called "progress reports"). It relies considerably more on student growth (60 out of 100 points) than absolute performance (25 points), and there are efforts to partially adjust most of the measures via peer group comparisons.* All of this indicates that the city's system is more focused on school rather than student test-based performance, compared with many other systems around the U.S. The ratings are high-stakes. Schools receiving low grades - a D or F in any given year, or a C for three consecutive years - enter a review process by which they might be closed. The number of schools meeting these criteria jumped considerably this year. There is plenty of controversy to go around about the NYC ratings, much of it pertaining to two important features of the system. They're worth discussing briefly, as they are also applicable to systems in other states."

"Here's a fact that may not surprise you: the children of the rich perform better in school, on average, than children from middle-class or poor families. Students growing up in richer families have better grades and higher standardized test scores, on average, than poorer students; they also have higher rates of participation in extracurricular activities and school leadership positions, higher graduation rates and higher rates of college enrollment and completion. Whether you think it deeply unjust, lamentable but inevitable, or obvious and unproblematic, this is hardly news. It is true in most societies and has been true in the United States for at least as long as we have thought to ask the question and had sufficient data to verify the answer. What is news is that in the United States over the last few decades these differences in educational success between high- and lower-income students have grown substantially."

"High school teachers in Seattle are saying no to the spread of high-stakes standardized tests. On January 10, the staff of Garfield High School voted unanimously to refuse to administer the Measures of Academic Progress (MAP) test to their ninth-grade students. For two weeks they've held firm, even as the superintendent of schools has threatened them with a 10-day unpaid suspension, and teachers at other schools have joined their boycott."