Group items matching

in title, tags, annotations or url

Each year millions of middle-school students nationwide spend angst-filled months waiting to hear if they scored high enough on an entrance exam to attend a selective public high school. In New York City alone more than 27,000 students apply for precious spots in the three best-known schools: Stuyvesant, Brooklyn Technical and Bronx High School of Science. What Mr. Dobbie and Mr. Fryer wanted to know was just how much of a difference attending one of these high schools makes in the long run for students with similar equal admissions test scores.

The state teacher's union declared a major court victory on Wednesday after a judge ruled that a school district cannot impose a teacher evaluation system where 40 percent is based on student test scores. Student test scores will count for at least 20 percent of an educator's evaluation, and could count for an additional 20 percent only if it is collectively bargained, NYSUT spokesman Carl Korn said.

It is unfortunate that the day after a court decision held that NY teachers should be evaluated by use of multiple assessments, with student scores on state standardized tests only one minor factor, today, the appellate court said that the DOE could release the teacher data reports to the public, based only on these same test scores. 

The Recovery School District (RSD) based its 2009-10 alleged success on School Performance Scores.  However, the RSD failed to indicate that 30% of its schools did not even have School Performance Scores.  The RSD's alleged success omitted 30% of its schools.  

Tests used to be just for evaluating students, but now the testing of students is used to evaluate teachers and, in fact, the entire educational system.  On an individual level, some students and parents have noticed a change - more standardized tests and more classroom and homework time devoted to preparation for them. So what exactly do test scores tell us?

I have discussed before how most testing data released to the public are cross-sectional, and how comparing them between years entails the comparison of two different groups of students. One way to address these issues is to calculate and release school- and district-level value-added scores. Value added estimates are not only longitudinal (i.e., they follow students over time), but the models go a long way toward for differences in the characteristics of students between schools and districts. Put simply, these models calculate "expectations" for student test score gains based on student (and sometimes school) characteristics, which are then used to gauge whether schools' students did better or worse than expected.

I've written so many posts about the misinterpretation of testing data in news stories that I'm starting to annoy myself. For example, I've shown that year-to-year changes in testing results might be attributable to the fact that, each year, a different set of students takes the test. I've discussed the fact that proficiency rates are not test scores - they only tell you the proportion of students above a given line - and that the rates and actual scores can move in opposite directions (see this simple illustration). And I've pleaded with journalists, most of whom I like and respect, to write with care about these issues (and, I should note, many of them do so).

A year ago, the Los Angeles Times created a media sensation when it obtained the names and test scores of thousands of teachers, then commissioned a researcher to rate them in relation to their "effectiveness" in raising test scores. The Times then published online the names and ratings of those thousands of teachers. U.S. Secretary of Education Arne Duncan saluted the Times for rating teachers and naming names, but the overwhelming majority of testing and evaluation experts thought it was a terrible idea.

The state has thrown out the test scores of a top-performing Los Angeles school and of the highest-scoring campus in the nationally known Green Dot charter group after cheating was uncovered involving several teachers.

In my last post, I was considering how ineffective it is to rely on numbers, scores, policies, sanctions, or rewards to bring about the results we want in education.  At my most optimistic, I believe that education stakeholders all want the same things - though some people have a hard time articulating what they want without relying on test scores as a proxy for achievement or learning.  Still, for the sake of argument, let's leave motives out of this discussion.

KIPP, Inc. offers a draconian test factory model that specializes in its own patented varieties of behavioral sterilization and cultural neutering.  KIPP's Madison Avenue advertising campaign focuses on KIPP test scores, which are often higher than public school scores.  But then spending 60 percent more time in school drilling within a total compliance regime that regularly shoves out low performers has its own kinds of perverse rewards.

Gov. Chris Christie has been touting his plans for education overhaul, including the use of student test scores to evaluate teachers. It's the first full week of school, a traditional time for politicians to roll out proposed changes. It's also the week a new book on education, Howard Wainer's "Uneducated Guesses," was released by the Princeton University Press. It raises significant questions about the premise on which much of Christie's crusade is based - using student test scores to evaluate teachers.

I am in the middle of a lot of the debates going on throughout the country about education today. On one hand I am encouraged by the fact that we are focused on learning, although too often I would say that gets translated as a focus on achievement. And they're not quite the same. When you focus on achievement, you focus on test scores and you could miss out on whether or not the kids are actually learning. And we have ample evidence, based on the test scores, that sometimes the kids still aren't learning

The No Child Left Behind (NCLB) Act compelled states to design school accountability systems based on annual student assessments. The effect of this federal legislation on the distribution of student achievement is a highly controversial but centrally important question. This study presents evidence on whether NCLB has influenced student achievement based on an analysis of state-level panel data on student test scores from the National Assessment of Educational Progress (NAEP). The impact of NCLB is identified using a comparative interrupted time series analysis that relies on comparisons of the test-score changes across states that already had school accountability policies in place prior to NCLB and those that did not. Our results indicate that NCLB generated statistically significant increases in the average math performance of fourth graders (effect size 5 0.23 by 2007) as well as improvements at the lower and top percentiles. There is also evidence of improvements in eighth-grade math achievement, particularly among traditionally low-achieving groups and at the lower percentiles. However, we find no evidence that NCLB increased fourth-grade reading achievement.

"Pundits love to make cross-state comparisons and rank states on a variety of indicators (I'm guilty too). A favorite activity is comparing NAEP test scores across subjects, including comparing which states have the biggest test score gaps between children who qualify for subsidized lunch and children who don't. The simple conclusion - States with big gaps are bad - inequitable - and states with smaller gaps must being doing something right! It is generally assumed by those who report these gaps and rank states on achievement gaps that these gaps are appropriately measured - comparably measured - across states. That a low-income child in one state is similar to a low-income child in another. That the average low-income child or the average of low-income children in one state is comparable to the average of low-income children in another, and that the average of non-low income children in one state is comparable to the average of non-low income children in another. LITTLE COULD BE FURTHER FROM THE TRUTH."

Former Florida Governor Jeb Bush and the Foundation for Excellence in Education have embarked on a well-funded campaign to spread selected Florida education reforms to other states. These reforms include assigning letter grades to schools, high-stakes testing, promotion and graduation requirements, bonus pay, a wide variety of alternative teacher credentialing policies, and various types of school choice mechanisms. This policy potpourri was recently presented by Gov. Bush in Michigan, and the documents used allow for a concrete consideration and review. Regrettably, Bush's Michigan speech relies on a selective misrepresentation of test score data. Further, he offers no evidence that the purported test score gains were caused by the recommended reforms. Other viable explanations, such as a major investment in class-size reduction and a statewide reading program, receive no or little attention. Moreover, the presentation ignores less favorable findings, while evidence showing limited or negative effects of the proposed strategies is omitted. Considering the overwhelming evidence that retention is ineffective (if not harmful), it is troubling to see Mr. Bush endorse such an approach. Finally, Florida's real problems of inequitable and inadequate education remain unaddressed.

While the Obama administration's proposed reform efforts continue the obsession with test scores and the folly of trying to be globally competitive without being globally competent, students in other countries are hard at work to ensure that they become globally competent. America is "woefully behind almost all other countries of the world, particularly industrialized countries" in terms of foreign language studies, as Marty Abbott, the education director at ACTFL, told Education Week's Erik Robelen. I have been aware of and worried about this well-known fact, but what I saw and heard over the last few weeks gave me more reason to worry.

There are much more subtle ways to fraudulently raise test scores than tampering with student test papers.  One that I've been thinking about a lot lately is the practice by many charter schools of improving their test scores through attrition.  Up until recently, these charters have not been very upfront about this factor contributing to their success.  With everything that these charters have at stake in preserving their reputations and their rich funders, I can understand why they might try to conceal what they're doing.  Of course they have the right to portray their business in the most favorable light possible.  That's what most businesses do.  The reason that I've become so involved in uncovering the truth behind these successes is that these ruses have tricked politicians into believing that one of the big solutions in fixing education is to expand the influence of charter schools.  Only states that agree to lift caps on charters were even eligible to apply to Obama's Race To The Top initiative.

As I predicted, there was no miracle in DC under Michelle Rhee's reign. At least not one you can see on the NAEP scores for fourth or eighth grade students in reading and math.

"This is the executive summary of the statement of the American Statistical Association on the use of value-added assessment to evaluate teachers. Please share it with other teachers, with principals, and school board members. Please share it with your legislators and other elected officials. Send it to your local news outlets. The words are clear: Teachers account for between 1 and 14% of the variation in test scores. And this is very important to remember: "Ranking teachers by their VAM scores can have unintended consequences that reduce quality.""