Group items tagged

Unlike many of my colleagues and friends, I personally support the use of standardized testing results in education policy, even, with caution and in a limited role, in high-stakes decisions. That said, I also think that the focus on test scores has gone way too far and their use is being implemented unwisely, in many cases to a degree at which I believe the policies will not only fail to generate improvement, but may even risk harm. In addition, of course, tests have a very productive low-stakes role to play on the ground - for example, when teachers and administrators use the results for diagnosis and to inform instruction. Frankly, I would be a lot more comfortable with the role of testing data - whether in policy, on the ground, or in our public discourse - but for the relentless flow of misinterpretation from both supporters and opponents. In my experience (which I acknowledge may not be representative of reality), by far the most common mistake is the conflation of student and school performance, as measured by testing results.

Now, in studies that threaten to shake the foundation of high-stakes test-based accountability, Mr. Stroup and two other researchers said they believe they have found the reason: a glitch embedded in the DNA of the state exams that, as a result of a statistical method used to assemble them, suggests they are virtually useless at measuring the effects of classroom instruction. Pearson, which has a five-year, $468 million contract to create the state's tests through 2015, uses "item response theory" to devise standardized exams, as other testing companies do. Using I.R.T., developers select questions based on a model that correlates students' ability with the probability that they will get a question right. That produces a test that Mr. Stroup said is more sensitive to how it ranks students than to measuring what they have learned.

In a provocative and influential paper, Jesse Rothstein (2010) finds that standard value-added models (VAMs) suggest implausible future teacher effects on past student achievement, a finding that obviously cannot be viewed as causal. This is the basis of a falsification test (the Rothstein falsification test) that appears to indicate bias in VAM estimates of current teacher contributions to student learning. Rothstein's finding is significant because there is considerable interest in using VAM teacher effect estimates for high-stakes teacher personnel policies, and the results of the Rothstein test cast considerable doubt on the notion that VAMs can be used fairly for this purpose. However, in this paper, we illustrate-theoretically and through simulations-plausible conditions under which the Rothstein falsification test rejects VAMs even when there is no bias in estimated teacher effects, and even when students are randomly assigned conditional on the covariates in the model. On the whole, our findings show that the "Rothstein falsification test" is not definitive in showing bias, which suggests a much more encouraging picture for those wishing to use VAM teacher effect estimates for policy purposes.

When I became a principal, nearly thirteen years ago, the era of high-stakes testing known as NCLB was just beginning. I was in a doctoral program at Teachers College at the time. I would argue with great passion for why we needed NCLB and testing to close the achievement gap. I can remember many a discussion with former commissioner, Thomas Sobol, who was one of our professors, on the topic of high stakes testing.  Frankly, Dr. Sobol was right, and I was wrong.  The downside of high-stakes standardized testing has far outweighed the good.

In a New York City middle school, the principal asked teachers to spend fifteen minutes a day with students practicing how to answer multiple-choice math questions in preparation for the state-mandated test. One teacher protested, explaining she taught Italian and English, not math. But the principal insisted, and she followed his directive. As you might suspect, the plan failed, and in the end, fewer than one in four New York City middle schoolers passed the exam. While the importance of the test dominated the formal curriculum, the lessons learned through the hidden curriculum were no less powerful. Students learned that test scores mattered more than English or Italian, and that teachers did not make the key instructional decisions. In fact once the test was over, one-third of the students in her class stopped attending school, skipping the last five weeks of the school year.

Tests used to be just for evaluating students, but now the testing of students is used to evaluate teachers and, in fact, the entire educational system.  On an individual level, some students and parents have noticed a change - more standardized tests and more classroom and homework time devoted to preparation for them. So what exactly do test scores tell us?

"The civil rights groups apparently are unaware if the history of standardized testing, and its ties to the eugenics movement. I wrote about that in chapter 4 of "Left Back." Historically, standardized tests were used to deny educational opportunities to under served groups and to re-enforce theories of white supremacy, based on test scores. Like school choice, standardized testing was a weapon used by racists to deny civil rights, not a force for civil rights."

We all know that high stakes testing changes the school environment. Teachers and administrators are stressed because the tests will be tied to their evaluations. Students and support staff are stressed because there is a great deal of pressure to do well on the test. After all, it wasn't too long ago that the New York Post and other media outlets published the Value Added scores of over 18,000 teachers. The test has pushed students, parents and educators to their limits, but perhaps that is the greater plan.

The big headline from the recent Gates/Scholastic survey of teachers is that only 28% of teachers see standardized tests as an essential or important gauge of student assessment, and only 26% say they are accurate as a reflection of student knowledge. Another question reveals part of the reason this may be so - only 45% of teachers think their students take these tests seriously, or perform to the best of their ability. We have been stuck in an accountability rut for the past decade, with most reform initiatives revolving around test scores of one sort or another. Even the concept of student learning has been subtly redefined to mean "achievement on end of year tests." There are several assumptions that have driven this.

Suspicious test scores in roughly 200 school districts resemble those that entangled Atlanta in the biggest cheating scandal in American history, an investigation by The Atlanta Journal-Constitution shows. The newspaper analyzed test results for 69,000 public schools and found high concentrations of suspect math or reading scores in school systems from coast to coast. The findings represent an unprecedented examination of the integrity of school testing. The analysis doesn't prove cheating. But it reveals that test scores in hundreds of cities followed a pattern that, in Atlanta, indicated cheating in multiple schools.

The DRA is a test that very few students take because it is given only in schools that don't go above second grade. In all other elementary schools, the state uses the New England Common Assessment Program (NECAP), a series of reading, writing, math and science tests used by Rhode Island, New Hampshire, Vermomt and Maine. Those tests start in grade 3. Last year, only 6 of the state's 168 elementary schools took the DRA, which makes Blackstone Valley Prep a big fish in a very small pond. "There are a lot of schools that are high-performing that don't take this test," said Krieger.

A few months back I got into an interesting discussion with my high school friends on Facebook about the books we read in our tenth grade advanced English class at Westhill High School in Stamford. My friend Debbie, who's clearly even more of a pack rat than my mother, still had the syllabus, and was able to rattle off impressively long list of books that we'd read and analyzed. When I compared it to the number of books my daughter, a high school sophomore, will get through this year in her advanced English class, it's really quite astounding. But actually, it's not. When I look at the school calendar, the entire month of March is lost to CMT/CAPT testing.  And that's just the actual testing. Much of the month before will be devoted to exercises that prepare students for the tests. Not for reading great works of literature and learning to use critical thinking skills, but rather for learning test taking skills.

Clearly our emphasis on testing and the consequent narrowing of the curriculum contributes to the problem.  School have, as George Wood of the Forum for Education and Democracy notes, "a perverse incentive to allow or encourage students to leave" especially if they are likely to be low scorers on the tests by which schools are evaluated.  Anyone who doubts this need merely look at the track record of Texas during the Governorship of George W. Bush, when its claimed remarkable improvements in state test scores later became the basis of the perversely named legislation No Child Left Behind.  In Texas, sometimes students were held back in 9th grade multiple times because the state tests were given in 10th.  After a second holding back students might be encouraged to leave, hiding the dropout rate by listing the child as having gone to an alternative educational program because s/he said s/he might eventually get a GED.  Or after being held back once, the child would be told s/he had made so much progress s/he was being skipped directly to 11th, and thus not tested.  Rod Paige became U. S. Secretary of Education, after being honored as supposedly the best Superintendent in the nation by a professional organization, largely on claims of more than a 90% graduation rate in Houston schools, at a time when only around 40% of those who entered in 7th grade graduated on time with their cohort.  Those forced out or held back and then skipped were heavily from poor families that were African-American or Hispanic.

The Bill & Melinda Gates Foundation's "Measures of Effective Teaching" (MET) Project seeks to validate the use of a teacher's estimated "value-added"-computed from the year-on-year test score gains of her students-as a measure of teaching effectiveness. Using data from six school districts, the initial report examines correlations between student survey responses and value-added scores computed both from state tests and from higher-order tests of conceptual understanding. The study finds that the measures are related, but only modestly. The report interprets this as support for the use of value-added as the basis for teacher evaluations. This conclusion is unsupported, as the data in fact indicate that a teachers' value-added for the state test is not strongly related to her effectiveness in a broader sense. Most notably, value-added for state assessments is correlated 0.5 or less with that for the alternative assessments, meaning that many teachers whose value-added for one test is low are in fact quite effective when judged by the other. As there is every reason to think that the problems with value-added measures apparent in the MET data would be worse in a high-stakes environment, the MET results are sobering about the value of student achievement data as a significant component of teacher evaluations.

Why are our 9 year olds subjected to state exams that last as long or are longer than entrance and certifying exams for adult professionals who make life and death decisions?  Why are the 75-minute third-grade state exams of 2005 no longer enough?  The honest answer is that testing is now hardly about students at all.

The New York State tests, going on now in middle and elementary schools, have always been high stakes for students, particularly in fourth and seventh grades, when their scores determine whether they end up in the very awful school they are zoned for or the very attractive magnet school that draws from a larger and more competitive pool. But the stakes have recently become equally high for teachers, whose ability to teach is being determined by their ability to improve students' test scores. Many people think it's about time. Teachers need to be held accountable for the work they are being paid to do, and many, many teachers need to get better at teaching. But tying teacher performance to student test scores is having an opposite effect: It's producing worse teachers.

Students across the state are still scratching their heads over an absurd state test question about a talking pineapple. The puzzler on the eighth-grade reading exam stumped even educators and has critics saying the tests, which are becoming more high stakes, are flawed. "I think it's weird that they put such a silly question on a state test. What were they thinking?" said Bruce Turley, 14, an eighth-grader at Lower Manhattan Community Middle School. "I thought it was a little strange, but I just answered it as best as I could," said his classmate Tyree Furman, 14. "You just have to give it your best answer. These are important tests."

While his research is limited, because E.T.S. is the only organization that has permitted him to test its product, he says the automated reader can be easily gamed, is vulnerable to test prep, sets a very limited and rigid standard for what good writing is, and will pressure teachers to dumb down writing instruction. The e-Rater's biggest problem, he says, is that it can't identify truth. He tells students not to waste time worrying about whether their facts are accurate, since pretty much any fact will do as long as it is incorporated into a well-structured sentence. "E-Rater doesn't care if you say the War of 1812 started in 1945," he said. Mr. Perelman found that e-Rater prefers long essays. A 716-word essay he wrote that was padded with more than a dozen nonsensical sentences received a top score of 6; a well-argued, well-written essay of 567 words was scored a 5. An automated reader can count, he said, so it can set parameters for the number of words in a good sentence and the number of sentences in a good paragraph. "Once you understand e-Rater's biases," he said, "it's not hard to raise your test score."

Many people have wondered how the New York State Education Department permitted the nonsensical story about the pineapple and the hare to get onto the state test. This is not the first time a really bad reading passage got onto the test and it won't be the last. State Commissioner John King was quick to issue a defensive statement saying that people were reading the story "out of context," as if the full story made sense (it didn't). And he was quick to pin the blame on teachers, who supposedly had reviewed all the test items. It was the teachers' fault, not his. In an era where Accountability is the hallmark of education policy, King was quick to refuse any accountability for what happened on his watch. These days, the ones at the top never accept accountability for what goes wrong, that's for the "little people" like teachers and students, not for the bigwigs. No one holds them accountable, and they never accept any. None of them ever says, as President Harry S Truman did, "the buck stops here."

Tests, tests and more tests won't fix the problems with our nation's schools. More funding would certainly help in an era of widespread state budget deficits. But the real problem, says Stephen Krashen, is poverty.