Group items matching

in title, tags, annotations or url

A recent article about the implementation of new teacher evaluations in Tennessee details some of the complicated issues with which state officials, teachers and administrators are dealing in adapting to the new system. One of these issues is somewhat technical - whether the various components of evaluations, most notably principal observations and test-based productivity measures (e.g., value-added) - tend to "match up." That is, whether teachers who score high on one measure tend to do similarly well on the other (see here for more on this issue).

The problem with education in Connecticut is income inequality, not teacher quality. Unfortunately, the plans Gov. Dannel P. Malloy has outlined for education reform - for the most part - take us in entirely the wrong direction. Education in Connecticut is a paradox. Though the National Assessment of Educational Progress consistently ranks the state among the highest scoring for student achievement, we also suffer from the highest black/white and poor/non-poor achievement gaps in the country.

Most of the controversy surrounding value-added and other test-based models of teacher productivity centers on the high-stakes use of these estimates. This is unfortunate - no matter what you think about these methods in the high-stakes context, they have a great deal of potential to improve instruction. When supporters of value-added and other growth models talk about low-stakes applications, they tend to assert that the data will inspire and motivate teachers who are completely unaware that they're not raising test scores. In other words, confronted with the value-added evidence that their performance is subpar (at least as far as tests are an indication), teachers will rethink their approach. I don't find this very compelling. Value-added data will not help teachers - even those who believe in its utility - unless they know why their students' performance appears to be comparatively low. It's rather like telling a baseball player they're not getting hits, or telling a chef that the food is bad - it's not constructive.

I can think of no other profession in the public or private sector in which this kind of unreliable and potentially damaging data is made public.  The only effect of this will be to further undermine teacher morale -- already at an all-time low in this city -- and to dissuade teachers from working in our public schools and with the highest needs children.  Yet so far, GothamSchools is the only media outlet that has pledged not to publish them.  Meanwhile, the Governor is pushing a deadline of Thursday for the state and city teacher unions to agree on a statewide evaluation system, called APPR,  for the Annual Professional Performance Review, that will rate teachers 20-40% on test scores, and the rest on principal evaluations. Yet nearly one third of all principals in the state have signed onto a letter protesting this system, for reasons that are further explained here and here.  In the city there is even more discord, because the DOE refuses to give teachers the right to appeal a principal's negative rating to an independent arbiter, despite numerous documented cases in which NYC principals have arbitrarily delivered unsatisfactory ratings to teachers for political or personal reasons. Below is a letter from eight esteemed Teachers of the Year, originally posted on the NY State Teachers website, sent to the NY State Board of Regents last spring, pointing out how the proposed APPR is likely to unfairly penalize many excellent professionals, especially those work with at-risk students.  Nevertheless, on Monday, the Regents voted to go full speed ahead with its NCLB waiver application to the US Department of Education, that will further commit them to this damaging evaluation system.

Connecticut has joined the national frenzy. A state panel has just approved a teacher evaluation plan that includes using standardized test scores to decide whether or not to fire a teacher.

First, let's quickly summarize the imprecision associated with the NYC value-added scores, using the raw datasets from the city. It has been heavily reported that the average confidence interval for these estimates - the range within which we can be confident the "true estimate" falls - is 35 percentile points in math and 53 in English Language Arts (ELA). But this oversimplifies the situation somewhat, as the overall average masks quite a bit of variation by data availability.

As New York City parents and teachers struggled Monday to make sense of recently published schoolteacher rankings, education officials considered making future releases illegal to protect a fragile truce on a new statewide system. Legal experts said a series of court rulings have made it increasingly clear that statistics-based portions of teacher evaluations are public information, unlike those of police officers, firefighters and other public workers specifically protected under state law. Only a change in law, experts said, would change that. Shielding teacher rankings from public view is likely to become a new pressure point in the debate over how to measure the effectiveness of teachers, lawmakers and officials said Monday. Merryl Tisch, chancellor of the New York State Board of Regents, which sets education policy, said that while she backs using tests scores to hold teachers accountable, she would support changing state law to hide their rankings from public view.

Believe it or not, I wake up every morning eager to go to work. I never know what's going to happen in my classes, but I invariably look forward to them. My students never fail to surprise me. I feel privileged to introduce newcomers to my language. But now, if they don't pass tests likely designed for English speakers, I face losing my job. This is particularly disturbing because I see patterns, especially among kids who did not actually want to be uprooted, torn away from their friends, family and quite often even their parents. I had several students last year who spoke almost no English, and learned next to nothing the entire year. When I checked their records, I learned that two of them had not only passed junior high English classes (not E.S.L., but regular English), but had also passed Spanish. Without my crystal ball I can only speculate on how they managed this. But it doesn't take a genius to figure out that their value-added scores would not have put me in a favorable light. Under New York State's new paradigm, two years of kids like that would leave me selling pencils on the corner.

The latest serving of data-driven mania from the city Education Department will likely produce screaming headlines about the city's "worst teachers." This virtual wall of shame (and fame) will live online for years to come. But does it actually help parents to find the best schools and teachers? Not really. Here's why. The ratings are based on a complicated formula that compares how much 4th through 8th-grade students have improved on standardized tests compared with how well they were predicted to do. The system tries to take into consideration factors like race, poverty and disabilities. Teachers are then graded on a curve. It's known as "value-added," because it tries show how much value an individual teacher has added to a student's test scores. Here are our top five reasons they won't help and why you won't be seeing them on Insideschools. Please add your own, or tell us why you think they will be useful.

With all this focus on individual teacher performance, I feel like we have missed the major factor in great education, the teaching environment, or context. While complex algorithms supposedly account for differences in student demographics for VAM scores, I am not convinced that these made-up numbers account for the context teachers are placed in and often have very little control over.

Since the reports were released last week, the debate has been raging about whether a formula prone to as much as 53% in margin of error is the best way to judge the effectiveness of teachers. Self-proclaimed reformers say yes; those who understand teaching say otherwise. There is no question that teachers are responsible for the learning and growth that take place inside of their classrooms. However, standardized tests are just not a reliable measure of learning. If we are truly interested in increasing the quality of education, the conversation surrounding accountability must shift. Imagine if doctors were held accountable based on the death rate of their patients, regardless of environmental factors and whether prescribed treatment was followed. Imagine if firefighters were held accountable based on fire injuries and deaths, even though they didn't start the fires, their budgets had been cut and most of the homes in their district didn't have fire alarms. That would be unreasonable. So why do we only apply this impossible standard to teachers?

Here we have the good cop's case for "multiple measures," very neatly made against a counterpoint, the bad cop's use of test scores only. But there is a big problem that remains. The evaluation model we are being offered is driven by several false assumptions.

The standard reformy template is that teachers should only be able to get tenure after 3 years of good ratings in a row and that teachers should be subject to losing tenure if they get 2 bad years in a row.  Further, it is possible that the evaluations might actually stipulate that you can only get a good rating if you achieve a certain rating on the quantitative portion of the evaluation - or the VAM score. Likewise for bad ratings (that is, the quantitative measure overrides all else in the system). The premise of the dice rolling activity from my previous post was that it is necessarily much less likely to roll the same number (or subset of numbers) three times in a row than twice (exponentially in fact). That is, it is much harder to overcome the odds based on error rates to achieve tenure, and much easier to lose it. Again, this is much due to the noisiness of the data, and less due to the difficulty of actually being "good" year after year. The ratings simply jump around a lot. See my previous post.

A recent statement by the New Jersey Department of Education (NJDOE) attempts to provide an empirical justification for that state's focus on the achievement gap - the difference in testing performance between subgroups, usually defined in terms of race or income. Achievement gaps, which receive a great deal of public attention, are very useful in that they demonstrate the differences between student subgroups at any given point in time. This is significant, policy-relevant information, as it tells us something about the inequality of educational outcomes between the groups, which does not come through when looking at overall average scores. Although paying attention to achievement gaps is an important priority, the NJDOE statement on the issue actually speaks directly to the fact, which is well-established and quite obvious, that one must exercise caution when interpreting these gaps, particularly over time, as measures of student performance.

Yesterday, acting education commissioner Chris Cerf tried to quell worries and said he would be against public disclosure of individual teachers' scores. "I don't believe in that," Cerf said in an interview last night. "It is counterproductive, and I believe it is not something we should put out. And especially putting that out in isolation, it's against everything we want to do."

Even people who don't agree with a court's decision to release data rating individual teachers in New York City public schools probably took a peak at the scores for teachers at schools near where they live. How couldn't they? Everyone loves to look at rankings. The problem is: the numbers, like many numbers released by New York City, appear to be incredibly unreliable.

The current narrative for improving education in the United States is based on two undeniable charges and several simple and compelling solutions. The current charges: Despite decades of effort we have failed to substantially mediate the effects of race and class on educational outcomes. Compared to product innovations in the private sector, innovations in the education sector are infrequently dispersed or institutionalized... they don't stick. We've all been there. Just when we think we nearing our destination.... real sustainable learning gains for students... the innovation just seems to slip away. The current solutions: Fire the worst teachers and hire the best, financially reward teachers who are most successful at improving student test scores, spur innovation and improvement through competition led by charter schools, and enact strict controls over schools that fail to demonstrate progress. This solution narrative is powerful and has gained political momentum because it has resonance with many intuitive beliefs. Unfortunately, none of these market-driven strategies are supported by substantial evidence.

Eleven New York City education reporters were huddling on e-mail last October 20, musing over ways to collectively pry a schedule of school closings out of a stubborn press office, when the chatter stopped cold. Word had filtered into their message bins that the city was about to release a set of spreadsheets showing performance scores for 12,000 of the city's 80,000 teachers-names included. Few understood better than the beat reporters that this wonky-sounding database was a game changer.

Despite all the heated talk about how to identify and dismiss low-performing teachers, there's relatively little research on how administrators choose whom to dismiss, whether various dismissal options might actually serve to improve performance, and other aspects in this area. A paper by economist Brian Jacob, released as working paper in 2010 and published late last year in the journal Education Evaluation and Policy Analysis, helps address at least one of these voids, by providing one of the few recent glimpses into administrators' actual (rather than simulated) dismissal decisions. Jacob exploits a change in Chicago Public Schools (CPS) personnel policy that took effect for the 2004-05 school year, one which strengthened principals' ability to dismiss probationary teachers, allowing non-renewal for any reason, with minimal documentation. He was able to link these personnel records to student test scores, teacher and school characteristics and other variables, in order to examine the characteristics that principals might be considering, directly or indirectly, in deciding who would and would not be dismissed. Jacob's findings are intriguing, suggesting a more complicated situation than is sometimes acknowledged in the ongoing debate over teacher dismissal policy.

It is rare that a pundit (and ardent reformer) lays out clearly and crisply the core assumption driving the past thirty years of school reform. It is not only rare but startling when that insider then questions the assumption, suggesting that it is a hunch, not a fact.