Group items tagged

Despite all the heated talk about how to identify and dismiss low-performing teachers, there's relatively little research on how administrators choose whom to dismiss, whether various dismissal options might actually serve to improve performance, and other aspects in this area. A paper by economist Brian Jacob, released as working paper in 2010 and published late last year in the journal Education Evaluation and Policy Analysis, helps address at least one of these voids, by providing one of the few recent glimpses into administrators' actual (rather than simulated) dismissal decisions. Jacob exploits a change in Chicago Public Schools (CPS) personnel policy that took effect for the 2004-05 school year, one which strengthened principals' ability to dismiss probationary teachers, allowing non-renewal for any reason, with minimal documentation. He was able to link these personnel records to student test scores, teacher and school characteristics and other variables, in order to examine the characteristics that principals might be considering, directly or indirectly, in deciding who would and would not be dismissed. Jacob's findings are intriguing, suggesting a more complicated situation than is sometimes acknowledged in the ongoing debate over teacher dismissal policy.

The specifics of how growth models should be constructed and used to evaluate schools and teachers is a topic of lively policy debate in states and school districts nationwide. In this paper we take up the question of model choice and examine three competing approaches. The first approach, reflected in the popular student growth percentiles (SGPs) framework, eschews all controls for student covariates and schooling environments. The second approach, typically associated with value-added models (VAMs), controls for student background characteristics and aims to identify the causal effects of schools and teachers. The third approach, also VAM-based, fully levels the playing field so that the correlation between school- and teacher-level growth measures and student demographics is essentially zero. We argue that the third approach is the most desirable for use in educational evaluation systems. Our case rests on personnel economics, incentive-design theory, and the potential role that growth measures can play in improving instruction in K-12 schools

Over the objections of teachers' unions and many Democrats, Louisiana's Republican governor and GOP-controlled legislature have crafted one of the most exhaustive education overhauls of any state in the country, through measures that will dramatically expand families' access to public money to cover the costs of both private school tuition and individual courses offered by a menu of providers. A pair of bills championed by Gov. Bobby Jindal, which he is expected to sign into law, will expand a state-run private-school-voucher program beyond New Orleans to other academically struggling schools around the state, give superintendents and principals direct control over personnel decisions, and set much higher standards for awarding teachers tenure.

Gates made a splash back in February when he came out against making Teacher Data Reports - or evaluations - public in New York City. Los Angeles Public Schools released similar data. This is a big deal, because his foundation has advocated for tougher accountability standards for teachers, something teachers unions haven't fully embraced. In an interview with Weekend Edition Saturday's host Scott Simon, Gates explained himself. "The goal is to help teachers be better," Gates said. "And when we run personnel systems where we want to be frank with employees about where they need to improve, having [evaluations] publicly available is not conducive to openness and a free exchange of views."

Any number of educators-principals, personnel directors, superintendents-can be called upon to evaluate teachers. But one school district in Indiana, Anderson, has decided that another group has perhaps the best expertise to judge quality teaching: other teachers. This type of peer review is catching on nationally but is rare in Indiana. That might soon change.

In a series of recent blog posts and in a forthcoming article I have discussed the potential problems with using bad, versus entirely inappropriate measures for determining teacher effectiveness.  I have pointed out, for example, that using value-added measures to estimate teacher effectiveness and then determine whether a teacher should be denied tenure, or have their tenure removed might raise due process concerns which arise from the imprecision and potential outright inaccuracy of teacher effectiveness estimates derived from such methods. I have also explained that in some states like New Jersey, which have adopted Student Growth Percentile measures as an evaluation tool, that where those measures are used as a basis for dismissing teachers, teachers (or their attorney's) might simply rely on the language of the authors of those methods to point out that they are not designed to, nor were they intended to attribute responsibility for the measured student growth to the teacher. Where attribution of responsibility is off the table the dismissing a teacher on an assumption of ineffectiveness based on these measures is entirely inappropriate, and a potential violation of the teacher's due process rights. But, the problem is that state legislatures are increasingly mandating that these measures absolutely be used when making high stakes personnel decisions.

Delegates to the National Education Association's Representative Assembly knew the news about their union's loss of membership would be bad, but it isn't clear that they knew it would be this bad. NEA officials said the union has lost more than 100,000 teachers and education support personnel since 2010, and it projects that it will lose even more in the future. By the end of its 2013-14 budget, NEA expects it will have lost 308,000 members and experienced a decline in revenue projected at some $65 million in all since 2010. (The figures are expressed in full-time equivalents, which means that the actual number of people affected is probably higher.)

"This post is about rolling out some of the left over data I have from my various endeavors this summer.  These data include data from New York State personnel master files (PMFs) linked to New York City public schools and charter schools, NYC teacher value-added scores, and various bits of data on New York City charter and district schools including school site budget/annual financial report information. Here, I use these data combined with some of my previous stuff, to take a first, cursory shot at characterizing the teaching workforce of charter school teachers in New York City. All findings use data from 2008 to 2010."

Faced with mounting policy pressures from federal and state accountability programs, school leaders are reallocating curricula, time, even diet in an attempt to boost student achievement. To explore whether they are using test score data to reallocate their teacher resources as well, I designed a cross-case, cross-sectional study and explored principals' reported staffing practices in one higher performing and one lower performing elementary school in each of five Florida school districts. Findings show that school leaders are "staffing to the test" by hiring, moving, and developing teachers in an effort to increase their schools' overall performance. The paper discusses the implications of evidence-based staffing for policy, practice and future research.

In the age of accountability, measuring teacher effectiveness has become king. But it's not enough merely to measure effectiveness, according to many leading thinkers and policymakers; personnel decisions-from pay and promotions to layoffs and outright firings-should be based on teacher-effectiveness data, they say.

Alexander Russo chose to portray corporate reform critics such as myself as Goliaths who are trampling on the hapless reformers. But this analysis is a bit simple-minded. The corporate reformers have plenty of resources and personnel capable of responding. They are deliberately choosing to take their arguments elsewhere - to the corporate boardrooms, to the ALEC conference, to NBC's Education Nation, and to legislative hearings, speaking through hired lobbyists, astro-turf groups, and well-prepared and vetted experts. They are getting the job done there, if you notice. Most of these groups are seeing revenues climb, and state legislatures across the country are busy adopting more "reform" laws every month.

There is a widespread consensus among practitioners, researchers, and policy makers that current teacher evaluation systems in most school districts do little to help teachers improve or to support personnel decision making. For this reason, new approaches to teacher evaluation are being developed and tested.  There is also a growing consensus that evidence of teachers' contributions to student learning should be a component of teacher evaluation systems, along with evidence about the quality of teachers' practice. Value-added models (VAMs) for examining gains in student test scores from one year to the next are promoted as tools to accomplish this goal. Policy makers can benefit from research about what these models can and cannot do, as well as from research about the effects of other approaches to teacher evaluation. This background paper addresses both of these important concerns. 

In a provocative and influential paper, Jesse Rothstein (2010) finds that standard value-added models (VAMs) suggest implausible future teacher effects on past student achievement, a finding that obviously cannot be viewed as causal. This is the basis of a falsification test (the Rothstein falsification test) that appears to indicate bias in VAM estimates of current teacher contributions to student learning. More precisely, the falsification test is designed to identify whether or not students are effectively randomly assigned conditional on the covariates included in the model. Rothstein's finding is significant because there is considerable interest in using VAM teacher effect estimates for high-stakes teacher personnel policies, and the results of the Rothstein test cast considerable doubt on the notion that VAMs can be used fairly for this purpose. However, in this paper, we illustrate-theoretically and through simulations-plausible conditions under which the Rothstein falsification test rejects VAMs even when students are randomly assigned, conditional on the covariates in the model, and even when there is no bias in estimated teacher effects.

Los Angeles Unified School District is embroiled in negotiations over teacher evaluations, and will now face pressure from outside the district intended to force counter-productive teacher evaluation methods into use.  Yesterday, I read this  Los Angeles Times article a lawsuit to be filed by an unnamed "group of parents and education advocates."  The article notes that, "The lawsuit was drafted in consultation with EdVoice, a Sacramento-based group. Its board includes arts and education philanthropist Eli Broad, former ambassador Frank Baxter and healthcare company executive Richard Merkin."  While the defendant in the suit is technically LAUSD, the real reason a lawsuit is necessary according to the article is that "United Teachers Los Angeles leaders say tests scores are too unreliable and narrowly focused to use for high-stakes personnel decisions."  Note that, once again, we see a journalist telling us what the unions say and think, without ever, ever bothering to mention why, offering no acknowledgment that the bulk of the research and the three leading organizations for education research and measurement (AERA, NCME, and APA) say the same thing as the union (or rather, the union is saying the same thing as the testing expert).  Upon what research does the other side base arguments in favor of using test scores and "value-added" measurement (VAM) as a legitimate measurement of teacher effectiveness?  They never answer, but the debate somehow continues ad nauseum.  

In Ed Schools housed within research universities, and in programs in educational leadership which are primarily charged with the training of school and district level leaders, we are constantly confronted with deliberations over how to balance teaching the "practical stuff" and "how to" information on running a school or school district, managing personnel, managing budgets, etc. etc. etc., and the "research stuff" like understanding how to interpret rigorous research in education and related social sciences (increasingly economic research).  Finding the right balance between theory, research and practice is an ongoing struggle and often the subject of bitter debate in professional programs housed in research universities.

Any reader of my blog knows already that I'm a skeptic of the usefulness of Value-added models for guiding high stakes decisions regarding personnel in schools. As I've explained on previous occasions, while statistical models of large numbers of data points - like lots of teachers or lots of schools - might provide us with some useful information on the extent of variation in student outcomes across schools or teachers and might reveal for us some useful patterns - it's generally not a useful exercise to try to say anything about any one single point within the data set. Yes, teacher "effectiveness" estimates tend to be based on the many student points across students taught by that teacher, but are still highly unstable. Unstable to the point, where even as a researcher hoping to find value in this information, I've become skeptical.

For the past couple of months, Steve Brill's new book has served to step up the eternally-beneath-the-surface hypothesis that teachers' unions are the primary obstacle to improving educational outcomes in the U.S. The general idea is that unions block "needed reforms," such as merit pay and other forms of test-based accountability for teachers, and that they "protect bad teachers" from being fired. Teachers' unions are a convenient target. For one thing, a significant proportion of Americans aren't crazy about unions of any type. Moreover, portraying unions as the villain in the education reform drama facilitates the (mostly false) distinction between teachers and the organizations that represent them - put simply, "love teachers, hate their unions." Under the auspices of this dichotomy, people can advocate for changes , such as teacher-level personnel policies based partially on testing results, without having to address why most teachers oppose them (a badly needed conversation).

"The question of what to do with bad teachers has stymied America's education system of late, sparking chaotic protests in state capitals and vitriolic debate in a recent congressional hearing. It has also stoked the movement known as 'education reform,' which has zeroed in on teacher quality by urging school districts to sort the star teachers from the duds, and reward or punish them accordingly. The idea is that America's schools would be able to increase their students' test scores if only they had better teachers. Since 2007, this wave of education reformers -- in particular Democrats for Education Reform, a group backed by President Barack Obama and hedge fund donors -- has clashed with teachers unions in their pursuit of making the field of education as discerning in its personnel choices as, say, that of finance. Good teachers should be promoted and retained, reformers contend, instead of being treated like identical pieces on an assembly line, who are rewarded with tenure for their staying power or seniority. But what to do with the underperformers?"

Every two years, the release of data from the National Assessment of Educational Progress (NAEP) generates a wave of research and commentary trying to explain short- and long-term trends. For instance, there have been a bunch of recent attempts to "explain" an increase in aggregate NAEP scores during the late 1990s and 2000s. Some analyses postulate that the accountability provisions of NCLB were responsible, while more recent arguments have focused on the "effect" (or lack thereof) of newer market-based reforms - for example, looking to NAEP data to "prove" or "disprove" the idea that changes in teacher personnel and other policies have (or have not) generated "gains" in student test scores. The basic idea here is that, for every increase or decrease in cross-sectional NAEP scores over a given period of time (both for all students and especially for subgroups such as minority and low-income students), there must be "something" in our education system that explains it. In many (but not all) cases, these discussions consist of little more than speculation.

In a provocative and influential paper, Jesse Rothstein (2010) finds that standard value-added models (VAMs) suggest implausible future teacher effects on past student achievement, a finding that obviously cannot be viewed as causal. This is the basis of a falsification test (the Rothstein falsification test) that appears to indicate bias in VAM estimates of current teacher contributions to student learning. Rothstein's finding is significant because there is considerable interest in using VAM teacher effect estimates for high-stakes teacher personnel policies, and the results of the Rothstein test cast considerable doubt on the notion that VAMs can be used fairly for this purpose. However, in this paper, we illustrate-theoretically and through simulations-plausible conditions under which the Rothstein falsification test rejects VAMs even when there is no bias in estimated teacher effects, and even when students are randomly assigned conditional on the covariates in the model. On the whole, our findings show that the "Rothstein falsification test" is not definitive in showing bias, which suggests a much more encouraging picture for those wishing to use VAM teacher effect estimates for policy purposes.