Group items tagged

There is a widespread consensus among practitioners, researchers, and policy makers that current teacher evaluation systems in most school districts do little to help teachers improve or to support personnel decision making. For this reason, new approaches to teacher evaluation are being developed and tested.  There is also a growing consensus that evidence of teachers' contributions to student learning should be a component of teacher evaluation systems, along with evidence about the quality of teachers' practice. Value-added models (VAMs) for examining gains in student test scores from one year to the next are promoted as tools to accomplish this goal. Policy makers can benefit from research about what these models can and cannot do, as well as from research about the effects of other approaches to teacher evaluation. This background paper addresses both of these important concerns. 

My new book, When Can You Trust the Experts: How to Tell Good Science from Bad in Education is now available. (There's a link for a free download of Chapter 1 on this page.) I wrote the book out of frustration with a particular problem: the word "research" has become meaningless in education. Every product is claimed to be research-based. But we all know that can't be the case. How are teachers and administrators supposed to know which claims are valid?

The use of value-added models in education research has expanded rapidly. These models allow researchers to explore how a wide variety of policies and measured school inputs affect the academic performance of students. An important question is whether such effects are sufficiently large to achieve various policy goals. For example, would hiring teachers having stronger academic backgrounds sufficiently increase test scores for traditionally low-performing students to warrant the increased cost of doing so? Judging whether a change in student achievement is important requires some meaningful point of reference. In certain cases a grade-equivalence scale or some other intuitive and policy relevant metric of educational achievement can be used. However, this is not the case with item response theory (IRT) scale-score measures common to the tests usually employed in value-added analyses. In such cases, researchers typically describe the impacts of various interventions in terms of effect sizes, although conveying the intuition of such a measure to policymakers often is a challenge.   

Scores of professors and researchers from 16 universities throughout the Chicago metropolitan area have signed an open letter to the city's mayor, Rahm Emanuel, and Chicago school officials warning against implementing a teacher evaluation system that is based on standardized test scores. This is the latest protest against "value-added" teacher evaluation models that purport to measure how much "value" a teacher adds to a student's academic progress by using a complicated formula involving a standardized test score. Researchers have repeatedly warned against using these methods, but school reformers have been doing it in state after state anyway. A petition in New York State by principals and others against a test-based evaluation system there has been gaining ground.

Just now the Supreme Court voted 5-4 to uphold the Affordable Care Act, President Obama's signature health-care initiative-including a controversial provision that would require individuals to buy health-care insurance. But what does this provision mean for schools? It could be more connected than you'd think, as research shows health-care disparities help drive achievement gaps among students.

As most people know, one of the big issues in charter school research, common elsewhere as well, is selection effects - the idea that applicants to charter schools are different from non-applicants in terms of unobserved characteristics such as motivation, social networks, family involvement in their education and whether or not they're thriving in their current school. Researchers who wish to isolate the effect of charter schools must address this issue by attempting to control for these differences between students, using variables such as prior achievement, lunch program eligibility and special education classification. When done correctly, this approach can be quite powerful, but it does entail the (unlikely and untestable) assumption that the two groups (treatment and control) do not differ on any observable or unobservable characteristics that might influence the results, at least to some extent.

Los Angeles Unified School District is embroiled in negotiations over teacher evaluations, and will now face pressure from outside the district intended to force counter-productive teacher evaluation methods into use.  Yesterday, I read this  Los Angeles Times article a lawsuit to be filed by an unnamed "group of parents and education advocates."  The article notes that, "The lawsuit was drafted in consultation with EdVoice, a Sacramento-based group. Its board includes arts and education philanthropist Eli Broad, former ambassador Frank Baxter and healthcare company executive Richard Merkin."  While the defendant in the suit is technically LAUSD, the real reason a lawsuit is necessary according to the article is that "United Teachers Los Angeles leaders say tests scores are too unreliable and narrowly focused to use for high-stakes personnel decisions."  Note that, once again, we see a journalist telling us what the unions say and think, without ever, ever bothering to mention why, offering no acknowledgment that the bulk of the research and the three leading organizations for education research and measurement (AERA, NCME, and APA) say the same thing as the union (or rather, the union is saying the same thing as the testing expert).  Upon what research does the other side base arguments in favor of using test scores and "value-added" measurement (VAM) as a legitimate measurement of teacher effectiveness?  They never answer, but the debate somehow continues ad nauseum.  

"The story in the Times Union on Sunday about what some people call a shadow government within the State Education Department didn't get up on the Times Union web site until well after breakfast on Monday. It is available here, and the document below is one of many collected in researching the story. It's a letter to Regents from former Education Commissioner David Steiner explaining the intent of the Regents Research Fund fellowship after a New York State United Teachers executive had alerted members of the board about it. The fellows are now helping Steiner's successor, John B. King Jr."

"Class Size: What Research Says and What It Means for State Policy argues that increasing average class size by one student will save about 2% of total education spending with negligible impact on academic achievement. It justifies this conclusion on the basis that Class-Size Reduction (CSR) is not particularly effective and is not as cost-effective as other reforms. However, this conclusion is based on a misleading review of the CSR research literature. The report puts too much emphasis on studies that are of poor quality or that do not focus on settings that are particularly relevant to the debate on class-size policy in the United States. It argues that class-size reduction is less cost-effective than other reform policies, but it bases this contention on an incomplete accounting of the benefits of smaller classes and an uncritical, unexamined list of alternative policies. The report's estimates of the potential cost savings are flawed as, in reality, schools cannot structurally reduce class size by only one student. Well-documented and long-term non-academic gains from CSR are not addressed. Likewise, the recommendation for releasing the ―least effective‖ teachers assumes a valid way of making such determinations is available. "

The Race to the Top and School Improvement Fund grants are generating a potential goldmine for researchers, with hundreds of schools producing in-depth data about their students, teachers, and the ways they are trying to improve.

As policymakers and school leaders seek new ways to measure and improve teacher effectiveness, it's important for journalists and others to understand what is known about the topic so far, and what remains unsettled or unknown. This research brief does not synthesize all the studies in this highly technical field. But it does aim to improve the accuracy and clarity of reporting by exploring what the research says about timely questions surrounding the complex topic of teacher effectiveness.

If you follow education research - or quantitative work in any field - you'll often hear the term "significant effect." For example, you will frequently read research papers saying that a given intervention, such as charter school attendance or participation in a tutoring program, had "significant effects," positive or negative, on achievement outcomes. This term by itself is usually sufficient to get people who support the policy in question extremely excited, and to compel them to announce boldly that their policy "works." They're often overinterpreting the results, but there's a good reason for this. The problem is that "significant effect" is a statistical term, and it doesn't always mean what it appears to mean. As most people understand the words, "significant effects" are often neither significant nor necessarily effects. Let's very quickly clear this up, one word at a time, working backwards.

"Charter schools that start out doing poorly aren't likely to improve, and charters that are successful from the beginning most often stay that way, according to a new study by researchers at Stanford University. The report, done by Stanford's Center for Research on Education Outcomes (CREDO) and funded by the Robertson Foundation, also found that charter management organizations on average do not do a "dramatically better" job than traditional public schools or charter schools that are individually managed."

"A charter school's performance in its first three years of operation is a solid predictor of the program's long-term chances of success, a new study by Stanford University researchers concludes. On Wednesday Stanford's Center for Research on Education Outcomes (CREDO) published Charter School Growth and Replication, which focuses on what can be learned from the track records of more than 1,300 independently managed public schools and nearly 170 Charter Management Organizations (CMOs). "

There is a dearth of research that supports paying teachers beyond their base salaries to improve student achievement, but there is a broad body of research that indicates that pay for performance might actually do damage as teachers feel a threat to their livelihoods because of this narrow method of measuring their efficacy. Pay for performance has been documented as compromising the good will and cooperation among teachers since it creates competition for a small amount of money, which can result in an "I'm out for myself only" attitude. Such a tone can hurt the necessary collaboration and communication found to nurture student achievement and success.

There's some research that shows charter schools suffer from higher teacher turnover than traditional public schools do. One recent estimate put turnover in charters at 25 percent per year, compared with just 14 percent in traditional public schools. Several explanations have been offered for this attrition. Charter school teachers, for instance, tend to be relatively young, and more susceptible to making quick exits from the profession, some studies suggest. Dissatisfaction with working conditions, and lack of administrative support have also been cited as reasons why charter teachers tend to head for the door. A new paper, based on research as well as a survey of charter school teachers, offers school leaders and charter management organizations advice on how they can keep more charter school teachers in the fold. Released by a Boston nonprofit called Teach Plus, the paper says charter schools can reduce teacher turnover by taking four steps.

In recent years, the casual observer of debates over public education policy might be led to believe that improving teacher quality and ensuring that low income and minority school children have access to high quality teachers has little or nothing to do with the equity or adequacy of financing of schools. The casual observer might be led to believe that there actually exists a sizable body of empirical research that confirms a) that high quality teaches matter, b) that money doesn't matter and c) by extension money has nothing to do with recruiting, retaining or redistributing teacher quality. These arguments, while politically convenient for those hoping to avoid thorny questions of tax policy and state aid formulas, are not actually grounded in any body of decisive, empirical research. Rather, to the contrary, it is reasonably well understood that while teacher quality does indeed matter, teacher wages also matter and teacher working conditions matter, both in terms of the level of quality of the overall teacher workforce and in the distribution of quality teachers.

Fall 2011 Issue of PATHWAYS A magazine on poverty, inequality, and social policy Trends in poverty and inequality: Periodic reports on key poverty and inequality indicators Cutting-edge research: Concise summaries of research that is changing how we understand the sources and consequences of poverty and inequality Bold new visions: Must-read discussions of how labor market, poverty, and inequality policy might be rethought and changed Debates: Leading scholars and policymakers weigh in on the crucial poverty and inequality questions of our time

A federal project designed to help schools "do more with less" relies overwhelmingly on speculative theorizing and other work that falls far short of the high-quality research available in the field, according to a new policy brief published today by the National Education Policy Center (NEPC) at the University of Colorado Boulder. The brief, entitled Productivity Research, the U.S. Department of Education, and High-Quality Evidence, concludes that although the current federal effort is fatally flawed, a well-done project of this type could be helpful. The brief offers recommendations for how the Department could move forward.

America's leaders have frequently invoked the principle that important policy decisions should be evidence-based. This rhetorical embrace, however, has not always prevailed against the appeal of policy ideas with political resonance or other perceived advantages. The following analysis describes a particularly egregious example of this phenomenon: the approach taken by the U.S. Department of Education in its "Increasing Educational Productivity" project. This example illustrates the harm done when leaders fail to ground policy in high-quality research.