Group items tagged

Educational interventions are often evaluated and compared on the basis of their impacts on test scores. Decades of research have produced two empirical regularities: interventions in later grades tend to have smaller effects than the same interventions in earlier grades, and the test score impacts of early educational interventions almost universally "fade out" over time. This paper explores whether these empirical regularities are an artifact of the common practice of rescaling test scores in terms of a student's position in a widening distribution of knowledge. If a standard deviation in test scores in later grades translates into a larger difference in knowledge, an intervention's effect on normalized test scores may fall even as its effect on knowledge does not. We evaluate this hypothesis by fitting a model of education production to correlations in test scores across grades and with college-going using both administrative and survey data. Our results imply that the variance in knowledge does indeed rise as children progress through school, but not enough for test score normalization to fully explain these empirical regularities.

John Mooney at NJ Spotlight provided a reasonable overview of the NJDOE waiver proposal to "reward" successful schools and sanction and/or takeover "failing" ones. The NJDOE waiver proposal includes explanation of a new classification system for identifying which schools should be subject to state intervention, ultimately to be managed by regional offices throughout the state. This new targeted intervention system classifies districts in need of intervention as "priority" districts, with specific emphasis on "focus" districts.

Response to intervention, or RTI, is a strategy that involves identifying students' learning problems quickly and using a series of focused lessons, or interventions, to address those problems before they become entrenched. The intensity of the interventions increase if a student doesn't respond. In this survey, full implementation of RTI involved universal screening of students at least three times a year, the use of clear decision rules to move students between tiers of instruction, and regular monitoring of students' progress based on their learning needs.

This report carefully reviews high-quality empirical evidence from the last several years on the test score effects of three approaches to modifying the organization of schools: (1) starting schools later in the morning, (2) favoring K-8 grade configuration instead of junior high or middle school configurations, and (3) increasing teacher specialization by grade and subject. It estimates the earnings benefits of each intervention and, for interventions (1) and (2), compares monetary benefits to costs. The report concludes that benefits outweigh costs, although the rough cost estimates suggest that better data are required to draw definite conclusions. The report's main conclusion is that organizational reforms deserve a more prominent place in education debates, and that individual school districts should carefully consider them alongside more popular reform options. The review points to a few shortcomings but concludes that the report's analyses are solid and helpful and that the results are presented carefully and cautiously.

Seven years after the Individuals with Disabilities Education Act was renewed with a provision allowing response to intervention to be used when deciding if a child has a specific learning disability, a new study shows 71 percent of school districts use the strategy in at least one school.

In April 2011, Spectrum K12 School Solutions and leading education organizations including AASA, CASE, NASDSE and the RTI Action Network/NCLD conducted a web-based survey of K-12 district administrators to gauge the extent to which Response to Intervention (RTI) has been adopted.

The American Recovery and Reinvestment Act of 2009 (ARRA) targeted substantial School Improvement Grants (SIGs) to the nation's "persistently lowest achieving" public schools (i.e., up to $2 million per school annually over 3 years) but required schools accepting these awards to implement a federally prescribed school-reform model. Schools that met the "lowest-achieving" and "lack of progress" thresholds within their state had prioritized eligibility for these SIG-funded interventions. Using data from California, this study leverages these two discontinuous eligibility rules to identify the effects of SIG-funded whole-school reforms. The results based on these "fuzzy" regression-discontinuity designs indicate that there were significant improvements in the test-based performance of schools on the "lowest-achieving" margin but not among schools on the "lack of progress" margin. Complementary panel-based estimates suggest that these improvements were largely concentrated among schools adopting the federal "turnaround" model, which compels more dramatic staff turnover.

Slides: Public Hearing on the Implementation of Race to the Top and Federal School Intervention Models in New York City

In recent years, interest has grown in using classroom observation as a means to several ends, including teacher development, teacher evaluation, and impact evaluation of classroom-based interventions. Although education practitioners and researchers have developed numerous observational instruments for these purposes, many developers fail to specify important criteria regarding instrument use. In this article, the authors argue that for classroom observation to succeed in its aims, improved observational systems must be developed. These systems should include not only observational instruments but also scoring designs capable of producing reliable and cost-efficient scores and processes for rater recruitment, training, and certification. To illustrate how such a system might be developed and improved, the authors provide an empirical example that applies generalizability theory to data from a mathematics observational instrument.

Randomized trials are a common way to provide rigorous evidence on the impacts of education programs. This article discusses the trade-offs associated with study designs that involve random assignment of students within schools and describes the experience from one such study of Teach for America (TFA). The TFA experiment faced challenges with recruitment, randomization of students, and analysis. The solutions to those challenges may be instructive for experimenters who wish to study future interventions at the student or classroom level. The article concludes that within-school random assignment studies such as the TFA evaluation are challenging but, under the right conditions, are also feasible and potentially very rewarding in terms of generating useful evidence for policy.

The NJ Department of Education (NJDOE) under Acting Commissioner Christopher Cerf is gearing up to intervene in 75 predominately Black and Latino "Priority" Schools, action that could lead to massive school closings within three years. The schools targeted by NJDOE for closure are in very poor neighborhoods across the state and have served these communities for decades. The NJDOE plan for "aggressive intervention" and potential school closures is the centerpiece of a new "accountability" initiative launched by the Christie Administration after obtaining a U.S. Department of Education waiver from certain provisions of the federal No Child Left Behind Act (NCLB) in 2011. The waiver allows NJDOE to use test scores and graduation rates to create three new classifications of schools: "Priority," "Focus" and "Reward."

"Most research on educational interventions, including school vouchers, focuses on impacts on short-term outcomes such as students' scores on standardized tests. Few studies are able to track longer-term outcomes, and even fewer are able to do so in the context of a randomized experiment. In the first study using a randomized experiment to measure the impact of school vouchers on college enrollment, we examine the college-going behavior through 2011 of students who participated in a voucher experiment as elementary school students in the late 1990s. We find no overall impacts on college enrollments but we do find large, statistically significant positive impacts on the college going of African American students who participated in the study. Our estimates indicate that using a voucher to attend private school increased the overall college enrollment rate among African Americans by 24 percent."

"I have reviewed, albeit superficially, the test-based components of several states' school rating systems (e.g., OH, FL, NYC, LA, CO), with a particular focus on the degree to which they are actually measuring student performance (how highly students score), rather than school effectiveness per se (whether students are making progress). Both types of measures have a role to play in accountability systems, even if they are often confused or conflated, resulting in widespread misinterpretation of what the final ratings actually mean, and many state systems' failure to tailor interventions to the indicators being used. One aspect of these systems that I rarely discuss is the possibility that the ratings systems are an end in themselves."

"New York state published a list of schools based on measures like test scores and graduation rates. At the top are "reward" schools. At the bottom are "priority" schools. This is the amazing discovery. The schools that enroll mostly white and Asian students in affluent neighborhoods are doing a great job; they get a reward. The schools that enroll mostly black and Hispanic students in poor neighborhoods are doing a bad job; they are in line to get sanctions, interventions."

Gov. Dannel P. Malloy today outlined six broad principles that he says will guide the debate on education reform next year, including "intensive interventions" by the state in troubled school systems and a lighter bureaucratic touch at successful ones. In a two-page letter addressed to legislators and stakeholders, Malloy hinted at a willingness to take up the politically charged issue of tenure and pay reform, saying teachers and principals should be valued for "skill and effectiveness" over "seniority and tenure."

he Project on Incentives in Teaching (POINT) was a three-year study conducted in the Metropolitan Nashville School System from 2006-07 through 2008-09, in which middle school mathematics teachers voluntarily participated in a controlled experiment to assess the effect of financial rewards for teachers whose students showed unusually large gains on standardized tests. The experiment was intended to test the notion that rewarding teachers for improved scores would cause scores to rise. It was up to participating teachers to decide what, if anything, they needed to do to raise student performance: participate in more professional development, seek coaching, collaborate with other teachers, or simply reflect on their practices. Thus, POINT was focused on the notion that a significant problem in American education is the absence of appropriate incentives, and that correcting the incentive structure would, in and of itself, constitute an efective intervention that improved student outcomes.By and large, results did not confirm this hypothesis

Contrary to the canned rhetoric flying around public discourse on education finance, high-quality research like that discussed in Baker's review does not lend itself to broad, sweeping conclusions. Some things work and others don't, and so the strength and consistency of the money/results relationship varies by how it's spent, the students on whom it spent, and other factors. Sometimes effects are small, and sometimes they're larger. Nevertheless, on the whole, Baker's review shows that there is a consistently positive effect of higher spending on achievement. Moreover, interventions that cost money, such as higher teacher salaries, have a proven track record of getting results, while state-level policies to increase the adequacy and equitability of school finance have also been shown to improve the level and distribution of student performance. Finally, and most relevant to the current budget context, the common argument that we can reduce education funding without any harm to (and, some argue, actual improvement of) achievement outcomes has no basis in empirical evidence.

The UFT and the Department of Education have been in intensive negotiations for the past two months over the details of a new teacher evaluation system for schools designated for the "restart" and "transformation" federal intervention models only. With a Dec. 31 deadline looming for finalizing an agreement, both sides are meeting in subcommittees and going back and forth on key issues.

There may be a mini-trend emerging in certain types of charter school analyses, one that seems a bit trivial but has interesting implications that bear on the debate about charter schools in general. It pertains to how charter effects are presented. Usually, when researchers estimate the effect of some intervention, the main finding is the overall impact, perhaps accompanied by a breakdown by subgroups and supplemental analyses. In the case of charter schools, this would be the estimated overall difference in performance (usually testing gains) between students attending charters versus their counterparts in comparable regular public schools. Two relatively recent charter school reports, however - both generally well-done given their scope and available data - have taken a somewhat different approach, at least in the "public roll-out" of their results.

After the release of high school progress reports last week, city Education Department officials identified 21 struggling high schools and secondary schools that, because of their low performance, could be closed or subject to other interventions beginning next year.