Group items tagged

As advocates for public education, The Learning Community has grave concerns about the RI-CAN school report cards that evaluate every Rhode Island public school based on faulty methodology. RI-CAN claims that their report cards "are designed to help families in Rhode Island access online information about their local schools" when in truth the report cards spread misinformation to concerned citizens. Instead of providing access to accurate data, RI-CAN summarizes a school's performance by using only one grade level's achievement on state standardized tests and mathematically incorrect calculations.  No efforts at holding schools accountable will succeed unless the measures used are fair and accurate. It is worth mentioning that we are expressing our strong opposition to the report cards despite the fact that The Learning Community ranked in the Top 10 schools in Rhode Island on 7 of the 14 indicators. The methodological deficiencies of the RI-CAN report cards render them at best useless and, at worst, harmful to our state's efforts to support the education of every child.

"Most of the attention to the methodological shortcomings of the law focuses on "adequate yearly progress" (AYP) - the crude requirement that all schools must make "adequate progress" toward the goal of 100 percent proficiency by 2014. And AYP is indeed an inept measure. But the problems are actually much deeper than AYP. Rather, it's the underlying methods and assumptions of NCLB (including AYP) that have had a persistent, negative impact on the way we interpret testing data. I'm not going to get into all of the details here, since I discuss them so frequently (follow the links below), but the most important issues include:"

"The Carnegie Knowledge Network seeks to provide education policymakers and practitioners with timely, authoritative research and information on the use of value-added methodologies and other metrics in teacher evaluation systems."

Reviewers Guarino and Stacy question the emphasis placed on validating classroom observations with test score gains. Observation scores may pick up different aspects of teacher quality than do test-based measures. It is possible that neither type of measure used in isolation captures a teacher's contribution to all the useful skills that students learn in schools. From this standpoint, the authors' conclusion that multiple measures of teacher effectiveness are needed is justifiable. The omission of relevant information is a shortcoming of the report. Key details regarding the study design and methodological approach are lacking.

Last Monday, a broad-based coalition of Providence community groups sent a letter to Governor Chafee announcing their opposition to the implantation of a network of Achievement First Mayoral Academies in Providence. It's a pretty long letter, but it's effectively summed upin the last paragraph, which reads: "Our repudiation of Achievement First is not an affirmation of the status quo nor is it a condemnation of all charter schools. The persistent achievement gaps that exist in our schools must be addressed, but no organization or methodology should claim to close those gaps while posting mixed academic results and undermining democratic processes. We implore your help in creating education policies, developed in conjunction with parents, teachers, students, and other local stakeholders, that help all young people enrolled in our public schools."

Unless you live under a rock (or if you don't track education policy matters through social media - same thing), you know about the "big" study about teacher effects that was conducted by Chetty, Friedman and Rockoff, disseminated through NBER and reported in multiple outlets, most notably the New York Times. This is an important study for at least a couple of reasons. First, methodologically, the study is massive and novel in some important ways. Second, from a policy perspective, even if the authors overreach in their interpretation, the study adds to the growing body of literature on teacher effectiveness and value-added measures. The more empirical evidence we have, the better; that's the nature of scientific research.

Kristof buries the fact that the study is based on value-added methodology and conflates student performance on test scores with good teaching. He alludes to value-added in the 11th paragraph, but never actually addresses the fact that test scores and value added analysis aren't infallible. The study authors (and this piece shouldn't detract from the worth and value of the study, which merits a read) are clear on this, even though Kristof is not.

This report details how charter schools are increasingly run by private, nonprofit management organizations called charter school management organizations (CMOs). The researchers find that most CMOs serve urban students from low-income families, operate small schools that offer more instructional time, and attract teachers loyal to each school's mission, based on survey data and site visits. The authors conducted an impact analysis focused only on middle school grades, finding that a small fraction of CMO-run middle schools boosted achievement growth at notable levels. But on average, student performance in the CMO-run schools did not outpace achievement growth in other charters or in host districts for a statistically matched set of students. This review finds that the report offers an objective assessment of the comparative benefits for middle-school students of a highly select set of CMOs. It also helps to identify organizational features that operate in successful CMO-run schools that are modestly associated with stronger student growth in the middle grades. However, the authors downplay aspects of their methodology that resulted in significant selectivity concerning which CMOs were studied, raising questions regarding the population of charter schools to which they hope to generalize.

ALEC's 17th edition of the Report Card on American Education contains a comprehensive overview of educational achievement levels (performance and gains for low-income students) for the 50 states and the District of Columbia (see full report for complete methodology). The Report Card details what education policies states currently have in place and provides a roadmap for legislators to follow to bring about educational excellence in their state.

For the second time, the Los Angeles Times published results of statistical testing examining the variation in teacher and school performance in the LA Unified School District. The resulting ranking system was found to be inaccurate due to the unreliable methodology.

"The problems with the index go beyond the criticism that is natural for school rankings of any kind: That ranking methodology is subjective, that there is no such thing as "best" when it comes to education. (Jay, I should say, doesn't use that word to describe his rankings, but what do you think people take away from them?) "

State Education Commissioner John B. King, Jr. today announced that American Institutes for Research (AIR) has been awarded a contract to develop methodologies and measures for the student growth component of the State's new teacher and principal evaluation system.  The goal, according to Commissioner King, is to ensure New York has a state-of-the-art approach to developing fair and reliable assessments of educators' contributions to their students' growth in learning.

Incomplete: How Middle Class Schools Aren't Making the Grade is a new report from Third Way, a Washington, D.C.-based policy think tank. The report aims to convince parents, taxpayers and policymakers that they should be as concerned about middle-class schools not making the grade as they are about the failures of the nation's large, poor, urban school districts. But, the report suffers from egregious methodological flaws invalidating nearly every bold conclusion drawn by its authors. First, the report classifies as middle class any school or district where the share of children qualifying for free or reduced-priced lunch falls between 25% and 75%. Seemingly unknown to the authors, this classification includes as middle class some of the poorest urban centers in the country, such as Detroit and Philadelphia. But, even setting aside the crude classification of middle class, none of the report's major conclusions are actually supported by the data tables provided. The report concludes, for instance, that middle-class schools perform much less well than the general public, parents and taxpayers believe they do. But, the tables throughout the report invariably show that the schools they classify as "middle class" fall precisely where one would expect them to-in the middle-between higher- and lower-income schools. 

V. Darleen Opfer has six months under her belt as head of RAND Corp.'s education division, and she's pushing to make sure the education research giant's studies actually make a difference in the field. Opfer, who replaced former director Susan J. Bodilly, said RAND is moving its focus from "pure research" to collaborating with districts and state education agencies. The group has expanded its research reviewers to beyond other researchers to gauge whether a study's methodology is sound, but also to include policymakers and practitioners to weigh in on whether and how a study's results could be relevant.

The Think Tank Review Project provides the public, policy makers, and the press with timely, academically sound reviews of selected think-tank publications. Reviewers for the Think Tank Review Project apply academic peer review standards to reports from think tanks and write brief reviews for the project web site. They are asked to examine the reports for the validity of assumptions, methodology, results, and strength of links between results and policy recommendations. The reviews, written in non-academic language, are intended to help policy makers, reporters, and others assess the merits of the reviewed reports.

Public School 30 and Public School 179 are about as alike as two schools can be. They are two blocks apart in the South Bronx. Both are 98 percent black and Latino. At P.S. 30, 97 percent of the children qualify for subsidized lunches; at P.S. 179, 93 percent. During city quality reviews - when Education Department officials make on-site inspections - both scored "proficient." The two have received identical grades for "school environment," a rating that includes attendance and a survey of parents', teachers' and students' opinions of a school. On the state math test, P.S. 30 did better in 2011, with 41 percent of students scoring proficient - a 3 or 4 - versus 29 percent for P.S. 179. But on the state English test, P.S. 179 did better, with 36 percent of its students scoring proficient compared with 32 percent for P.S. 30. And yet, when the department calculated the most recent progress report grades, P.S. 30 received an A. And P.S. 179 received an F.

"Earlier this summer, the New York City Independent Budget Office (IBO) presented findings from a longitudinal analysis of NYC student performance. That is, they followed a cohort of over 45,000 students from third grade in 2005-06 through 2009-10 (though most results are 2005-06 to 2008-09, since the state changed its definition of proficiency in 2009-10). The IBO then simply calculated the proportion of these students who improved, declined or stayed the same in terms of the state's cutpoint-based categories (e.g., Level 1 ["below basic" in NCLB parlance], Level 2 [basic], Level 3 [proficient], Level 4 [advanced]), with additional breakdowns by subgroup and other variables. The short version of the results is that almost two-thirds of these students remained constant in their performance level over this time period - for instance, students who scored at Level 2 (basic) in third grade in 2006 tended to stay at that level through 2009; students at the "proficient" level remained there, and so on. About 30 percent increased a category over that time (e.g., going from Level 1 to Level 2). The response from the NYC Department of Education (NYCDOE) was somewhat remarkable. It takes a minute to explain why, so bear with me."

"When given the opportunity, many teachers choose to leave schools serving poor, low-performing, and nonwhite students. While a substantial research literature has documented this phenomenon, far less research effort has gone into understanding what features of the working conditions in these schools drive this relatively higher turnover rate. This paper explores the relationship between school contextual factors and teacher retention decisions in New York City. The methodological approach separates the effects of teacher characteristics from school characteristics by modeling the relationship between the assessments of school contextual factors by one set of teachers and the turnover decisions by other teachers within the same school. Teachers' perceptions of the school administration have by far the greatest influence on teacher-retention decisions. This effect of administration is consistent for first-year teachers and the full sample of teachers and is confirmed by a survey of teachers who have recently left teaching in New York City."