Group items tagged

This study examines and compares the recruitment and retention of minority and White elementary and secondary teachers and attempts to empirically ground the debate over minority teacher shortages. The data we analyze are from the National Center for Education Statistics' nationally representative Schools and Staffing Survey and its longitudinal supplement, the Teacher Follow-up Survey.  

After the second year of CPS rolling out TAP, we found no evidence that the program raised student test scores. Student achievement growth as measured by average math and reading scores on the Illinois Standards Achievement Test (ISAT) did not differ significantly between TAP and comparable non-TAP schools.We also found that TAP did not have a detectable impact on rates of teacher retention in the school or district during the second year it was rolled out in the district. We did not find statistically significant differences between TAP and non-TAP retention rates for teachers overall or for subgroups defined by teaching assignment and years of service in CPS. 

Joe Nystrom, who teaches math at a low-income high school here, used to think that only a tiny group of students -- the ''smart kids'' -- were capable of advanced coursework. But two years ago, spurred by a national program that offered cash incentives and other support for students and teachers, Mr. Nystrom's school, South High Community School, adopted a come one, come all policy for Advanced Placement courses. Today Mr. Nystrom teaches A.P. statistics to eight times as many students as he used to, and this year 70 percent of them scored high enough to qualify for college credit, compared with 50 percent before. One in four earned the top score possible, far outpacing their counterparts worldwide.

This report details how charter schools are increasingly run by private, nonprofit management organizations called charter school management organizations (CMOs). The researchers find that most CMOs serve urban students from low-income families, operate small schools that offer more instructional time, and attract teachers loyal to each school's mission, based on survey data and site visits. The authors conducted an impact analysis focused only on middle school grades, finding that a small fraction of CMO-run middle schools boosted achievement growth at notable levels. But on average, student performance in the CMO-run schools did not outpace achievement growth in other charters or in host districts for a statistically matched set of students. This review finds that the report offers an objective assessment of the comparative benefits for middle-school students of a highly select set of CMOs. It also helps to identify organizational features that operate in successful CMO-run schools that are modestly associated with stronger student growth in the middle grades. However, the authors downplay aspects of their methodology that resulted in significant selectivity concerning which CMOs were studied, raising questions regarding the population of charter schools to which they hope to generalize.

I have a lot of reservations about using this study or the previous study to justify the small schools initiative and especially to justify the current massive round of school closings.  I am no expert in statistics, but my concerns revolve around these issues

Look at those numbers above. Think about how many of the Hispanic kids speak Spanish at home. Look at the economic statistics. Think about how this may affect test scores. Charter schools have freedom, all right - the freedom to exclude the most difficult-to-teach students from their rosters. This, apparently, is Christie's great new vision for schools: economic segregation.

With teacher quality repeatedly cited as the most important schooling factor influencing student achievement, there has been increased interest in examining the efficacy of teacher training programs. This paper presents research examining the variation between and impact that individual teacher training institutions in Washington state have on the effectiveness of teachers they train. Using administrative data linking teachers' initial endorsements to student achievement on state reading and math tests, we find the majority of teacher training programs produce teachers who are no more or less effective than teachers who trained out-of-state. However, we do find a number of cases where there are statistically significant differences between estimates of training program effects for teachers who were credentialed at various in-state programs. These findings are robust to a variety of different model specifications.

A new report, commissioned by the District of Columbia Mayor Vincent Gray and conducted by the Chicago-based consulting organization IFF, was supposed to provide guidance on how the District might act and invest strategically in school improvement, including optimizing the distribution of students across schools, many of which are either over- or under-enrolled. Needless to say, this is a monumental task. Not only does it entail the identification of high- and low-performing schools, but plans for improving them as well. Even the most rigorous efforts to achieve these goals, especially in a large city like D.C., would be to some degree speculative and error-prone. This is not a rigorous effort. IFF's final report is polished and attractive, with lovely maps and color-coded tables presenting a lot of summary statistics. But there's no emperor underneath those clothes. The report's data and analysis are so deeply flawed that its (rather non-specific) recommendations should not be taken seriously.

Currently, many states plan to have teachers of non-tested subjects use a make-shift version of value-added measures where teachers identify learning objectives, choose assessments that correspond with these objectives, monitor student progress over the course of the year, and then present that progress as evidence of student growth. While this process may very well improve the overall quality of teaching, it is an ineffective way to evaluate teachers. If it takes complex statistical algorithms to measure student growth for English and math teachers, what makes us think that teachers of non-tested subjects can validly and reliably measure student growth on their own?

A new national study on the effectiveness of the networks that operate charter schools finds that their students' test scores in math, science, and social studies improve after they have spent a year or two at the school, but not by much. Overall, the report out today finds that middle school student achievement varies widely at schools run by charter-management organizations, which are the groups that establish and operate multiple charter schools. Most networks seem to produce a positive effect on student achievement, compared with results for students in district-run schools in the same area that are not run by CMOs. Some actually have a negative effect. The overall impact, however, tends not to be statistically significant, according to the report by from Mathematica and the Center on Reinventing Public Education at the University of Washington.

Any reader of my blog knows already that I'm a skeptic of the usefulness of Value-added models for guiding high stakes decisions regarding personnel in schools. As I've explained on previous occasions, while statistical models of large numbers of data points - like lots of teachers or lots of schools - might provide us with some useful information on the extent of variation in student outcomes across schools or teachers and might reveal for us some useful patterns - it's generally not a useful exercise to try to say anything about any one single point within the data set. Yes, teacher "effectiveness" estimates tend to be based on the many student points across students taught by that teacher, but are still highly unstable. Unstable to the point, where even as a researcher hoping to find value in this information, I've become skeptical.

In our fruitless, deadlocked debate over whether charter schools "work," charter opponents frequently cite the so-called CREDO study (discussed here), a 2009 analysis of charter school performance in 16 states. The results indicated that overall charter effects on student achievement were negative and statistically significant in both math and reading, but both effects sizes were tiny. Given the scope of the study, it's perhaps more appropriate to say that it found wide variation in charter performance within and between states - some charters did better, others did worse and most were no different. On the whole, the size of the aggregate effects, both positive and negative, tended to be rather small. Recently, charter opponents' tendency to cite this paper has been called "cherrypicking." Steve Brill sometimes levels this accusation, as do others. It is supposed to imply that CREDO is an exception - that most of the evidence out there finds positive effects of charter schools relative to comparable regular public schools.

Value-added, or growth, models track individual students' test scores from year to year, which advocates say can help isolate the effect of the instruction that students receive during one school year from their academic backgrounds and prior education experiences. Critics-including many teachers' unions-argue that annual test scores do not give a full picture of student growth, and that the statistical models used for the measures are not designed to evaluate teachers.

In August 2010 the Los Angeles Times published a special report on their website featuring performance ratings for nearly 6,000 Los Angeles Uni!ed School District teachers. The move was controversial because the ratings were based on so-called value-added estimates of teachers' contributions to student learning. These estimates statistically account for the different academic backgrounds children bring to teachers' classes, but they are estimates nonetheless, and opaque ones at that. (see text box below) But the newspaper maximized the controversy - and perhaps the number of hits it drew to web pages with advertising - by attaching teachers' names to the ratings. Parents and other interested members of the public could look up specific teachers in the database and see how they ranked in both math and English, from least effective to most e#ective.

This article explores student mobility in the Milwaukee Public Schools (MPS) and its effects on student achievement. An urban district with plentiful opportunities for school choice, Milwaukee has a transient student population.  From 2003-04 through 2007-08, 11% of MPS students switched schools or left the district between the fall and spring of a given school year, while 22% were mobile between the spring of one year and the fall of the following year. Using both student fixed-effects and instrumental variables approaches, we examined the effects of this mobility on both the students who moved and their classmates who did not. We found evidence that mobile students' test score gains dropped immediately after they switched schools, but these students typically recovered their losses in the subsequent year. We detected modest but statistically significant negative effects of grade-level turnover on non-mobile students' academic achievement.   

Even as research has begun to document that teachers matter, there is less certainty about what attributes of teachers make the most difference in raising student achievement. Numerous studies have estimated the relationship between teachers' characteristics, such as work experience and academic performance, and their value-added to student achievement; but, few have explored whether instructional practices predict student test score gains. In this study, we ask what classroom practices, if any, differentiate teachers with high impact on student achievement in middle school English Language Arts from those with lower impact. In so doing, the study also explores to what extent value-added measures signal differences in instructional quality.  Even with the small sample used in our analysis, we find consistent evidence that high value-added teachers have a different profile of instructional practices than do low value-added teachers. Teachers in the fourth (top) quartile according to value-added scores score higher than second-quartile teachers on all 16 elements of instruction that we measured, and the differences are statistically significant for a subset of practices including explicit strategy instruction.

This report presents data from the School District Finance Survey (F-33) of the Common Core of Data (CCD) survey system for school year (SY) 2008-09 (fiscal year [FY] 2009). The F-33 is a district-level financial survey that consists of data submitted annually to the National Center for Education Statistics (NCES) and the Governments Division of the U.S. Census Bureau (Census) by state education agencies (SEAs) in the 50 states and the District of Columbia.

Now, in studies that threaten to shake the foundation of high-stakes test-based accountability, Mr. Stroup and two other researchers said they believe they have found the reason: a glitch embedded in the DNA of the state exams that, as a result of a statistical method used to assemble them, suggests they are virtually useless at measuring the effects of classroom instruction. Pearson, which has a five-year, $468 million contract to create the state's tests through 2015, uses "item response theory" to devise standardized exams, as other testing companies do. Using I.R.T., developers select questions based on a model that correlates students' ability with the probability that they will get a question right. That produces a test that Mr. Stroup said is more sensitive to how it ranks students than to measuring what they have learned.

"In numerous previous posts I have summarized New Jersey charter school enrollment data, frequently pointing out that the highest performing charter schools in New Jersey tend to be demographically very different from schools in their surrounding neighborhoods and similar grade level schools throughout their host districts or cities. I have tried to explain over and over that the reason these differences are important is because they constrain the scalability of charter schooling as a replicable model of "success." Again, to the extent that charter successes are built on serving vastly different student populations, we can simply never know (even with the best statistical analyses attempting to sort out peer factors, control for attrition, etc.) whether the charter schools themselves, their instructional strategies/models are effective and/or would be effective with larger numbers of more representative students. Here, I take a quick look at the other side of the picture, again focusing on the city of Newark. Specifically, I thought it would be interesting to evaluate the effect on Newark schools enrollment of the shift in students to charter schools, now that charters have taken on a substantial portion of students in the city. If charter enrollments are - as they seem to be - substantively different from district schools enrollments, then as those charter populations grow and remain different from district schools, we can expect the district schools population to change.  In particular, given the demography of charter schools in Newark, we would expect those schools to be leaving behind a district of escalating disadvantage - but still a district serving the vast majority of kids in the city."

In the last decade, many cities around the country have needed to close schools due to declining enrollments and low achievement. School closings raise concerns about the possible negative impacts on student achievement, neighborhoods, families, and teaching staff. This study examines an anonymous urban district that, faced with declining enrollment, chose to make student achievement a major criterion in determining which schools would be closed. The district targeted low-performing schools in its closure plan, and sought to move their students to higher-performing schools. We estimate the impact of school closures on student test scores and attendance rates by comparing the growth of these measures among students differentially affected by the closures. We use residential assignment to school as an instrument to address non-random sorting of students into new schools. We also statistically control for the contemporaneous effects of other reforms within the district. Results show that students displaced by school closures can experience adverse effects on test scores and attendance, but these effects can be minimized when students move to schools that are higher-performing (in value-added terms). Moreover, the negative effect on attendance disappears after the first year in the new school. Meanwhile, we find no adverse effects on students in the schools that are receiving the transferring students.