Group items tagged

In this paper we consider the challenges involved in evaluating teacher preparation programs when controlling for school contextual bias. Including school fixed effects in the achievement models  used toestimate preparation program effects controls for school environment by relying on differences among student outcomes within the same schools to identify the program effects. However, identification of preparation program effects using school fixed effects requires teachers from different programs to teach in the same school. Even if program effects are identified, the precision of the estimated effects will depend on the degree to which graduates from different programs overlap across schools. In addition, if the connections between preparation programs result from the overlap of atypical graduates or from graduates teaching in atypical school environments, use of school effects could produce bias. Using statewide data from Florida, we show that teachers tend to teach in schools near the programs in which they received their training, but there is still sufficient overlap across schools to identify preparation program effects. We show that the ranking of preparation programs varies significantly depending on whether or not school environment is taken into account via school fixed effects. We find that schools and teachers that are integral to connecting preparation programs are atypical, with disproportionately high percentages of Hispanic teachers and students compared to the state averages. Finally, we  find significant variance inflation in the estimated program effects when controlling for school fixed effects, and that the size of the variance inflation factor depends crucially on the length of the window used to compare graduates teaching in the same schools.

"During the 2010, 2011, and 2012 legislative sessions, a combination of federal policy incentives and newly elected governors and legislative majorities in many states following the 2010 elections sparked a wave of legislation addressing teacher effectiveness. More than 20 states passed legislation designed to address educator effectiveness by mandating annual evaluations based in part on student learning and linking evaluation results to key personnel decisions, including tenure, reductions in force, dismissal of underperforming teachers, and retention. In many cases states passed multiple laws, with later laws building on previous legislation, and also promulgated regulations to implement legislation. A few states acted through regulation only. In an effort to help policymakers, educators, and the public better understand how this flurry of legislative activity shifted the landscape on teacher effectiveness issues-both nationally and at the state level-Bellwether Education Partners analyzed recent teacher effectiveness legislation, regulation, and supporting policy documents from 21 states that took major legislative or regulatory action on teacher effectiveness in the past three years. This analysis builds on a previous analysis of teacher effectiveness legislation in five states that Bellwether published in 2011. Our expanded analysis includes nearly all states that took major legislative action on teacher effectiveness over the past three years."

This paper examines the effect of early childhood investments on college enrollment and degree completion. We use the random assignment in the Project STAR experiment to estimate the effect of smaller classes in primary school on college entry, college choice, and degree completion. We improve on existing work in this area with unusually detailed data on college enrollment spells and the previously unexplored outcome of college degree completion. We find that assignment to a small class increases the probability of attending college by 2.7 percentage points, with effects more than twice as large among blacks. Among those with the lowest ex ante probability of attending college, the effect is 11 percentage points. Smaller classes increase the likelihood of earning a college degree by 1.6 percentage points and shift students towards high-earning fields such as STEM (science, technology, engineering and medicine), business and economics. We confirm the standard finding that test score effects fade out by middle school, but show that test score effects at the time of the experiment are an excellent predictor of long-term improvements in postsecondary outcomes. We compare the costs and impacts of this intervention with other tools for increasing postsecondary attainment, such as Head Start and financial aid, and conclude that early investments are no more cost effective than later investments in boosting adult educational attainment.

We investigate the effectiveness of Idaho charter schools relative to traditional public schools, using the average difference in test score gains in the two sectors as well as the student fixed effects estimator favored in the literature.  Our findings are quite sensitive to the choice of estimator.  When student fixed effects are included, charter schools appear more effective than traditional public schools in the elementary grades.  When student fixed effects are omitted, this is no longer true.  We attribute the difference to biases associated with heterogeneity in schools and in the quality of school-student matches when the fixed effects estimator is used.  We find much less evidence of selection bias, the standard rationale for the fixed effects estimator. 

We develop a framework that may be used to determine the degree to which a school choice program may harm public school stayers by luring the best students to other schools. This framework results in a simple formula showing that the "cream-skimming" effect is increasing in the degree of heterogeneity within schools, the school choice takeup rate of strong students relative to weak students, and the importance of peers. We use the formula to investigate the effects of a voucher program on the high school graduation rate of the students who would remain in public school. We employ NELS:88 data to measure the characteristics of public school students, to estimate a model of the private school entrance decision, and to estimate peer group effects on graduation. We supplement the econometric estimates with a wide range of alternative assumptions about school choice and peer effects. We find that the cream skimming effect is negative but small and that this result is robust across our specifications.

"We use quantile treatment effects estimation to examine the consequences of a school voucher experiment across the distribution of student achievement. In 1997, the School Choice Scholarship Foundation granted $1,400 private school vouchers to a randomly-selected group of low-income New York City elementary school students. Prior research indicates that this program had no average effect on student achievement. If vouchers boost achievement at one part of the distribution and hurt achievement at another, zero or small mean effects may obscure theoretically important but offsetting program effects. Drawing upon prior research related to Catholic schools and school choice, we derive three hypotheses regarding the program's distributional consequences. Our analyses suggest that the program had no significant effect at any point in the skill distribution."

If you follow education research - or quantitative work in any field - you'll often hear the term "significant effect." For example, you will frequently read research papers saying that a given intervention, such as charter school attendance or participation in a tutoring program, had "significant effects," positive or negative, on achievement outcomes. This term by itself is usually sufficient to get people who support the policy in question extremely excited, and to compel them to announce boldly that their policy "works." They're often overinterpreting the results, but there's a good reason for this. The problem is that "significant effect" is a statistical term, and it doesn't always mean what it appears to mean. As most people understand the words, "significant effects" are often neither significant nor necessarily effects. Let's very quickly clear this up, one word at a time, working backwards.

The massive Measures of Effective Teaching Project is finding that teacher effectiveness assessments similar to those used in some district value-added systems aren't good at showing which differences are important between the most and least effective educators, and often totally misunderstand the "messy middle" that most teachers occupy. Yet the project's latest findings suggest more nuanced teacher tests, multiple classroom observations and even student feedback can all create a better picture of what effective teaching looks like. Researchers dug into the latest wave of findings from the study of more than 3,000 classes for a standing-room-only ballroom at the American Educational Research Association's annual conference here on Saturday.

"Our findings are consistent with research that finds models including student background and classroom characteristics are highly correlated with simpler specifications that only include a single-subject lagged test score, while value-added models estimated with school or student fixed effects have a lower correlation. Interestingly, teacher effectiveness estimates based on median student growth percentiles are highly correlated with estimates from VAMs that include only a lagged test score and those that also include lagged scores and student background characteristics, despite the fact that the two methods for estimating teacher effectiveness are, at least conceptually, quite different. However, even when the correlations between job performance estimates generated by different models are quite high, differences in the composition of students in teachers' classrooms can have sizable effects on the differences in their effectiveness estimates."

Ensuring that all students in America's public schools are taught by good teachers is an educational and moral imperative. The teacher is the most important school-based influence on student achievement, and poor children and those of color are less likely to be taught by well-qualified, experienced, and effective teachers than other students. Yet teacher incentive proposals - including those promoted by President Obama's Race to the Top program - are rarely grounded on what high-quality research indicates are the kinds of teacher incentives that lead to school excellence and equity. Few of the current approaches to creating teacher incentives take into account how specific conditions influence whether or not effective teachers will work in high-need schools and will be able to teach effectively in them. This review of research finds little support for a simplistic system of measuring value-added growth, evaluating teachers more "rigorously", and granting bonuses. Instead, the brief supports four recommendations: use the current federal Teacher Incentive Fund to attract qualified, effective teachers to high-needs schools, expand incentives by creating strategic compensation, create working conditions that allow teachers to teach effectively, and more aggressively promote the best practices and policies that spur school excellence and equity. The accompanying legal brief offers legislative language to implement these recommendations.

Unlike in elementary schools, high school teacher effects may be confounded with unobserved track-level treatments (such as the AVID program) that are correlated with individual teachers. I present a strategy that exploits detailed course-taking information to credibly estimate the effects of 9th grade Algebra and English teachers on test scores. I document substantial bias due to track-specific treatments and I show that traditional tests for the existence of teacher effects are flawed. After accounting for bias, I find sizable algebra teacher effects and little evidence of English teacher effects. I find little evidence of teacher spillovers across subjects.

Much has been written about the importance of school leadership, but there is surprisingly little systematic evidence on this topic. This paper presents preliminary estimates of key elements of the market for school principals, employing rich panel data on principals from Texas State. The consideration of teacher movements across schools suggests that principals follow patterns quite similar to those of teachers - preferring schools that have less demands as indicated by higher income students, higher achieving students, and fewer minority students. Looking at the impact of principals on student achievement, the authors find some small but significant effects of the tenure of a principal in a school. More significant, however, are the estimates of variations in principal effectiveness. The variation in principal effectiveness tends to be largest in high poverty schools, consistent with hypothesis that principal ability is most important in schools serving the most disadvantaged students. Finally, considering principal mobility, the authors find that principals who stay in a school tend to be more effective than those who move to other schools.

In the last decade, many cities around the country have needed to close schools due to declining enrollments and low achievement. School closings raise concerns about the possible negative impacts on student achievement, neighborhoods, families, and teaching staff. This study examines an anonymous urban district that, faced with declining enrollment, chose to make student achievement a major criterion in determining which schools would be closed. The district targeted low-performing schools in its closure plan, and sought to move their students to higher-performing schools. We estimate the impact of school closures on student test scores and attendance rates by comparing the growth of these measures among students differentially affected by the closures. We use residential assignment to school as an instrument to address non-random sorting of students into new schools. We also statistically control for the contemporaneous effects of other reforms within the district. Results show that students displaced by school closures can experience adverse effects on test scores and attendance, but these effects can be minimized when students move to schools that are higher-performing (in value-added terms). Moreover, the negative effect on attendance disappears after the first year in the new school. Meanwhile, we find no adverse effects on students in the schools that are receiving the transferring students.

For decades, policymakers have promulgated legislation that requires schools to hire effective teachers in all classrooms. Simultaneously, the education research community has attempted to define what effective teachers do in the classroom. A decade ago, No Child Left Behind provided a framework for defining effective teachers as "highly qualified," which required schools to ensure all of their teachers fit the new standard. This standard, however, is no longer appropriate, as continued evidence indicates that the relationship between credentials and achievement is tenuous. Therefore, policymakers and researchers need to revise the term "highly qualified," and, by utilizing the advances in educational accountability over the previous decade, replace it with a term grounded in practice and directly connected to achievement and effectiveness.

"The Measures of Effective Teaching Project (MET) is the Gates Foundation's flagship effort to fill what they believe is a huge void in the teaching profession. According to them, up until this project, there was no way to know how effective any given teacher is. Their goal has been to develop scientifically accurate means to accomplish this. I would have no problem with the Gates Foundation's Measuring Effective Teaching process if it was conducted as pure research. The MET's Tom Kane, in "Capturing the Dimensions of Effective Teaching," illustrates the good that could have come from the experiment had "reformers" considered evidence before imposing their theories on teachers across the nation."

In the age of accountability, measuring teacher effectiveness has become king. But it's not enough merely to measure effectiveness, according to many leading thinkers and policymakers; personnel decisions-from pay and promotions to layoffs and outright firings-should be based on teacher-effectiveness data, they say.

This article explores student mobility in the Milwaukee Public Schools (MPS) and its effects on student achievement. An urban district with plentiful opportunities for school choice, Milwaukee has a transient student population.  From 2003-04 through 2007-08, 11% of MPS students switched schools or left the district between the fall and spring of a given school year, while 22% were mobile between the spring of one year and the fall of the following year. Using both student fixed-effects and instrumental variables approaches, we examined the effects of this mobility on both the students who moved and their classmates who did not. We found evidence that mobile students' test score gains dropped immediately after they switched schools, but these students typically recovered their losses in the subsequent year. We detected modest but statistically significant negative effects of grade-level turnover on non-mobile students' academic achievement.   

A substantial literature documents large variation in teacher effectiveness at raising student achievement, providing motivation to identify highly effective and ineffective teachers early in their careers. Using data from New York City public schools, we estimate whether subjective evaluations of teacher effectiveness have predictive power for the achievement gains made by teachers' future students. We find that these subjective evaluations have substantial power, comparable with and complementary to objective measures of teacher effectiveness taken from a teacher's first year in the classroom.

A substantial literature documents large variation in teacher effectiveness at raising student achievement, providing motivation to identify highly effective and ineffective teachers early in their careers. Using data from New York City public schools, we estimate whether subjective evaluations of teacher effectiveness have predictive power for the achievement gains made by teachers' future students. We find that these subjective evaluations have substantial power, comparable with and complementary to objective measures of teacher effectiveness taken from a teacher's first year in the classroom.

"Some see us as education's odd couple-one, the president of a democratic teachers' union; the other, a director at the world's largest philanthropy. While we don't agree on everything, we firmly believe that students have a right to effective instruction and that teachers want to do their very best. We believe that one of the most effective ways to strengthen both teaching and learning is to put in place evaluation systems that are not just a stamp of approval or disapproval but a means of improvement. We also agree that in too many places, teacher evaluation procedures are broken-unconstructive, superficial, or otherwise inadequate. And so, for the past four years, we have worked together to help states and districts implement effective teacher development and evaluation systems carefully designed to improve teacher practice and, ultimately, student learning."