Group items tagged

I don't think there's any way to avoid publication, given that about a dozen newspapers will receive the data, and it's unlikely that every one of them will decline to do so. So, in addition to expressing my firm opposition, I would offer what I consider to be an absolutely necessary suggestion: If newspapers are going to publish the estimates, they need to publish the error margins too. Value-added and other growth model scores are statistical estimates, and must be interpreted as such. Imagine that a political poll found that a politician's approval rate was 40 percent, but, due to an unusually small sample of respondents, the error margin on this estimate was plus or minus 20 percentage points. Based on these results, the approval rate might actually be abysmal (20 percent), or it might be pretty good (60 percent). Should a newspaper publish the 40 percent result without mentioning that level of imprecision? Of course not. In fact, they should refuse to publish the result at all. Value-added estimates are no different.

What many Americans do not realize is that the program that was dismantled had been extraordinarily successful in graduating Latino students and sending them to college. Nationally, Latino students drop out of high school at a much higher rate than any other racial or ethnic group-about 18 percent, according to the National Center for Education Statistics. Among Latinos, aged 18 to 24, 27 percent have not graduated from high school, according to the Pew Hispanic Center. Nationally, the college-enrollment rate of Latinos, while at an all-time high, is only 32 percent-lower than that of other racial and ethnic groups, according to the same study. In contrast, students completing Tucson's Mexican-American studies program graduate high school and enter college at a higher rate in a district that is 60 percent Latino.

In an era when school reform and budget savings are championed by representatives of both major political parties, Long Island cannot economically, politically, or culturally afford to maintain small racially segregated school districts. Based on demographic data available in New York: The State of Learning, an annual statistical profile of New York State school districts, Malverne schools and schools in surrounding communities do not have to be racially segregated. In near by Rockville Centre, 80 percent of the students are white. If Malverne, Lakeview, and Rockville Centre were combined into one school district, the student population would be 53 percent white, 30 percent black, 13 percent Latino, and 4 percent Asian. If we think even more broadly and Malverne, Lakeview, Rockville Centre, West Hempstead, Lynbrook, and East Rockaway were consolidated into a manageable district with under 11,000 students, the student population would be 69 percent white, 14 percent black, 13 percent Hispanic, and 4 percent Asian.

Blatant forms of discrimination against women in academia have diminished since the Equal Pay Act and Title IX became law in 1964 and 1972, respectively. Yet gender differences in salary, tenure status, and leadership roles still persist among men and women in higher education. In particular, wage differences among male and female professors have not been fully explained, even when productivity, teaching experience, institutional size and prestige, disciplinary fields, type of appointment, and family-related responsibilities are controlled for statistically (see here).

Journalists play an essential role in our society. They are charged with informing the public, a vital function in a representative democracy. Yet, year after year, large pockets of the electorate remain poorly-informed on both foreign and domestic affairs. For a long time, commentators have blamed any number of different culprits for this problem, including poverty, education, increasing work hours and the rapid proliferation of entertainment media. There is no doubt that these and other factors matter a great deal. Recently, however, there is growing evidence that the factors shaping the degree to which people are informed about current events include not only social and economic conditions, but journalist quality as well. Put simply, better journalists produce better stories, which in turn attract more readers. On the whole, the U.S. journalist community is world class. But there is, as always, a tremendous amount of underlying variation. It's likely that improving the overall quality of reporters would not only result in higher quality information, but it would also bring in more readers. Both outcomes would contribute to a better-informed, more active electorate. We at the Shanker Institute feel that it is time to start a public conversation about this issue. We have requested and received datasets documenting the story-by-story readership of the websites of U.S. newspapers, large and small. We are using these data in statistical models that we call "Readers-Added Models," or "RAMs."

I have written much on this blog about problems with the use of Value-added Estimates of teacher effect (used loosely) on student test score gains on this blog. I have addressed problems with both the reliability and validity of VAM estimates, and I have pointed out how SGP based estimates of student growth are invalid on their face for determining teacher effectiveness. But, I keep hearing two common refrains from the uber-reformy (those completely oblivious to the statistics and research of VAM while also lacking any depth of understanding of the complexities of the social systems [schools] into which they propose to implement VAM as a de-selection tool) crowd. Sadly, these are the people who seem to be drafting policies these days.

"Most research on educational interventions, including school vouchers, focuses on impacts on short-term outcomes such as students' scores on standardized tests. Few studies are able to track longer-term outcomes, and even fewer are able to do so in the context of a randomized experiment. In the first study using a randomized experiment to measure the impact of school vouchers on college enrollment, we examine the college-going behavior through 2011 of students who participated in a voucher experiment as elementary school students in the late 1990s. We find no overall impacts on college enrollments but we do find large, statistically significant positive impacts on the college going of African American students who participated in the study. Our estimates indicate that using a voucher to attend private school increased the overall college enrollment rate among African Americans by 24 percent."

This technical report contains four main sections:  1) Data. Description of the data used to implement the student growth model, including data processing rules and relevant issues that arose during processing.  2) Model. Statistical description of the model.  3) Reporting. Description of reporting metrics and computation of effectiveness scores.  4) Results. Overview of key model results aimed at providing information on model quality and characteristics. It is important to note that results presented in this report are based on 2010-11 and prior school years' data. The model will be re-estimated with 2011-12 data when they are available"

"The education reform debate is fueled by a seemingly endless and even fruitless point-counterpoint among the corporate reformers-typically advocates for and from the Gates Foundation (GF), Teach for America (TFA), and charter chains such as Knowledge Is Power Program (KIPP)-and educators/scholars of education. Since the political and public machines have embraced the corporate reformers, GF, TFA, and KIPP have acquired the bully pulpit of the debate and thus are afforded most often the ability to frame the point, leaving educators and scholars to be in a constant state of generating counter-points. This pattern disproportionately benefits corporate reformers, but it also exposes how those corporate reformers manage to maintain the focus of the debate on data. The statistical thread running through most of the point-counterpoint is not only misleading (the claims coming from the corporate reformers are invariably distorted, while the counter-points of educators and scholars remain ignored among politicians, advocates, the public, and the media), but also a distraction. Since the metrics debate (test scores, graduation rates, attrition, populations of students served, causation/correlation) appears both enduring and stagnant, I want to make a clear statement with some elaboration that I reject the "ends-justify-the-means" assumptions and practices-the broader "no excuses" ideology-underneath the numbers, and thus, we must stop focusing on the outcomes of programs endorsed by the GF or TFA and KIPP. Instead, we must unmask the racist and classist policies and practices hiding beneath the metrics debate surrounding GF, TFA, and KIPP (as prominent examples of practices all across the country and types of schools)."

"Brookings recently released an evaluation of New York City's voucher program, called the School Choice Scholarship Foundation Program (SCSF), which was implemented in the late 1990s. Voucher offers were randomized, and the authors looked at the impact of being offered/accepting them on a very important medium-term outcome - college enrollment (they were also able to follow an unusually high proportion of the original voucher recipients to check this outcome). The short version of the story is that, overall, the vouchers didn't have any statistically discernible impact on college enrollment. But, as is often the case, there was some underlying variation in the results, including positive estimated impacts among African-American students, which certainly merit discussion.* Unfortunately, such nuance was not always evident in the coverage of and reaction to the report, with some voucher supporters (strangely, given the results) exclaiming that the program was an unqualified success, and some opponents questioning the affiliations of the researchers. For my part, I'd like to make a quick, not-particularly-original point about voucher studies in general: Even the best of them don't necessarily tell us much about whether "vouchers work.""

A keystone of this reform movement is the replacement of subjective evaluation with quantifiable measures of each teacher's effectiveness. The quantitative method is known as value-added modeling (VAM), a statistical analysis of student scores that seeks to identify how much an individual teacher contributes to a pupil's progress over the years. The use of VAM in teacher evaluations is growing, but the method remains extremely controversial. Critics often claim that it does not and cannot measure actual teacher quality. This paper addresses that claim. Part I analyzes data from Florida public schools to show that a VAM score in a teacher's third year is a good predictor of that teacher's success in his or her fifth year. Having established that VAM is a useful predictive tool, Part II of the paper addresses the most effective ways that VAM can be used in tenure reform."

For this podcast, we spoke with Jack Buckley, commissioner at the National Center for Education Statistics, the center which administers the NAEP test. He took Early Ed Watch on a tour of the data from the most recent NAEP scores. Among the highlights is a trend that Buckley says suggests that students who are both at the top and bottom of their grade level may be improving-a finding contrary to the notion that No Child Left Behind has led teachers to focus on low-performing students at the expense of high-performing ones.

The Center For American Progress has published another report justifying the firing of teachers today, based on statistical models that may some day become valid. "Designing High Quality Evaluation Systems," by John Tyler, recounts the standard reasons why of educators do not trust high-stakes test-driven algorithms, and even contributes a couple of new insights into problems that are unique to high school test scores. An urban teacher reading Tyler's evidence would likely conclude that he has written an ironclad indictment of value-added models for high-stakes purposes. But, as is usually true of CAP's researchers, he concludes that the work of economists in improving value-added models is so impressive that education will benefit from their experiments if educators don't blow it.

Value-added and other types of growth models are probably the most controversial issue in education today. These methods, which use sophisticated statistical techniques to attempt to isolate a teacher's effect on student test score growth, are rapidly assuming a central role in policy, particularly in the new teacher evaluation systems currently being designed and implemented.

After reading the New York times op-ed on school choice in DC, I asked some folks close to what's happening in education there for their thoughts. Mary Levy sent me what is written below and (with her permission), I decided to use it as a guest post. Mary Levy has analyzed DC Public School staffing, budget and expenditures, and monitored the progress of education reform for thirty years. She is a major source for fiscal, statistical and general information on DCPS for the media, government officials and non-profit, business and civic groups. She directed the Public Education Reform Project at the Washington Lawyers' Committee for Civil Rights & Urban Affairs for 19 years, during which she played a major role in developing the District of Columbia's school funding systems, wrote numerous reports on DCPS, and participated in every major reform planning initiative. Previously, in private practice with Rauh, Lichtman, Levy & Turner, she did civil litigation in civil rights, labor law, and school finance, including major litigations in New York  and Maryland.

Education policies that affect millions of students have long been tied to test scores, but a new paper suggests those scores are regularly misinterpreted. According to the new research out of Mathematica, a statistical research group, the comparisons sometimes used to judge school performance are more indicative of demographic change than actual learning.

You'll often hear the argument that half or almost half of all beginning U.S. public school teachers leave the profession within five years. The implications of this statistic are, of course, that we are losing a huge proportion of our new teachers, creating a "revolving door" of sorts, with teachers constantly leaving the profession and having to be replaced. This is costly, both financially (it is expensive to recruit and train new teachers) and in terms of productivity (we are losing teachers before they reach their peak effectiveness). And this doesn't even include teachers who stay in the profession but switch schools and/or districts (i.e., teacher mobility).* Needless to say, some attrition is inevitable, and not all of it is necessarily harmful, Many new teachers, like all workers, leave (or are dismissed) because they are just aren't good at it - and, indeed, there is test-based evidence that novice leavers are, on average, less effective. But there are many other excellent teachers who exit due to working conditions or other negative factors that might be improved

Good teaching is not something that can be calculated through a statistical analysis of students' results on a standardized test. What would be much more meaningful would be to determine teachers' effectiveness by the data we collect from our students and parents. We should ask our clients how they would rate teacher planning, pedagogy, effectiveness, and content knowledge. Good teaching begins with good intentions, and is coupled with a love of knowledge, a passion for pedagogy, and most importantly a belief that all children regardless of their home language or zip code can achieve.

A follow-up to a major national study on the performance of charter school networks shows that they have varied results on their students' high school graduation rates and on their postsecondary enrollment. The study shows that, of the six charter-management organizations for which data were available, three have significant positive impacts on graduation compared to the traditional public schools in their area. One of those organizations increased the probability that its students graduate from high school in four years by 23 percentage points. Two other charter-management groups have positive but not statistically significant impacts on graduation. And one network had a serious negative impact on the graduation rates of its students compared to the local public schools, reducing the probability that students would graduate on time by 22 percentage points.

A persistent -- and, I think, powerful -- theme in conservatism is the emphasis upon limits and doubt about the wisdom of centralized actors.  One of the strongest pragmatic defenses of the free market is that centralized authority is not efficient enough and wise enough to direct the economic energies of the nation; hence, a diversity of economic actors should make their own, personal economic decisions.  Moreover, a mainstream of conservative philosophy from Burke onwards suggests that the richness of a given human society and culture goes beyond mere statistics -- something about the weave of human life resists quantification. However, a great many "conservatives" ignore these teachings when the topic of education "reform" comes up.