Group items matching

in title, tags, annotations or url

"In numerous previous posts I have summarized New Jersey charter school enrollment data, frequently pointing out that the highest performing charter schools in New Jersey tend to be demographically very different from schools in their surrounding neighborhoods and similar grade level schools throughout their host districts or cities. I have tried to explain over and over that the reason these differences are important is because they constrain the scalability of charter schooling as a replicable model of "success." Again, to the extent that charter successes are built on serving vastly different student populations, we can simply never know (even with the best statistical analyses attempting to sort out peer factors, control for attrition, etc.) whether the charter schools themselves, their instructional strategies/models are effective and/or would be effective with larger numbers of more representative students. Here, I take a quick look at the other side of the picture, again focusing on the city of Newark. Specifically, I thought it would be interesting to evaluate the effect on Newark schools enrollment of the shift in students to charter schools, now that charters have taken on a substantial portion of students in the city. If charter enrollments are - as they seem to be - substantively different from district schools enrollments, then as those charter populations grow and remain different from district schools, we can expect the district schools population to change.  In particular, given the demography of charter schools in Newark, we would expect those schools to be leaving behind a district of escalating disadvantage - but still a district serving the vast majority of kids in the city."

"In a sense, these Gulen salary structures and claims of insufficient teacher supply especially in math and science may be providing us with some insights as to what happens when we choose to pay teachers so poorly and when we strip them of any expectation of increased wages with experience. Maybe they do really have a domestic teacher supply problem. But their solution to that problem is not a scalable solution for American public schooling at large (cheap imported and temporary labor)."

This post provides a quick follow up to yesterday's post (late last night) when I critiqued a questionable graph from an NJDOE presentation here: State of NJ Schools presentation 2-29-2012 It turns out that the slide presentation had many comparable graphs that deserve at least some attention.

The standard reformy template is that teachers should only be able to get tenure after 3 years of good ratings in a row and that teachers should be subject to losing tenure if they get 2 bad years in a row.  Further, it is possible that the evaluations might actually stipulate that you can only get a good rating if you achieve a certain rating on the quantitative portion of the evaluation - or the VAM score. Likewise for bad ratings (that is, the quantitative measure overrides all else in the system). The premise of the dice rolling activity from my previous post was that it is necessarily much less likely to roll the same number (or subset of numbers) three times in a row than twice (exponentially in fact). That is, it is much harder to overcome the odds based on error rates to achieve tenure, and much easier to lose it. Again, this is much due to the noisiness of the data, and less due to the difficulty of actually being "good" year after year. The ratings simply jump around a lot. See my previous post.

How, for example, if I was the principal of a given, average sized school in NYC, might I use the VA data on my teachers to council them? to suggest personnel changes? assignment changes, or so on? Would these data, as they are, provide me any useful information about my staff and how to better my school? For this exercise, I've decided to look at the year to year ratings of teachers in a relatively average school. Now, why would I bother looking at the year to year ratings when we know that the multi-year averages are supposed to more accurate - more representative of the teacher's over time contributions? Well, you'll see in the graphs below that those multi-year averages also may not be that useful.

Readers of my blog know I'm both a data geek and a skeptic of the usefulness of Value-added data specifically as a human resource management tool for schools and districts. There's been much talk this week about the release of the New York City teacher ratings to the media, and subsequent publication of those data by various news outlets. Most of the talk about the ratings has focused on the error rates in the ratings, and reporters from each news outlet have spent a great deal of time hiding behind their supposed ultra-responsibleness of being sure to inform the public that these ratings are not absolute, that they have significant error ranges, etc.  Matt Di Carlo over at Shanker Blog has already provided a very solid explanatory piece on the error ranges and how those ranges affect classification of teachers as either good or bad. But, the imprecision - as represented by error ranges - of each teacher's effectiveness estimate is but one small piece of this puzzle. And in my view, the various other issues involved go much further in undermining the usefulness of the value added measures which have been presented by the media as necessarily accurate albeit lacking in precision.

The apparent inference here? Either poverty itself really isn't that important a factor in determining student success rates on state assessments, or, alternatively, free and reduced lunch simply isn't a very good measure of poverty even if poverty is a good predictor. Either way, something's clearly amiss if we have so many higher poverty schools outperforming lower poverty ones. In fact, the only dots included in the graph are high poverty districts outperforming lower poverty ones. There can't be much of a pattern between these two variables at all, can there? If anything, the trendline must be sloped up hill? (that is, higher poverty leads to higher outcomes!)

...a large section of our report summarizes the relevant methods - those rigorous and appropriate designs that should be applied to the questions at hand, but are noticeably absent even at the most cursory level in Roza and Hill's materials. To save you all the trouble of actually reading our entire brief, I've copied and pasted below the section of our brief where we address relevant methods

Pau Hill and Marguerite Roza's response to my recent report - with Kevin Welner - and series of blog posts seems to offer as its central argument that we're simply a curmudgeons, offering lots of complaints about the rigor of their arguments and their suggestions for improving schooling productivity and efficiency, but providing no creative or immediately useful ideas or solutions for school districts or states in these tough economic times. My first response would be that bad ideas are bad ideas, even in the absence of alternatives. The fact that budgets are tight and many schools are underperforming is not an argument for implementing unproven, ill-considered policy solutions. That said, my second response is that Kevin Welner and I did in fact offer our own solutions, both in our policy brief and elsewhere.

For a short while yesterday - more than I would have liked to - I followed the circus of testimony and tweets about proposed education reform legislation in Connecticut. The reform legislation - SB 24 - includes the usual reformy elements of teacher tenure reform, ending seniority preferences, expanding and promoting charter schooling, etc. etc. etc. And the reformy circus had twitpics of of eager undergrads (SFER) & charter school students (as young as Kindergarten?) shipped in and carrying signs saying CHARTER=PUBLIC (despite a body of case law to the contrary, and repeated arguments, some lost in state courts [oh], by charter operators that they need not comply with open records/meetings laws or disclose employee contracts), and tweeting reformy platitudes and links to stuff they called research supporting the reformy platform (Much of it tweeted as "fact checking" by the ever-so-credible ConnCAN). Ignored in all of this theatre-of-the-absurd was any actual substantive, knowledgeable conversation about the state of public education in Connecticut, the nature of the CT achievement gap and the more likely causes of it, and other problems/failures of Connecticut education policy.

List of downloadable papers and posters

John Mooney at NJ Spotlight provided a reasonable overview of the NJDOE waiver proposal to "reward" successful schools and sanction and/or takeover "failing" ones. The NJDOE waiver proposal includes explanation of a new classification system for identifying which schools should be subject to state intervention, ultimately to be managed by regional offices throughout the state. This new targeted intervention system classifies districts in need of intervention as "priority" districts, with specific emphasis on "focus" districts.

Neal McClusky at Cato Institute makes a good point about our casual, imprecise use of the term "democracy" in the post linked here. I did not delve into this in my previous post, and more or less allowed the imprecise terminology to slip past. Clearly there are huge differences between simple majority rule through direct democracy and our constitutional republic with separation of powers, and I certainly favor the latter.

This note is in response to James O'Keefe's attempt to discredit me on his Project Veritas web site (though I think his point was intended to larger than this). I was lucky (?) enough to be part of one of his investigative set ups earlier this fall. I wrote and held on to this post and all related e-mails.

"Much has been made of late regarding the erroneous classification of 44 teachers in Washington DC as ineffective, thus facing job consequences. This particular erroneous rating was based on an "error" in the calculation of the teachers' total ratings, as acknowledged by the consulting firm applying the ratings. That is, in this case, the consultants simply did not carry out their calculations as intended. This is not to suggest by any stretch that the intended calculations are necessarily more accurate or precise than the unintended error. That is, there certainly may be far more - are likely far more than these 44 teachers whose ratings fall arbitrarily and capriciously in the zone whereby those teachers would face employment consequences. So, how can we tell… how can we identify such teachers."

"My problem with the current false market scenario regarding TFA is its intersection with the false market for charter schools. Just as charter school expansion - demand - has become heavily dependent on manipulation of markets by policy makers, TFA expansion - demand - has become dependent on those major charter network operators who are dependent on charter market manipulation (forced closure of district schools). Put simply, this is not market based reform, nor should anyone pretend that market mechanisms (rather than policy preferences and market manipulation) are driving any of this."

"Let's take another trip back to Philadelphia for the day, because the reformy conversation around Philadelphia is just so darn illustrative of how reformy thinking works. Here's a synopsis of the reformy approach to pushing pre-established, fact free, ideological reforms:"

"This bizarre video got me thinking about a series of previous posts where I've looked across numerous indicators to try to tease out the relationships among them, across states.  I've selectively scoured scatterplots of relationships between various state level indicators and outcome measures, but have not for a while now, simply stepped back and evaluated the correlations across all of them, and then tried to tease out what states, if any really do stand out."

"Yesterday gave us the release of the 2013 NAEP results, which of course brings with it a bunch of ridiculous attempts to cast those results as supporting the reform-du-jour. Most specifically yesterday, the big media buzz was around the gains from 2011 to 2013 which were argued to show that Tennessee and Washington DC are huge outliers - modern miracles - and that because these two settings have placed significant emphasis on teacher evaluation policy - that current trends in teacher evaluation policy are working - that tougher evaluations are the answer to improving student outcomes - not money… not class size… none of that other stuff."

"Moody's Investors Service downgraded the bond ratings of 53 school districts in Michigan. Public schools are losing enrollment to charter schools, and losing the ability to balance their budgets. More than 80% of the charter schools in Michigan are operated for-profit."