Group items tagged

Released in August, the U.S. Department of Education study mapping state proficiency standards onto the National Assessment of Educational Progress scales produced a remarkable statement from Joanne Weiss, the chief of staff to U.S. Secretary of Education Arne Duncan. According to an article in the Aug. 24 issue of Education Week, Weiss said the practice of permitting each state to set its own proficiency standards amounts to "lying to parents, lying to children, lying to teachers and principals about the work they're doing." Her intemperate outburst crosses the line, not only by the standards of what passes for civil discourse in Washington these days, but also for what it says about the assessment itself. Indeed, a plausible case can be made that when it comes to telling fibs about proficiency, NAEP has a nose that annually grows longer, for its definition of proficiency is seriously flawed.

The media react with alarm every time the NAEP scores appear because only about one-third or so of students is rated "proficient." This is supposed to be something akin to a national tragedy because presumably almost every child should be "proficient." Remember, under No Child Left Behind, ALL students are supposed to be proficient in reading and math by the year 2014. Since I served on NAGB for seven years, I can explain what the board's "achievement levels" mean. There are four levels. At the top is "advanced." Then comes "proficient." Then "basic." And last, "below basic."

"A couple of weeks ago, Mike Petrilli of the Fordham Institute made the case that absolute proficiency rates should not be used as measures of school effectiveness, as they are heavily dependent on where students "start out" upon entry to the school. A few days later, Fordham president Checker Finn offered a defense of proficiency rates, noting that how much students know is substantively important, and associated with meaningful outcomes later in life. They're both correct. This is not a debate about whether proficiency rates are at all useful (by the way, I don't read Petrilli as saying that). It's about how they should be used and how they should not."

"Earlier this summer, the New York City Independent Budget Office (IBO) presented findings from a longitudinal analysis of NYC student performance. That is, they followed a cohort of over 45,000 students from third grade in 2005-06 through 2009-10 (though most results are 2005-06 to 2008-09, since the state changed its definition of proficiency in 2009-10). The IBO then simply calculated the proportion of these students who improved, declined or stayed the same in terms of the state's cutpoint-based categories (e.g., Level 1 ["below basic" in NCLB parlance], Level 2 [basic], Level 3 [proficient], Level 4 [advanced]), with additional breakdowns by subgroup and other variables. The short version of the results is that almost two-thirds of these students remained constant in their performance level over this time period - for instance, students who scored at Level 2 (basic) in third grade in 2006 tended to stay at that level through 2009; students at the "proficient" level remained there, and so on. About 30 percent increased a category over that time (e.g., going from Level 1 to Level 2). The response from the NYC Department of Education (NYCDOE) was somewhat remarkable. It takes a minute to explain why, so bear with me."

Public School 30 and Public School 179 are about as alike as two schools can be. They are two blocks apart in the South Bronx. Both are 98 percent black and Latino. At P.S. 30, 97 percent of the children qualify for subsidized lunches; at P.S. 179, 93 percent. During city quality reviews - when Education Department officials make on-site inspections - both scored "proficient." The two have received identical grades for "school environment," a rating that includes attendance and a survey of parents', teachers' and students' opinions of a school. On the state math test, P.S. 30 did better in 2011, with 41 percent of students scoring proficient - a 3 or 4 - versus 29 percent for P.S. 179. But on the state English test, P.S. 179 did better, with 36 percent of its students scoring proficient compared with 32 percent for P.S. 30. And yet, when the department calculated the most recent progress report grades, P.S. 30 received an A. And P.S. 179 received an F.

The use of rate changes is still proliferating rapidly at all levels of our education system. These measures, which play an important role in the provisions of No Child Left Behind, are already prominent components of many states' core accountability systems (e..g, California), while several others will be using some version of them in their new, high-stakes school/district "grading systems." New York State is awarding millions in competitive grants, with almost half the criteria based on rate changes. District consultants issue reports recommending widespread school closures and reconstitutions based on these measures. And, most recently, U.S. Secretary of Education Arne Duncan used proficiency rate increases as "preliminary evidence" supporting the School Improvement Grants program. Meanwhile, on the public discourse front, district officials and other national leaders use rate changes to "prove" that their preferred reforms are working (or are needed), while their critics argue the opposite. Similarly, entire charter school sectors are judged, up or down, by whether their raw, unadjusted rates increase or decrease. So, what's the problem? In short, it's that year-to-year changes in proficiency rates are not valid evidence of school or policy effects. These measures cannot do the job we're having them do, even on a limited basis. This really has to stop.

"The accountability provisions in Virginia's original application for "ESEA flexibility" (or "waiver") have received a great deal of criticism (see here, here, here and here). Most of this criticism focused on the Commonwealth's expectation levels, as described in "annual measurable objectives" (AMOs) - i.e., the statewide proficiency rates that its students are expected to achieve at the completion of each of the next five years, with separate targets established for subgroups such as those defined by race (black, Hispanic, Asian, white), income (subsidized lunch eligibility), limited English proficiency (LEP), and special education. Last week, in response to the criticism, Virginia agreed to amend its application, and it's not yet clear how specifically they will calculate the new rates (only that lower-performing subgroups will be expected to make faster progress). In the meantime, I think it's useful to review a few of the main criticisms that have been made over the past week or two and what they mean."

New York State is set to release its annual testing data tomorrow. Throughout the state, and especially in New York City, we will hear a lot about changes in school and district proficiency rates. The rates themselves have advantages - they are easy to understand, comparable across grades and reflect a standards-based goal. But they also suffer severe weaknesses, such as their sensitivity to where the bar is set and the fact that proficiency rates and the actual scores upon which they're based can paint very different pictures of student performance, both in a given year as well as over time. I've discussed this latter issue before in the NYC context (and elsewhere), but I'd like to revisit it quickly.

About a week ago, Florida officials went into crisis mode after revealing that the proficiency rate on the state's writing test (FCAT) dropped from 81 percent to 27 percent among fourth graders, with similarly large drops in the other two grades in which the test is administered (eighth and tenth). The panic was almost immediate. For one thing, performance on the writing FCAT is counted in the state's school and district ratings. Many schools would end up with lower grades and could therefore face punitive measures. Understandably, a huge uproar was also heard from parents and community members. How could student performance decrease so dramatically? There was so much blame going around that it was difficult to keep track - the targets included the test itself, the phase-in of the state's new writing standards, and test-based accountability in general. Despite all this heated back-and-forth, many people seem to have overlooked one very important, widely-applicable lesson here: That proficiency rates, which are not "scores," are often extremely sensitive to where you set the bar.

The number of students in virtual schools run by educational management organizations rose sharply last year, according to a new report being published Friday, and far fewer of them are proving proficient on standardized tests compared with their peers in other privately managed charter schools and in traditional public schools.

Given our extreme reliance on test scores as measures of educational success and failure, I'm sorry I have to make this point: proficiency rates are not test scores, and changes in proficiency rates do not necessarily tell us much about changes in test scores.

In response to my Nation piece on achievement gaps in Washington, D.C. district public schools, commenter E.B. wondered how things would look different if we measured student proficiency instead of raw NAEP scores. This is a great question, since proficiency--defined as "solid academic performance"--is the standard to which we should hold most children.

This report finds that the exclusion of thousands of students with disabilities from reported Connecticut Mastery Test results has distorted reported trends in test scores. Following test scores from year to year in the same grade, the study finds that statewide improvements in standard Connecticut Mastery Test (CMT) scores reported by the Connecticut State Department of Education (SDE) between 2008 and 2009 -- the period of the largest reported gains -- were largely the result of the exclusion of students with disabilities from these standard test results, rather than overall improvements in performance. For example, 84% of the reported improvement in 4th grade math proficiency between 2008 and 2009 and 69% of the improvement in 8th grade reading proficiency could be attributed to the exclusion of these students. Much of the reported improvements in later years could also be attributed to this exclusion, though there were some modest overall gains as well.

"The rationale here is muddled at best, but the detriments are obvious. For instance, young students in New York State who are developing as they should will be placed in remedial services, forgoing enrichment in the arts because they are a "2" and thus below the new proficiency level. That is where the vast majority of students fall on the new scales - below proficiency and off the "road to college readiness."  Students, who in reality may not need support will be sorted into special education or "response to intervention" services.  Parents will worry for their children's future. The newspapers will bash the public schools and their teachers at a time when morale is already at an extreme low. The optimism teachers first felt about the Common Core State Standards is fading as the standards and their tests roll into classrooms."

"A dominant narrative of the past decade and a half of education reform has been to highlight alleged persistent failures of our education system.  While this tale began long ago with the Reagan Administration report A Nation at Risk, it has been put into overdrive in the era of test based accountability that began with the No Child Left Behind Act.  That series of amendments to the Elementary and Secondary Education Act mandated annual standardized testing of all students in grades 3-8 and once in high school, set a target for 100% proficiency for all students in English and mathematics, and imposed consequences for schools and districts that either failed to reach proficiency targets or failed to test all students.  Under the Obama administration, the federal Department of Education has freed states from the most stringent requirements to meet those targets, but in return, states had to commit themselves to specific reforms such as the adoption of common standards, the use of standardized test data in the evaluation of teachers, and the expansion of charter schools.  All of these reforms are predicated on the constantly repeated belief that our citizens at all levels are falling behind international competitors, that our future workforce lacks the skills they will need in the 21st century, and that we have paid insufficient attention to the uneven distribution of equal opportunity in our nation. But what if we've gotten the entire thing wrong the whole time?"

The Star Tribune declares it a miracle school on 9/24/11 with the headline 'At this school, usual excuses don't apply' http://www.startribune.com/opinion/otherviews/130474133.html Claim from the article: In this year's state math tests in grades three through eight, this school outperformed every metro-area school district, including Edina and Wayzata. Its students outperformed all state students in reading proficiency (77 percent to 75 percent), and state white students in math proficiency (82 percent to 65 percent).

New York State Education Commissioner John B. King has posted a Back to School message for educators, with a transcript included. The message for educators is that all adults in schools should constantly be asking, "Who is proficient? How do I know and how do I increase those levels of proficiency?" From the district superintendent, to the superintendent, to the principal, to the teacher, to the office staff, everyone has to be continuously asking, "Where are we? Where are we in terms of our goals and where are we in terms of our students' college and career readiness and how do we get there?"

Latino students in California - nearly 1.3 million of them English learners - are struggling to achieve academic success at the same level as their white peers. In "State has one of nation's highest gaps in Hispanic-white reading proficiency," Sarah Garland reports that only 12 percent of Hispanic fourth-graders in California tested proficient in reading in 2009. Nonprofits, government agencies and parents have all launched campaigns over the years to close the learning gap, but little progress has been made.

"New York City has seen some of the more far-reaching educational reforms over the past decade, as Mayor Michael Bloomberg and Chancellor Joel Klein set in motion an array of market-based reforms. Both Bloomberg and Klein argued vigorously that the New York City schools had substantially closed the achievement gap, pointing to a shrinking difference in the percentage of white students and Black and Latino students classified as proficient on the New York State English Language Arts and mathematics assessments administered in grades 3 to 8. Many scholars have demonstrated, however, that differences in proficiency rates are potentially misleading, and especially so if the tests have inflated scores reflecting predictable and easier questions. Has the achievement gap in New York City decreased over time? What happened to the achievement gap when the state of New York, recognizing the flaws in its testing system, raised the "cut scores" defining proficiency on its tests in 2010?"

There comes a time when you look at the rug on the floor, the one you've seen many times, and you see a pattern that you had never noticed before. You may have seen this squiggle or that flower, but you did not see the pattern into which the squiggles and flowers and trails of ivy combined. In American education, we can now discern the pattern on the rug. Consider the budget cuts to schools in the past four years. From the budget cuts come layoffs, rising class sizes, less time for the arts and physical education, less time for history, civics, foreign languages, and other non-tested subjects. Add on the mandates of No Child Left Behind, which demands 100 percent proficiency in math and reading and stigmatizes more than half the public schools in the nation as "failing" for not reaching an unattainable goal. Along comes the Obama administration with the Race to the Top, and the pattern on the rug gets clearer.