Skip to main content

Home/ CTLT and Friends/ Group items tagged accountability

Rss Feed Group items tagged

Joshua Yeidel

The Answer Sheet - A principal on standardized vs. teacher-written tests - 0 views

  •  
    High school principal George Wood eloquently contrasts standardized NCLB-style testing with his school's performance assessments.
Joshua Yeidel

Performance Assessment | The Alternative to High Stakes Testing - 0 views

  •  
    " The New York Performance Standards Consortium represents 28 schools across New York State. Formed in 1997, the Consortium opposes high stakes tests arguing that "one size does not fit all." Despite skepticism that an alternative to high stakes tests could work, the New York Performance Standards Consortium has done just that...developed an assessment system that leads to quality teaching, that enhances rather than compromises our students' education. Consortium school graduates go on to college and are successful."
Joshua Yeidel

Higher Education: Assessment & Process Improvement Group News | LinkedIn - 0 views

  •  
    High School Principal George Wood eloquently contrasts standardized NCLB-style testing and his school's term-end performance testing.
Gary Brown

Home - Journal of Assessment and Accountability Systems in Educator Preparation - 1 views

  •  
    a new journal to note
Gary Brown

71 Presidents Pledge to Improve Their Colleges' Teaching and Learning - Faculty - The C... - 0 views

  • In a venture known as the Presidents' Alliance for Excellence in Student Learning and Accountability, they have promised to take specific steps to gather more evidence about student learning, to use that evidence to improve instruction, and to give the public more information about the quality of learning on their campuses.
  • The 71 pledges, officially announced on Friday, are essentially a dare to accreditors, parents, and the news media: Come visit in two years, and if we haven't done these things, you can zing us.
  • deepen an ethic of professional stewardship and self-regulation among college leaders
  • ...4 more annotations...
  • Beginning in 2011, all first-year students at Westminster will be required to create electronic portfolios that reflect their progress in terms of five campuswide learning goals. And the college will expand the number of seniors who take the Collegiate Learning Assessment, so that the test can be used to help measure the strength of each academic major.
  • "The crucial thing is that all of our learning assessments have been designed and driven by the faculty," says Pamela G. Menke, Miami Dade's associate provost for academic affairs. "The way transformation of learning truly occurs is when faculty members ask the questions, and when they're willing to use what they've found out to make change.
  • Other assessment models might point some things out, but they won't be useful if faculty members don't believe in them."
  • "In the long term, as more people join, I hope that the Web site will provide a resource for the kinds of innovations that seem to be successful," he says. "That process might be difficult. Teaching is an art, not a science. But there is still probably a lot that we can learn from each other."
Joshua Yeidel

News: Measuring 2-Year Students' Success - Inside Higher Ed - 0 views

  •  
    Measuring student success for federal purposes in community colleges -- and only one passing mention of learning. The factory model prevails -- stamp that widget (student) and send it out the door (they sell themselves!)
Joshua Yeidel

Higher Education: Assessment & Process Improvement Group News | LinkedIn - 2 views

  •  
    "Colleges and universities have transformed themselves from participants in an audit culture to accomplices in an accountability regime."
  •  
    A philosophical critique of a rapidly-approaching "metric future", with "comensuration" (assigning meaning to measurements) run amok. While the application of student learning outcomes given in the article is not ours, the critique of continuous quality improvement challenges some of our assumptions.
Theron DesRosier

Assessing Learning Outcomes at the University of Cincinnati: Comparing Rubric Assessmen... - 2 views

  •  
    "When the CLA results arrived eight months later, the UC team compared the outcomes of the two assessments. "We found no statistically significant correlation between the CLA scores and the portfolio scores," Escoe says. "In some ways, it's a disappointing finding. If we'd found a correlation, we could tell faculty that the CLA, as an instrument, is measuring the same things that we value and that the CLA can be embedded in a course. But that didn't happen." There were many factors that may have contributed to the lack of correlation, she says, including the fact that the CLA is timed, while the rubric assignments are not; and that the rubric scores were diagnostic and included specific feedback, while the CLA awarded points "in a black box": if a student referred to a specific piece of evidence in a critical-thinking question, he or she simply received one point. In addition, she says, faculty members may have had exceptionally high expectations of their honors students and assessed the e-portfolios with those high expectations in mind-leading to results that would not correlate to a computer-scored test. In the end, Escoe says, the two assessments are both useful, but for different things. The CLA can provide broad institutional data that satisfies VSA requirements, while rubric-based assessment provides better information to facilitate continuous program improvement. "
  •  
    Another institution trying to make sense of the CLA. This study compared student's CLA scores with criteria-based scores of their eportfolios. The study used a modified version of the VALUE rubrics developed by the AACU. Our own Gary Brown was on the team that developed the critical thinking rubric for the VALUE project.
  •  
    "The CLA can provide broad institutional data that satisfies VSA requirements, while rubric-based assessment provides better information to facilitate continuous program improvement. " This begs some questions: what meaning can we attach to these two non-correlated measures? What VSA requirements can rubric-based assessment NOT satisfy? Are those "requirements" really useful?
Joshua Yeidel

GOVT Week: David Bernstein on Top 10 Indicators of Performance Measurement Quality | AE... - 2 views

  •  
    Not surprisingly, the #1 indicator of performance measurement quality is "usefulness".
Judy Rumph

Blog U.: It Boils Down to... - Confessions of a Community College Dean - Inside Higher Ed - 4 views

  • I had a conversation a few days ago with a professor who helped me understand some of the otherwise-puzzling opposition faculty have shown to actually using the general education outcomes they themselves voted into place.
  • Yet getting those outcomes from ‘adopted’ to ‘used’ has proved a long, hard slog.
  • The delicate balance is in respecting the ambitions of the various disciplines, while still maintaining -- correctly, in my view -- that you can’t just assume that the whole of a degree is equal to the sum of its parts. Even if each course works on its own terms, if the mix of courses is wrong, the students will finish with meaningful gaps. Catching those gaps can help you determine what’s missing, which is where assessment is supposed to come in. But there’s some local history to overcome first.
  •  
    This is an interesting take on what we are doing and the comments interesting
Gary Brown

A Measure of Learning Is Put to the Test - Faculty - The Chronicle of Higher Education - 1 views

  • Others say those who take the test have little motivation to do well, which makes it tough to draw conclusions from their performance.
  • "Everything that No Child Left Behind signified during the Bush administration—we operate 180 degrees away from that," says Roger Benjamin, president of the Council for Aid to Education, which developed and promotes the CLA. "We don't want this to be a high-stakes test. We're putting a stake in the ground on classic liberal-arts issues. I'm willing to rest my oar there. These core abilities, these higher-order skills, are very important, and they're even more important in a knowledge economy where everyone needs to deal with a surplus of information." Only an essay test, like the CLA, he says, can really get at those skills.
  • "The CLA is really an authentic assessment process," says Pedro Reyes, associate vice chancellor for academic planning and assessment at the University of Texas system.
  • ...20 more annotations...
  • "The Board of Regents here saw that it would be an important test because it measures analytical ability, problem-solving ability, critical thinking, and communication. Those are the skills that you want every undergraduate to walk away with." (Other large systems that have embraced the CLA include California State University and the West Virginia system.)
  • value added
  • We began by administering a retired CLA question, a task that had to do with analyzing crime-reduction strategies,
  • performance task that mirrors the CLA
  • Mr. Ernsting and Ms. McConnell are perfectly sincere about using CLA-style tasks to improve instruction on their campuses. But at the same time, colleges have a less high-minded motive for familiarizing students with the CLA style: It just might improve their scores when it comes time to take the actual test.
  • by 2012, the CLA scores of more than 100 colleges will be posted, for all the world to see, on the "College Portrait" Web site of the Voluntary System of Accountability, an effort by more than 300 public colleges and universities to provide information about life and learning on their campuses.
  • If familiarizing students with CLA-style tasks does raise their scores, then the CLA might not be a pure, unmediated reflection of the full range of liberal-arts skills. How exactly should the public interpret the scores of colleges that do not use such training exercises?
  • Trudy W. Banta, a professor of higher education and senior adviser to the chancellor for academic planning and evaluation at Indiana University-Purdue University at Indianapolis, believes it is a serious mistake to publicly release and compare scores on the test. There is too much risk, she says, that policy makers and the public will misinterpret the numbers.
  • most colleges do not use a true longitudinal model: That is, the students who take the CLA in their first year do not take it again in their senior year. The test's value-added model is therefore based on a potentially apples-and-oranges comparison.
  • freshman test-takers' scores are assessed relative to their SAT and ACT scores, and so are senior test-takers' scores. For that reason, colleges cannot game the test by recruiting an academically weak pool of freshmen and a strong pool of seniors.
  • students do not always have much motivation to take the test seriously
  • seniors, who are typically recruited to take the CLA toward the end of their final semester, when they can already taste the graduation champagne.
  • Of the few dozen universities that had already chosen to publish CLA data on that site, roughly a quarter of the reports appeared to include erroneous descriptions of the year-to-year value-added scores.
  • It is clear that CLA scores do reflect some broad properties of a college education.
  • Students' CLA scores improved if they took courses that required a substantial amount of reading and writing. Many students didn't take such courses, and their CLA scores tended to stay flat.
  • Colleges that make demands on students can actually develop their skills on the kinds of things measured by the CLA.
  • Mr. Shavelson believes the CLA's essays and "performance tasks" offer an unusually sophisticated way of measuring what colleges do, without relying too heavily on factual knowledge from any one academic field.
  • Politicians and consumers want easily interpretable scores, while colleges need subtler and more detailed data to make internal improvements.
  • The CLA is used at more than 400 colleges
  • Since its debut a decade ago, it has been widely praised as a sophisticated alternative to multiple-choice tests
Gary Brown

Higher Education: Assessment & Process Improvement Group News | LinkedIn - 3 views

  •  
    as we ponder our own issues related to transparency....
Gary Brown

GAO - Generally Accepted Government Auditing Standards - 1 views

  • Our evaluator colleagues who work at GAO, and many others working in agencies and organizations that are responsible for oversight of, and focus on accountability for, government programs, often refer to the Yellow Book Standards. These agencies or organizations emphasize the importance of their independence from program officials and enjoy significant protections for their independence through statutory provisions, organizational location apart from program offices, direct reporting channels to the highest level official in their agency and governing legislative bodies, heightened tenure protections, and traditions emphasizing their independence.
  •  
    Good to have on the radar as DOE challenges the efficacy of accreditation, and not incidentally underpinning a principle of good evaluation.
Gary Brown

Colleges May Be Missing a Chance for Change - International - The Chronicle of Higher E... - 1 views

    • Gary Brown
       
      And what are people for, after all?
  • Peter P. Smith, senior vice president for academic strategies and development at Kaplan Higher Education, said that if traditional universities did not adjust, new institutions would evolve to meet student needs. Those new institutions, said Mr. Smith, whose company is a for-profit education provider, would be more student-centric, would deliver instruction with greater flexibility, and would offer educational services at a lower cost.
  • both education and research must become more relevant and responsive to society.
  • ...5 more annotations...
  • The gathering drew about 500 government officials, institutional leaders, and researchers
  • Speakers at an international conference here delivered a scathing assessment of higher education: Universities, they said, are slow to change, uncomfortable in dealing with real-world problems, and culturally resistant to substantive internationalization.
  • many faculty members may be "uncomfortable" with having deeper links to industry because they don't understand that world. Students, however, are highly practical, Mr. Fadel said, and are specifically seeking education that will get them a job or give them an advantage in the workplace.
  • "I'm sorry, as a student, you do not go to university to learn. You go to get a credential," he said.
    • Gary Brown
       
      And if you graduate more appreciative of the credential than what and how you have learned, then the education.
  • That does not mean colleges simply ought to turn out more graduates for in-demand professions like science and engineering, Mr. Fadel added. Colleges need to infuse other disciplines with science and engineering skills.
Gary Brown

Under Obama, Accreditation Is Still in the Hot Seat - Government - The Chronicle of Hig... - 1 views

  • George Miller, a California Democrat who is chairman of the House education committee, said defining a credit hour is critical to ensure that students and taxpayers, through federal student aid, are not footing the bill for courses that are not worth the amount of credit being awarded.
    • Gary Brown
       
      "Worth" opens up some interesting implications.  Intended I suspect, to dampen courses like basket-weaving, the production of outcomes cannot be far off, the production of economic impact related to those outcomes a step or less behind. 
  • Senators also questioned the independence of accreditors, which are supported by dues from member institutions and governed by representatives of the colleges they accredit.
  • Sen. Michael B. Enzi, the top Republican on the Senate Education Committee, has said he wants Congress to look beyond just problems in the for-profit sector. He said at a hearing last month that he would be "working to lay the groundwork for a broader, thorough, and more fair investigation into higher education" that would ask whether taxpayers are getting an appropriate value for the money they spend on all colleges.
  • ...10 more annotations...
  • State and federal governments are better equipped to enforce consumer protections for students, say accreditors, who have traditionally focused on preserving academic quality.
  • Judith S. Eaton, president of the Council for Higher Education Accreditation, which represents about 3,000 colleges, said that over the past several years accrediting organizations have responded to the growing calls for accountability and transparency from the public and lawmakers. The groups, she said, have worked to better identify and judge student achievement and share more information about what they do and how well the institutions are performing.
  • Peter T. Ewell, vice president of the National Center for Higher Education Management Systems, said the debate boils down to whether accreditors should serve primarily as consumer protectors or continue their traditional role of monitoring academic quality more broadly.
  • Richard K. Vedder, director of the Center for College Affordability & Productivity and a member of the Spellings Commission
  • "We should be moving to more of a Consumer Reports for colleges, to provide the public with information that the college rankings do imperfectly," he said.
  • accreditation will have to evolve to meet not only government's expectations but also the changing college
  • market
  • Nearly two years into the Obama Administration, colleges have not gotten the relief they expected from the contentious battles over measuring quality that defined the Bush Education Department.
  • Bracing for the prospect of new rules and laws that could expand their responsibilities, accreditors and the institutions they monitor are defending the self-regulation colleges use to ensure academic quality. But they are also responding to the pressures from the White House and Capitol Hill by making some changes on their own, hoping to stanch the possibility of more far-reaching federal requirements.
  • Advocates of change say the six regional and seven national accreditors have varying standards that are sometimes too lax, allowing for limited oversight of how credits are awarded, how much learning is accomplished, and what happens to the mission of institutions that change owners.
Gary Brown

Details | LinkedIn - 0 views

  • Although different members of the academic hierarchy take on different roles regarding student learning, student learning is everyone’s concern in an academic setting. As I specified in my article comments, universities would do well to use their academic support units, which often have evaluation teams (or a designated evaluator) to assist in providing boards the information they need for decision making. Perhaps boards are not aware of those serving in evaluation roles at the university or how those staff members can assist boards in their endeavors.
  • Gary Brown • We have been using the Internet to post program assessment plans and reports (the programs that support this initiative at least), our criteria (rubric) for reviewing them, and then inviting external stakeholders to join in the review process.
Gary Brown

Many College Boards Are at Sea in Assessing Student Learning, Survey Finds - Leadership... - 0 views

  • While oversight of educational quality is a critical responsibility of college boards of trustees, a majority of trustees and chief academic officers say boards do not spend enough time discussing student-learning outcomes, and more than a third say boards do not understand how student learning is assessed, says a report issued on Thursday by the Association of Governing Boards of Universities and Colleges.
  • While boards should not get involved in the details of teaching or ways to improve student-learning outcomes, they must hold the administration accountable for identifying needs in the academic programs and then meeting them, the report says. Boards should also make decisions on where to allocate resources based on what works or what should improve.
  • The most commonly received information by boards was college-ranking data
  • ...1 more annotation...
  • Boards should expect to receive useful high-level information on learning outcomes, the report says, and should make comparisons over time and to other institutions. Training in how to understand academic and learning assessments should also be part of orientation for new board members.
  •  
    This piece coupled with the usual commentary reveal again the profound identity crisis shaking education in this country.
Gary Brown

Texas A&M's Faculty Ratings: Right and Wrong - Commentary - The Chronicle of Higher Edu... - 0 views

  • "Academia is highly specialized. We don't mean to be exclusive. We are a public-serving group of people. But at the same time, that public isn't well-enough aware of what we do and who we are to evaluate us."
  • But the think tank is correct that taxpayers deserve to know how their money is being spent. Public-university operating costs in Texas have gone up more than 60 percent in the last two decades, even after adjusting for inflation, and professors are among the state's highest-paid public employees. The state needs accountability measures, and they must be enforced by a party other than the faculty, who, it could easily be charged, have a conflict of interest. That's what Texas A&M got right.
  • Moosally is right about one thing: The public isn't well aware of what she and many of her colleagues do. But they should be. That is not to say that the public will be able to understand what goes on in all of the chemistry laboratories in Texas. But Moosally teaches English at a college that is not exactly tasked with performing cutting-edge research. Houston-Downtown's mission is to provide "educational opportunities and access to students from a variety of backgrounds including many first-generation college students."
  • ...7 more annotations...
  • No doubt there is useful research coming out of the university system. But plenty could be omitted without a great deal of detriment to students' education. For instance, Hugill's most recent contributions have included a chapter on "Transitions in Hegemony: A Theory Based on State Type and Technology" and the article "German Great-Power Relations in the Pages of Simplicissimus, 1896-1914." Moosally's master's thesis was titled "Resumptive Pronouns in Modern Standard Arabic: A Head-Driven Phrase Structure Grammar Account," and her current research interests include "interactions between grammar knowledge and writing abilities/interest [and] cross-linguistic patterns of agreement."
  • According to a 2004 survey by The Chronicle, 71 percent of Americans thought it was very important for colleges to prepare undergraduates for careers, while only 56 percent thought it was very important for colleges to "discover more about the world through research."
  • Only 35 percent of respondents felt it was very important for colleges to "provide useful information to the public on issues affecting their daily lives."
  • What Texas A&M officials have also missed is that faculty members must be held accountable for what they teach.
  • Professors receive more credit for teaching higher-level students. But again, that is backward. The idea should be to give senior faculty members more credit for teaching introductory classes.
  • Moreover, the metric entirely ignores teaching quality. Who cares how many "student hours" professors put in if they are not particularly good teachers anyway?
  • Ultimately there needs to be a systemic solution to the problem of teacher quality. Someone—a grown-up, preferably—needs to get into the classroom and watch what is being done there.
  •  
    Another one in which the comments say more than I might--but the range of these accountability pieces underscore the work to do....
Gary Brown

Texas A&M System Will Rate Professors Based on Their Bottom-Line Value - Faculty - The ... - 2 views

  • Under the proposal, officials will add the money generated by each professor and subtract that amount from his or her salary to get a bottom-line value for each, according to the article.
  • the public wanted accountability. "It's something that we're really not used to in higher education: for someone questioning whether we're working hard, whether our students are learning. That accountability is going to be with us from now on."
  • American Association of University Professors, blamed a conservative think tank with ties to Gov. Rick Perry for coming up with an idea that he said is simplistic and relies on "a silly measure" of accountability.
  •  
    Nothing more to say about this....
  •  
    I would simply like to note the thoughtless slide from a desire to know "whether we're working hard, whether our students are learning" to revenue measures.
  •  
    Our colleagues in science disciplines, who had seen this, pointed out that unlike other institutions where this kind of system goes largely unspoken, at least at Texas AM there is some value included in the metric for those who teach undergraduates.
Joshua Yeidel

Blog U.: The Challenge of Value-Added - Digital Tweed - Inside Higher Ed - 0 views

  •  
    Quoting a 1984 study, "higher education should ensure that the mounds of data already collected on students are converted into useful information and fed back [to campus officials and faculty] in ways that enhance student learning and lead to improvement in programs, teaching practices, and the environment in which teaching and learning take place." The example given is an analysis of test scores in the Los Angeles Unified School District by the LA Times.
  •  
    It's going to take some assessment (and political) smarts to deflect the notion that existing data can be re-purposed easily to assess "value-added".
1 - 20 of 62 Next › Last »
Showing 20 items per page