Group items tagged

Pat Hutchings Nice summary of the barriers to faculty involvemetn in useful assessment, and a suggestion that we look at the specific cases where it has been successful, despite the barriers. This short piece might be a good prompt for discussion with chairs, deans, assessment committees. Do we have any really strong models?

CTLT and Friends

"Our web-based assessment solution for tracking, comparing, and reporting on student progress and performance gives faculty and administrators the tools they need to assess a class, department, or institution based on your standards, goals, or objectives. The Digication AMS integrates tightly with our award winning e-Portfolio system, enabling students to record and showcase learning outcomes within customizable, media friendly templates."

Could this start out as with program portfolios, and bgrow to include student work?

We should take this opportunity seriously and write up our work. Let me know if you want to join me.

Lumina Foundation pushes a Eurpopean-style process to define education goals state- and nation-wide, with mixed success. "Chemistry, history, math, and physics have been among the most successful", whileothers have had a hard time beginning. "Supporters of the Lumina project say it holds the promise of turning educational assessment from a process that some academics might view as a threat into one that holds a solution, while also creating more-rigorous expectations for student learning. Mr. Jones, the Utah State history-department chairman, recounted in an essay published in the American Historical Association's Perspectives on History how he once blithely told an accreditation team that "historians do not measure their effectiveness in outcomes." But he has changed his mind. The Lumina project, and others, help define what learning is achieved in the process of earning a degree, he said, moving beyond Americans' heavy reliance on the standardized student credit hour as the measure of an education. "The demand for outcomes assessment should be seized as an opportunity for us to actually talk about the habits of mind our discipline needs to instill in our students," Mr. Jones wrote. "It will do us a world of good, and it will save us from the spreadsheets of bureaucrats."

This is a critique of 1.0 assessment with few suggestions for remedy. Modernising assessment in the age of Web 2.0

"What is AHELO? The OECD Assessment of Higher Education Learning Outcomes (AHELO) is a ground-breaking initiative to assess learning outcomes on an international scale by creating measures that would be valid for all cultures and languages. Between ten and thirty-thousand higher education students in over ten different countries will take part in a feasibility study to determine the bounds of this ambitious project, with an eye to the possible creation of a full-scale AHELO upon its completion."

An interesting perspective on the future of assessment using the analogy of game design.

"Q: What elements help make institutional effectiveness assessment successful? Suskie: One factor really stands out: If institutional leaders really value assessment results and use them to inform important decisions on important goals, your institutional effectiveness efforts will be a resounding success."

"Despite those diverging starting points, the discussion revealed quite a bit more common ground than any of the panelists probably would have predicted. Let's be clear: Where they ended up was hardly a breakthrough on the scale of solving the Middle East puzzle. But there was general agreement among them that: * Any effort to try to measure learning in the humanities through what McCulloch-Lovell deemed "[Margaret] Spellings-type assessment" -- defined as tests or other types of measures that could be easily compared across colleges and neatly sum up many of the learning outcomes one would seek in humanities students -- was doomed to fail, and should. * It might be possible, and could be valuable, for humanists to reach broad agreement on the skills, abilities, and knowledge they might seek to instill in their students, and that agreement on those goals might be a starting point for identifying effective ways to measure how well students have mastered those outcomes. * It is incumbent on humanities professors and academics generally to decide for themselves how to assess whether their students are learning, less to satisfy external calls for accountability than because it is the right thing for academics, as professionals who care about their students, to do. "

Assessment meeting at the accreditors -- driven by expectations of a demand for accountability, with not one mention of improvement.

Another day, another press, again thinking comparisons

Kinship here, and familiar friends.

A featured article in Educause Quarterly contains this quote: "The importance of authentic, web-enabled learner assessment is clearly behind Caulfield's notion of "loosely coupled assessment" (first coined in a blog post by Mike Caulfield July 31, 2007) and WSU's harvesting gradebook project, with which we claim shared intellectual roots."

Video clips -- sometimes the copyright question comes up, so this green light is good news. Video clips may lend themselves to scenario-based assessments -- instead of reading a long article, students could look at a digitally presented case to analyze and critique -- might open up a lot of possibilities for assessment activities. a latest round of rule changes, issued Monday by the U.S. Copyright Office, dealing with what is legal and what is not as far as decrypting and repurposing copyrighted content. One change in particular is making waves in academe: an exemption that allows professors in all fields and "film and media studies students" to hack encrypted DVD content and clip "short portions" into documentary films and "non-commercial videos." (The agency does not define "short portions.") This means that any professors can legally extract movie clips and incorporate them into lectures, as long as they are willing to decrypt them - a task made relatively easy by widely available programs known as "DVD rippers." The exemption also permits professors to use ripped content in non-classroom settings that are similarly protected under "fair use" - such as presentations at academic conferences.

The challenge--not the control study this person calls for, but the perception that outcomes assessment produces outcomes....

"The professor said he did find a way to greatly reduce cheating on homework in his class. He switched to a "studio" model of teaching, in which students sit in small groups working through tutorials on computers while professors and teaching assistants roam the room answering questions, rather than a traditional lecture. With lectures, he detected cheating on about 11 percent of homework problems, but now he detects copying on only about 3 percent of them. It might help that he shares findings from his study to his students, showing them that cheaters are much more likely to get C's and D's on exams than those who work out homework problems on their own. "

A physics professor figures out how to detect cheating on homework, (an assessment), triangulates with another assessment (OK, it's an exam) to determine the impact on student learning, and modifies his teaching practice, reducing cheating from 11% to 3%.

This is a happy ending and an interesting useful outcome from assessment. There is also a kind of "duh" quality to it, but that is not a problem. What I'm pondering, though, is the time and energy expended for a 7% gain. What I hope follows is further assessment of the gains for the other 93% relative to learning and attitude toward learning, school, and the subject matter.....

"Assessing student learning outcomes relative to a valid and reliable standard that is academically-sound and employer-relevant presents a challenge to the scholarship of teaching and learning. In this paper, readers are guided through a method for collaboratively developing and validating a rubric that integrates baseline data collected from academics and professionals. The method addresses two additional goals: (1) to formulate and test a rubric as a teaching and learning protocol for a multi-section course taught by various instructors; and (2) to assure that students' learning outcomes are consistently assessed against the rubric regardless of teacher or section. Steps in the process include formulating the rubric, collecting data, and sequentially analyzing the techniques used to validate the rubric and to insure precision in grading papers in multiple sections of a course."

Since it is not clear if the incentive to participate in this study influenced the decision to participate, it remains similarly unclear if incentives to participate correlate with performance.

Note our friends Randy Bass (AAEEBL) and Mary Huber are prominent.

"What we want is for assessment to become a public, shared responsibility, so there should be departmental leadership." George Kuh director of the National Institute for Learning Outcomes Assessment.

Kuh also says, "So we're going to spend some time looking at the impact of the Voluntary System of Accountability. It's one thing for schools to sign up, it's another to post the information and to show that they're actually doing something with it. It's not about posting a score on a Web site-it's about doing something with the data." He doesn't take the next step and ask if it is even possible for schools to actually do anything with the data collected from the CLA or ask who has access to the criteria: Students? Faculty? Anyone?