Skip to main content

Home/ CTLT and Friends/ Group items tagged evaluating teaching

Rss Feed Group items tagged

Filter: All | Bookmarks | Topics Simple Middle
Gary Brown

Evaluations That Make the Grade: 4 Ways to Improve Rating the Faculty - Teaching - The ... - 1 views

chronicle.com/...65226

evaluating teaching

shared by Gary Brown on 27 Apr 10 - Cached
  • For students, the act of filling out those forms is sometimes a fleeting, half-conscious moment. But for instructors whose careers can live and die by student evaluations, getting back the forms is an hour of high anxiety
  • "They have destroyed higher education." Mr. Crumbley believes the forms lead inexorably to grade inflation and the dumbing down of the curriculum.
  • Texas enacted a law that will require every public college to post each faculty member's student-evaluation scores on a public Web site.
  • ...10 more annotations...
  • The IDEA Center, an education research group based at Kansas State University, has been spreading its particular course-evaluation gospel since 1975. The central innovation of the IDEA system is that departments can tailor their evaluation forms to emphasize whichever learning objectives are most important in their discipline.
  • (Roughly 350 colleges use the IDEA Center's system, though in some cases only a single department or academic unit participates.)
  • The new North Texas instrument that came from these efforts tries to correct for biases that are beyond an instructor's control. The questionnaire asks students, for example, whether the classroom had an appropriate size and layout for the course. If students were unhappy with the classroom, and if it appears that their unhappiness inappropriately colored their evaluations of the instructor, the system can adjust the instructor's scores accordingly.
  • The survey instrument, known as SALG, for Student Assessment of their Learning Gains, is now used by instructors across the country. The project's Web site contains more than 900 templates, mostly for courses in the sciences.
  • "So the ability to do some quantitative analysis of these comments really allows you to take a more nuanced and effective look at what these students are really saying."
  • Mr. Frick and his colleagues found that his new course-evaluation form was strongly correlated with both students' and instructors' own measures of how well the students had mastered each course's learning goals.
  • Elaine Seymour, who was then director of ethnography and evaluation research at the University of Colorado at Boulder, was assisting with a National Science Foundation project to improve the quality of science instruction at the college level. She found that many instructors were reluctant to try new teaching techniques because they feared their course-evaluation ratings might decline.
  • "Students are the inventory," Mr. Crumbley says. "The real stakeholders in higher education are employers, society, the people who hire our graduates. But what we do is ask the inventory if a professor is good or bad. At General Motors," he says, "you don't ask the cars which factory workers are good at their jobs. You check the cars for defects, you ask the drivers, and that's how you know how the workers are doing."
  • William H. Pallett, president of the IDEA Center, says that when course rating surveys are well-designed and instructors make clear that they care about them, students will answer honestly and thoughtfully.
  • In Mr. Bain's view, student evaluations should be just one of several tools colleges use to assess teaching. Peers should regularly visit one another's classrooms, he argues. And professors should develop "teaching portfolios" that demonstrate their ability to do the kinds of instruction that are most important in their particular disciplines. "It's kind of ironic that we grab onto something that seems fixed and fast and absolute, rather than something that seems a little bit messy," he says. "Making decisions about the ability of someone to cultivate someone else's learning is inherently a messy process. It can't be reduced to a formula."
  • Gary Brown on 27 Apr 10
     
    Old friends at the Idea Center, and an old but persistent issue.
Gary Brown

Education ambivalence : Nature : Nature Publishing Group - 1 views

www.nature.com/...465525b.html

science undergraduate teaching

shared by Gary Brown on 02 Jun 10 - Cached
  • Academic scientists value teaching as much as research — but universities apparently don't
  • Nature Education, last year conducted a survey of 450 university-level science faculty members from more than 30 countries. The first report from that survey, freely available at http://go.nature.com/5wEKij, focuses on 'postsecondary' university- and college-level education. It finds that more than half of the respondents in Europe, Asia and North America feel that the quality of undergraduate science education in their country is mediocre, poor or very poor.
  • 77% of respondents indicated that they considered their teaching responsibilities to be just as important as their research — and 16% said teaching was more important.
  • ...6 more annotations...
  • But the biggest barrier to improvement is the pervasive perception that academic institutions — and the prevailing rewards structure of science — value research far more than teaching
  • despite their beliefs that teaching was at least as important as research, many respondents said that they would choose to appoint a researcher rather than a teacher to an open tenured position.
  • To correct this misalignment of values, two things are required. The first is to establish a standardized system of teaching evaluation. This would give universities and professors alike the feedback they need to improve.
  • The second requirement is to improve the support and rewards for university-level teaching.
  • systematic training in how to teach well
  • But by showering so many rewards on research instead of on teaching, universities and funding agencies risk undermining the educational quality that is required for research to flourish in the long term.
  • Gary Brown on 02 Jun 10
     
    Attention to this issue from this resource--Nature--is a breakthrough in its own right. Note the focus on "flourish in the long term...".
Gary Brown

News: Different Paths to Full Professor - Inside Higher Ed - 1 views

www.insidehighered.com/...osu

SoTL research Tenure and Promotion

shared by Gary Brown on 11 Mar 10 - Cached
  • Ohio State is embarking on discussions on how to change the way professors are evaluated for promotion to full professor. University officials argue that, as in tenure reviews, research appears to be the dominant factor at that stage, despite official policies to weigh teaching and service as well.
  • The concept in play would end the myth that candidates for full professor (and maybe, someday, candidates for tenure) should be great in everything. Why? Because most professors aren't great at everything.
  • Once research eminence is verified, teaching and service must be found only to be "adequate."
  • ...3 more annotations...
  • This approach is insidiously harmful," Alutto said. "First, it generates cynicism among productive faculty, as they realize the 'game' being played. Second, it frustrates productive faculty who contribute to their disciplines and the university in unique and powerful ways other than -- or in addition to -- traditional research. Third, it flies in the face of everything we know about the need for a balanced portfolio of skills to achieve institutional success."
  • Measuring impact is always difficult, particularly when it comes to teaching and service," he said. "But it can be done if we focus on the significance of these activities as it extends beyond our own institution -- just as we expect such broad effects with traditional scholarship. Thus, indicators of impact on other institutions, recognition by professional associations, broad adoption of teaching materials (textbooks, software, etc.) by other institutions, evidence of effects on policy formulation and so on -- all these are appropriate independent indicators of effectiveness."
  • Gerber said, the idea of "counting" such contributions in faculty evaluations is an embrace of Ernest Boyer's ideas about "the scholarship of teaching," ideas that have had much more influence outside research universities than within them.
  • Gary Brown on 11 Mar 10
     
    Reconsidering SoTL at Ohio State
  • Kimberly Green on 29 Mar 10
     
    Responding to this portion: This approach is insidiously harmful," Alutto said. "First, it generates cynicism among productive faculty, as they realize the 'game' being played. Second, it frustrates productive faculty who contribute to their disciplines and the university in unique and powerful ways other than -- or in addition to -- traditional research. Third, it flies in the face of everything we know about the need for a balanced portfolio of skills to achieve institutional success." How does OAI navigate these real concerns / hurdles with our program assessment efforts? If we convince/force leadership to "value" teaching and SoTL but it carries little or no weight in terms of promotion and tenure (I give you Carol Anelli, for example), then don't we become part of that "game"?
Gary Brown

What's Wrong With the American University System - Culture - The Atlantic - 3 views

www.theatlantic.com/...60458

higher education

shared by Gary Brown on 31 Jul 10 - Cached
  • But when the young superstar sat down with the department chair, he seemed to have only one goal: to land a tenure-track position that involved as many sabbaticals and as little teaching as possible
  • Hacker and his coauthor, New York Times writer Claudia Dreifus, use this cautionary tale to launch their new book, a fierce critique of modern academia called Higher Education? "The question mark in our title," they write, "is the key to this book." To their minds, little of what takes place on college campuses today can be considered either "higher" or "education."
  • They blame a system that favors research over teaching and vocational training over liberal arts.
  • ...10 more annotations...
  • Tenure, they argue, does anything but protect intellectual freedom
  • Schools get status by bringing on professors who are star researchers, star scholars. That's all we really know about Caltech or MIT or Stanford. We don't really know about the quality of undergraduate teaching at any of these places. And it's the students who suffer.
  • Claudia and I were up at Harvard talking to students, and they said they get nothing from their classes, but that doesn't matter. They're smart already—they can breeze through college. The point is that they're going to be Harvard people when they come out.
  • So tenure is, in fact, the enemy of spontaneity, the enemy of intellectual freedom.
  • Good teaching can't be quantified at the college level.
  • or instance, Evergreen College, a sweet little state school in Olympia, Washington. We spent three days there and it was fantastic. They don't give grades, and they don't have academic departments. There are no faculty rankings. Almost all the classes we saw were taught by two professors—say, one from philosophy and one from psychology, teaching jointly on Henry and William James. Even though they don't give grades, the professors write out long evaluations for students. And the students have no problem getting into graduate schools.
  • I like Missouri Western State. It's a third-tier university, but the faculty realize they're going to stay there, they're not going to get hired away by other colleges, so they pitch in and take teaching seriously. At a school like that, you have a decent chance of finding a mentor who will write you a strong recommendation, better than you would at Harvard.
  • We believe the current criteria for admissions—particularly the SAT—are just so out of whack. It's like No Child Left Behind. It really is. It's one of the biggest crimes that's ever been perpetrated.
  • Professor X. He argued that some students just aren't ready for college. What's your view on that? Our view is that the primary obligation belongs to the teacher. Good teaching is not just imparting knowledge, like pouring milk into a jug. It's the job of the teacher to get students interested and turned on no matter what the subject is. Every student can be turned on if teachers really engage in this way. We saw it at Evergreen and other places that have this emphasis.
  • This is the hand I was dealt this semester. This is my job." Some people say to me, "Your students at Queens, are they any good?" I say, "I make them good." Every student is capable of college. I know some people have had difficult high school educations. But if you have good teachers who really care, it's remarkable how you can make up the difference.
  • Matthew Tedder on 31 Jul 10
     
    In case you haven't already seen this.  While don't deny higher education needs attention, I personal wish there'd be far more attention paid to lower education and regressive education (my own term for, redressing and improving the education of all U.S. citizens).  We are in the process of destroying our country and our world.  Education as at the very heart of any solution.
  • Gary Brown on 11 Aug 10
     
    More of the discussion in the news--the Atlantic
Gary Brown

Professors Who Focus on Honing Their Teaching Are a Distinct Breed - Research - The Chr... - 1 views

chronicle.com/...65262

teaching

shared by Gary Brown on 27 Apr 10 - Cached
  • Professors who are heavily focused on learning how to improve their teaching stand apart as a very distinct subset of college faculties, according to a new study examining how members of the professoriate spend their time.
  • those who are focused on tackling societal problems stand apart as their own breed. Other faculty members, it suggests, are pretty much mutts, according to its classification scheme.
  • 1,000 full-time faculty members at four-year colleges and universities gathered as part of the Faculty Professional Performance Survey administered by Mr. Braxton and two Vanderbilt doctoral students in 1999. That survey had asked the faculty members how often they engaged in each of nearly 70 distinct scholarly activities, such as experimenting with a new teaching method, publishing a critical book review in a journal, or being interviewed on a local television station. All of the faculty members examined in the new analysis were either tenured or tenure-track and fell into one of four academic disciplines: biology, chemistry, history, or sociology.
  • ...7 more annotations...
  • cluster analysis,
  • nearly two-thirds of those surveyed were involved in the full range of scholarly activity they examined
  • Just over a third, however, stood out as focused almost solely on one of two types of scholarship: on teaching practices, or on using knowledge from their discipline to identify or solve societal problems.
  • pedagogy-focused scholars were found mainly at liberal-arts colleges and, compared with the general population surveyed, tended to be younger, heavily represented in history departments, and more likely to be female and untenured
  • Those focused on problem-solving were located mainly at research and doctoral institutions, and were evenly dispersed across disciplines and more likely than others surveyed to be male and tenured.
  • how faculty members rate those priorities are fairly consistent across academic disciplines,
  • The study was conducted by B. Jan Middendorf, acting director of Kansas State University's office of educational innovation and evaluation; Russell J. Webster, a doctoral student in psychology at Kansas State; and Steve Benton, a senior research officer at the IDEA Center
  • Gary Brown on 27 Apr 10
     
    Another study that documents the challenge and suggests confirmation of the 50% figure of faculty who are not focused on either research or teaching.
Theron DesRosier

Education Data Model (National Forum on Education Statistics). Strategies for building ... - 0 views

nces.ed.gov/...aboutThe.aspx

data education

shared by Theron DesRosier on 11 Mar 10 - Cached
  • Theron DesRosier on 11 Mar 10
     
    "The National Education Data Model is a conceptual but detailed representation of the education information domain focused at the student, instructor and course/class levels. It delineates the relationships and interdependencies between the data elements necessary to document, operate, track, evaluate, and improve key aspects of an education system. The NEDM strives to be a shared understanding among all education stakeholders as to what information needs to be collected and managed at the local level in order to enable effective instruction of students and superior leadership of schools. It is a comprehensive, non-proprietary inventory and a map of education information that can be used by schools, LEAs, states, vendors, and researchers to identify the information required for teaching, learning, administrative systems, and evaluation of education programs and approaches. "
Gary Brown

Texas A&M's Faculty Ratings: Right and Wrong - Commentary - The Chronicle of Higher Edu... - 0 views

chronicle.com/...124330

accountability

shared by Gary Brown on 08 Sep 10 - No Cached
  • "Academia is highly specialized. We don't mean to be exclusive. We are a public-serving group of people. But at the same time, that public isn't well-enough aware of what we do and who we are to evaluate us."
  • But the think tank is correct that taxpayers deserve to know how their money is being spent. Public-university operating costs in Texas have gone up more than 60 percent in the last two decades, even after adjusting for inflation, and professors are among the state's highest-paid public employees. The state needs accountability measures, and they must be enforced by a party other than the faculty, who, it could easily be charged, have a conflict of interest. That's what Texas A&M got right.
  • Moosally is right about one thing: The public isn't well aware of what she and many of her colleagues do. But they should be. That is not to say that the public will be able to understand what goes on in all of the chemistry laboratories in Texas. But Moosally teaches English at a college that is not exactly tasked with performing cutting-edge research. Houston-Downtown's mission is to provide "educational opportunities and access to students from a variety of backgrounds including many first-generation college students."
  • ...7 more annotations...
  • No doubt there is useful research coming out of the university system. But plenty could be omitted without a great deal of detriment to students' education. For instance, Hugill's most recent contributions have included a chapter on "Transitions in Hegemony: A Theory Based on State Type and Technology" and the article "German Great-Power Relations in the Pages of Simplicissimus, 1896-1914." Moosally's master's thesis was titled "Resumptive Pronouns in Modern Standard Arabic: A Head-Driven Phrase Structure Grammar Account," and her current research interests include "interactions between grammar knowledge and writing abilities/interest [and] cross-linguistic patterns of agreement."
  • Only 35 percent of respondents felt it was very important for colleges to "provide useful information to the public on issues affecting their daily lives."
  • According to a 2004 survey by The Chronicle, 71 percent of Americans thought it was very important for colleges to prepare undergraduates for careers, while only 56 percent thought it was very important for colleges to "discover more about the world through research."
  • What Texas A&M officials have also missed is that faculty members must be held accountable for what they teach.
  • Professors receive more credit for teaching higher-level students. But again, that is backward. The idea should be to give senior faculty members more credit for teaching introductory classes.
  • Moreover, the metric entirely ignores teaching quality. Who cares how many "student hours" professors put in if they are not particularly good teachers anyway?
  • Ultimately there needs to be a systemic solution to the problem of teacher quality. Someone—a grown-up, preferably—needs to get into the classroom and watch what is being done there.
  • Gary Brown on 08 Sep 10
     
    Another one in which the comments say more than I might--but the range of these accountability pieces underscore the work to do....
Gary Brown

As Colleges Switch to Online Course Evaluations, Students Stop Filling Them Out - The T... - 1 views

chronicle.com/...22376

evaluating teaching

shared by Gary Brown on 09 Apr 10 - Cached
  • Colleges thought they were enhancing efficiency when they moved their course evaluations online, but an unintended consequence of the shift to evaluations not filled out in class is that students started skipping them altogether, The Boston Globe reported today.
  • Gary Brown on 09 Apr 10
     
    the letters are more interesting than the article--especially the Boston Globe link where Mazur weighs in.
  • Kimberly Green on 12 Apr 10
     
    The whole issue of online evals is a big one, esp relevant as part of prog eval. Perhaps a design circle topic? (...I couldn't find Mazur's comments)
Nils Peterson

Views: Changing the Equation - Inside Higher Ed - 1 views

www.insidehighered.com/...bassis

shared by Nils Peterson on 25 Mar 10 - Cached
  • But each year, after some gnashing of teeth, we opted to set tuition and institutional aid at levels that would maximize our net tuition revenue. Why? We were following conventional wisdom that said that investing more resources translates into higher quality and higher quality attracts more resources
  • But each year, after some gnashing of teeth, we opted to set tuition and institutional aid at levels that would maximize our net tuition revenue. Why? We were following conventional wisdom that said that investing more resources translates into higher quality and higher quality attracts more resource
  • But each year, after some gnashing of teeth, we opted to set tuition and institutional aid at levels that would maximize our net tuition revenue. Why? We were following conventional wisdom that said that investing more resources translates into higher quality and higher quality attracts more resources
  • ...19 more annotations...
  • year we strug
  • year we strug
  • those who control influential rating systems of the sort published by U.S. News & World Report -- define academic quality as small classes taught by distinguished faculty, grand campuses with impressive libraries and laboratories, and bright students heavily recruited. Since all of these indicators of quality are costly, my college’s pursuit of quality, like that of so many others, led us to seek more revenue to spend on quality improvements. And the strategy worked.
  • Based on those concerns, and informed by the literature on the “teaching to learning” paradigm shift, we began to change our focus from what we were teaching to what and how our students were learning.
  • No one wants to cut costs if their reputation for quality will suffer, yet no one wants to fall off the cliff.
  • When quality is defined by those things that require substantial resources, efforts to reduce costs are doomed to failure
  • some of the best thinkers in higher education have urged us to define the quality in terms of student outcomes.
  • Faculty said they wanted to move away from giving lectures and then having students parrot the information back to them on tests. They said they were tired of complaining that students couldn’t write well or think critically, but not having the time to address those problems because there was so much material to cover. And they were concerned when they read that employers had reported in national surveys that, while graduates knew a lot about the subjects they studied, they didn’t know how to apply what they had learned to practical problems or work in teams or with people from different racial and ethnic backgrounds.
  • Our applications have doubled over the last decade and now, for the first time in our 134-year history, we receive the majority of our applications from out-of-state students.
  • We established what we call college-wide learning goals that focus on "essential" skills and attributes that are critical for success in our increasingly complex world. These include critical and analytical thinking, creativity, writing and other communication skills, leadership, collaboration and teamwork, and global consciousness, social responsibility and ethical awareness.
  • despite claims to the contrary, many of the factors that drive up costs add little value. Research conducted by Dennis Jones and Jane Wellman found that “there is no consistent relationship between spending and performance, whether that is measured by spending against degree production, measures of student engagement, evidence of high impact practices, students’ satisfaction with their education, or future earnings.” Indeed, they concluded that “the absolute level of resources is less important than the way those resources are used.”
  • After more than a year, the group had developed what we now describe as a low-residency, project- and competency-based program. Here students don’t take courses or earn grades. The requirements for the degree are for students to complete a series of projects, captured in an electronic portfolio,
  • students must acquire and apply specific competencies
  • Faculty spend their time coaching students, providing them with feedback on their projects and running two-day residencies that bring students to campus periodically to learn through intensive face-to-face interaction
  • At the very least, finding innovative ways to lower costs without compromising student learning is wise competitive positioning for an uncertain future
  • As the campus learns more about the demonstration project, other faculty are expressing interest in applying its design principles to courses and degree programs in their fields. They created a Learning Coalition as a forum to explore different ways to capitalize on the potential of the learning paradigm.
  • a problem-based general education curriculum
  • After a year and a half, the evidence suggests that students are learning as much as, if not more than, those enrolled in our traditional business program
  • the focus of student evaluations has changed noticeably. Instead of focusing almost 100% on the instructor and whether he/she was good, bad, or indifferent, our students' evaluations are now focusing on the students themselves - as to what they learned, how much they have learned, and how much fun they had learning.
    • Nils Peterson
      Nils Peterson on 25 Mar 10
       
      gary diigoed this article. this comment shines another light -- the focus of the course eval shifted from faculty member to course & student learning when the focus shifted from teaching to learning
  • Gary Brown on 25 Mar 10
     
    A must read spotted by Jane Sherman--I've highlighed, as usual, much of it.
Gary Brown

Details | LinkedIn - 0 views

www.linkedin.com/groupItem

accountability

shared by Gary Brown on 11 Sep 10 - No Cached
  • Although different members of the academic hierarchy take on different roles regarding student learning, student learning is everyone’s concern in an academic setting. As I specified in my article comments, universities would do well to use their academic support units, which often have evaluation teams (or a designated evaluator) to assist in providing boards the information they need for decision making. Perhaps boards are not aware of those serving in evaluation roles at the university or how those staff members can assist boards in their endeavors.
  • Gary Brown • We have been using the Internet to post program assessment plans and reports (the programs that support this initiative at least), our criteria (rubric) for reviewing them, and then inviting external stakeholders to join in the review process.
Joshua Yeidel

Evaluations That Make the Grade: 4 Ways to Improve Rating the Faculty - Teaching - The ... - 0 views

chronicle.com/...65226

evaluations assessment

shared by Joshua Yeidel on 27 Apr 10 - Cached
  • Joshua Yeidel on 27 Apr 10
     
    Four (somewhat) different types of course evals, closing with a tip of the hat to multiple measures.
Nils Peterson

2009 Annual Meeting | Conference Program - 0 views

www.aacu.org/...program.cfm

shared by Nils Peterson on 22 Jan 09 - Cached
  • This session explores the notion that assessment for transformational learning is best utilized as a learning tool. By providing timely, transparent, and appropriate feedback, both to students and to the institution itself, learning is enhanced – a far different motive for assessment than is external accountability.
    • Nils Peterson
      Nils Peterson on 22 Jan 09
       
      need to get to these guys with our harvesting gradebook ideas...
    • Nils Peterson
      Nils Peterson on 22 Jan 09
       
      decided to attend another session. Hersh was OK before lunch, but the talk by Pan looks more promising
  • Academic and corporate communities agree on the urgent need for contemporary, research-based pedagogies of engagement in STEM fields. Participants will learn how leaders from academic departments and institutions have collaborated with leaders from the corporate and business community in regional networks to ensure that graduates meet the expectations of prospective employers and the public.
    • Nils Peterson
      Nils Peterson on 22 Jan 09
       
      here is another session with links to CTLT work, both harvesting gradebook and the ABET work
  • Professor Pan will discuss the reflective teaching methods used to prepare students to recognize and mobilize community assets as they design, implement, and evaluate projects to improve public health.
    • Nils Peterson
      Nils Peterson on 22 Jan 09
       
      Students tasked to learn about a community, ride the bus, make a Doc appt. Then tasked to do a non-clinical health project in that community (they do plenty of clinical stuff elsewhere in the program). Project must build capacity in the community to survive after the student leaves. Example. Work with hispanic parents in Sacramento about parenting issue, ex getting kids to sleep on time. Student had identified problem in the community, but first project idea was show a video, which was not capacity building. Rather than showing the video, used the video as a template and made a new video. Families were actors. Result was spanish DVD that the community could own. Pan thinks this is increased capacity in the community.
  • ...17 more annotations...
  • Freshman Survey annually examines the academic habits of mind of entering first-year students.  Along with academic involvement, the survey examines diversity, civic engagement, college admissions and expectations of college. 
  • The project aims to promote faculty and student assessment of undergraduate research products in relation to outcomes associated with basic research skills and general undergraduate learning principles (communication and quantitative reasoning, critical thinking, and integration and application of knowledge).
  • They focus educators on the magnitude of the challenge to prepare an ever-increasingly diverse, globally-connected student body with the knowledge, ability, processes, and confidence to adapt to diverse environments and respond creatively to the enormous issues facing humankind.
  • One challenge of civic engagement in the co-curriculum is the merging of cost and outcome: creating meaningful experiences for students and the community with small staffs, on small budgets, while still having significant, purposeful impact. 
  • a)claims that faculty are the sole arbiters of what constitutes a liberal education and b) counter claims that student life professionals also possess the knowledge and expertise critical to defining students’ total learning experiences.  
    • Nils Peterson
      Nils Peterson on 22 Jan 09
       
      also, how many angels can dance on the head of a pin?
  • This session introduces a three-year national effort to document how colleges and universities are using assessment data to improve teaching and learning and to facilitate the dissemination and adoption of best practices in the assessment of college learning outcomes.
  • Exciting pedagogies of engagement abound, including undergraduate research, community-engaged learning, interdisciplinary exploration, and international study.  However, such experiences are typically optional and non-credit-bearing for students, and/or “on top of” the workload for faculty. This session explores strategies for integrating engaged learning into the institutional fabric (curriculum, student role, faculty role) and increasing access to these transformative experiences.
  • hands-on experiential learning, especially in collaboration with other students, is a superior pedagogy but how can this be provided in increasingly larger introductory classes? 
  • As educators seek innovative ways to manage knowledge and expand interdisciplinary attention to pressing global issues, as students and parents look for assurances that their tuition investment will pay professional dividends, and as alumni look for meaningful ways to give back to the institutions that nurtured and prepared them, colleges and universities can integrate these disparate goals through the Guilds, intergenerational membership networks that draw strength from the contributions of all of their members.
    • Nils Peterson
      Nils Peterson on 22 Jan 09
       
      see Theron's ideas for COMM.
  • Civic engagement learning derives its power from the engagement of students with real communities—local, national, and global. This panel explores the relationship between student learning and the contexts in which that learning unfolds by examining programs that place students in diverse contexts close to campus and far afield.
  • For institutional assessment to make a difference for student learning its results must result in changes in classroom practice. This session explores ways in which the institutional assessment of student learning, such as the Wabash National Study of Liberal Arts Education and the Collegiate Learning Assessment, can be connected to our classrooms.
  • Interdisciplinary Teaching and Object-Based Learning in Campus Museums
  • To address pressing needs of their communities, government and non-profit agencies are requesting higher education to provide education in an array of human and social services. To serve these needs effectively, higher educationneeds to broaden and deepen its consultation with practitioners in designing new curricula. Colleges and universities would do well to consider a curriculum development model that requires consultation not only with potential employers, but also with practitioners and supervisors of practitioners.
  • Should Academics be Active? Campuses and Cutting Edge Civic Engagement
  • If transformational liberal education requires engaging the whole student across the educational experience, how can colleges and universities renew strategy and allocate resources effectively to support it?  How can assessment be used to improve student learning and strengthen a transformational learning environment? 
    • Nils Peterson
      Nils Peterson on 22 Jan 09
       
      Purpose of university is not to grant degrees, it has something to do with learning. Keeling's perspective is that the learning should be transformative; changing perspective. Liberating and emancipatory Learning is a complex interaction among student and others, new knowledge and experience, event, own aspirations. learners construct meaning from these elements. "we change our minds" altering the brain at the micro-level Brain imaging research demonstrates that analogical learning (abstract) demands more from more areas of the brain than semantic (concrete) learning. Mind is not an abstraction, it is based in the brain, a working physical organ .Learner and the environment matter to the learning. Seeds magazine, current issue on brain imaging and learning. Segway from brain research to need for university to educate the whole student. Uses the term 'transformative learning' meaning to transform the learning (re-wire the brain) but does not use transformative assessment (see wikipedia).
  • But as public debates roil, higher education has been more reactive than proactive on the question of how best to ensure that today’s students are fully prepared for a fast-paced future.
    • Nils Peterson
      Nils Peterson on 22 Jan 09
       
      Bologna process being adopted (slowly) in EU, the idea is to make academic degrees more interchangeable and understandable across the EU three elements * Qualification Frameworks (transnational, national, disciplinary). Frameworks are graduated, with increasing expertise and autonomy required for the upper levels. They sound like broad skills that we might recognize in the WSU CITR. Not clear how they are assessed * Tuning (benchmarking) process * Diploma Supplements (licensure, thesis, other capstone activities) these extend the information in the transcript. US equivalent might be the Kuali Students system for extending the transcript. Emerging dialog on American capability This dialog is coming from 2 directions * on campus * employers Connect to the Greater Exceptions (2000-2005) iniative. Concluded that American HE has islands of innovation. Lead to LEAP (Liberal Education and America's Promise) Initiative (2005-2015). The dialog is converging because of several forces * Changes in the balance of economic and political power. "The rise of the rest (of the world)" * Global economy in which innovation is key to growth and prosperity LEAP attempts to frame the dialog (look for LEAP in AACU website). Miami-Dade CC has announced a LEAP-derived covenant, the goals must span all aspects of their programs. Define liberal education Knowledge of human cultures and the physical and natural world intellectual and practical skills responsibility integrative skills Marker of success is (here is where the Transformative Gradebook fits in): evidence that students can apply the essential learning outcomes to complex, unscripted problems and real-world settings Current failure -- have not tracked our progress, or have found that we are not doing well. See AACU employer survey 5-10% percent of current graduates taking courses that would meet the global competencies (transcript analysis) See NSSE on Personal and social responsibility gains, less tha
  • Dr. Pan will also talk about strategies for breaking down cultural barriers.
    • Nils Peterson
      Nils Peterson on 24 Jan 09
       
      Pan. found a non-profit agency to be a conduit and coordinator to level the power between univ and grass roots orgs. helped with cultural gaps.
Nils Peterson

AAC&U News | April 2010 | Feature - 1 views

www.aacu.org/...feature.cfm

shared by Nils Peterson on 01 Apr 10 - Cached
  • Comparing Rubric Assessments to Standardized Tests
  • First, the university, a public institution of about 40,000 students in Ohio, needed to comply with the Voluntary System of Accountability (VSA), which requires that state institutions provide data about graduation rates, tuition, student characteristics, and student learning outcomes, among other measures, in the consistent format developed by its two sponsoring organizations, the Association of Public and Land-grant Universities (APLU), and the Association of State Colleges and Universities (AASCU).
  • And finally, UC was accepted in 2008 as a member of the fifth cohort of the Inter/National Coalition for Electronic Portfolio Research, a collaborative body with the goal of advancing knowledge about the effect of electronic portfolio use on student learning outcomes.  
  • ...13 more annotations...
  • outcomes required of all UC students—including critical thinking, knowledge integration, social responsibility, and effective communication
  • “The wonderful thing about this approach is that full-time faculty across the university  are gathering data about how their  students are doing, and since they’ll be teaching their courses in the future, they’re really invested in rubric assessment—they really care,” Escoe says. In one case, the capstone survey data revealed that students weren’t doing as well as expected in writing, and faculty from that program adjusted their pedagogy to include more writing assignments and writing assessments throughout the program, not just at the capstone level. As the university prepares to switch from a quarter system to semester system in two years, faculty members are using the capstone survey data to assist their course redesigns, Escoe says.
  • the university planned a “dual pilot” study examining the applicability of electronic portfolio assessment of writing and critical thinking alongside the Collegiate Learning Assessment,
  • The rubrics the UC team used were slightly modified versions of those developed by AAC&U’s Valid Assessment of Learning in Undergraduate Education (VALUE) project. 
  • In the critical thinking rubric assessment, for example, faculty evaluated student proposals for experiential honors projects that they could potentially complete in upcoming years.  The faculty assessors were trained and their rubric assessments “normed” to ensure that interrater reliability was suitably high.
  • “It’s not some nitpicky, onerous administrative add-on. It’s what we do as we teach our courses, and it really helps close that assessment loop.”
  • There were many factors that may have contributed to the lack of correlation, she says, including the fact that the CLA is timed, while the rubric assignments are not; and that the rubric scores were diagnostic and included specific feedback, while the CLA awarded points “in a black box”:
  • faculty members may have had exceptionally high expectations of their honors students and assessed the e-portfolios with those high expectations in mind—leading to results that would not correlate to a computer-scored test. 
  • “The CLA provides scores at the institutional level. It doesn’t give me a picture of how I can affect those specific students’ learning. So that’s where rubric assessment comes in—you can use it to look at data that’s compiled over time.”
  • Their portfolios are now more like real learning portfolios, not just a few artifacts, and we want to look at them as they go into their third and fourth years to see what they can tell us about students’ whole program of study.”  Hall and Robles are also looking into the possibility of forming relationships with other schools from NCEPR to exchange student e-portfolios and do a larger study on the value of rubric assessment of student learning.
  • “We’re really trying to stress that assessment is pedagogy,”
  • “We found no statistically significant correlation between the CLA scores and the portfolio scores,”
  • In the end, Escoe says, the two assessments are both useful, but for different things. The CLA can provide broad institutional data that satisfies VSA requirements, while rubric-based assessment provides better information to facilitate continuous program improvement.
    • Nils Peterson
      Nils Peterson on 01 Apr 10
       
      CLA did not provide information for continuous program improvement -- we've heard this argument before
  • Gary Brown on 01 Apr 10
     
    The lack of correlation might be rephrased--there appears to be no corrlation between what is useful for faculty who teach and what is useful for the VSA. A corollary question: Of what use is the VSA?
Gary Brown

Outsourced Grading, With Supporters and Critics, Comes to College - Teaching - The Chro... - 3 views

chronicle.com/...64954

grades

shared by Gary Brown on 06 Apr 10 - Cached
  • Lori Whisenant knows that one way to improve the writing skills of undergraduates is to make them write more. But as each student in her course in business law and ethics at the University of Houston began to crank out—often awkwardly—nearly 5,000 words a semester, it became clear to her that what would really help them was consistent, detailed feedback.
  • She outsourced assignment grading to a company whose employees are mostly in Asia.
  • The graders working for EduMetry, based in a Virginia suburb of Washington, are concentrated in India, Singapore, and Malaysia, along with some in the United States and elsewhere. They do their work online and communicate with professors via e-mail.
  • ...8 more annotations...
  • The company argues that professors freed from grading papers can spend more time teaching and doing research.
  • "This is what they do for a living," says Ms. Whisenant. "We're working with professionals." 
  • Assessors are trained in the use of rubrics, or systematic guidelines for evaluating student work, and before they are hired are given sample student assignments to see "how they perform on those," says Ravindra Singh Bangari, EduMetry's vice president of assessment services.
  • Professors give final grades to assignments, but the assessors score the papers based on the elements in the rubric and "help students understand where their strengths and weaknesses are," says Tara Sherman, vice president of client services at EduMetry. "Then the professors can give the students the help they need based on the feedback."
  • The assessors use technology that allows them to embed comments in each document; professors can review the results (and edit them if they choose) before passing assignments back to students.
  • But West Hills' investment, which it wouldn't disclose, has paid off in an unexpected way. The feedback from Virtual-TA seems to make the difference between a student's remaining in an online course and dropping out.
  • Because Virtual-TA provides detailed comments about grammar, organization, and other writing errors in the papers, students have a framework for improvement that some instructors may not be able to provide, she says.
  • "People need to get past thinking that grading must be done by the people who are teaching," says Mr. Rajam, who is director of assurance of learning at George Washington University's School of Business. "Sometimes people get so caught up in the mousetrap that they forget about the mouse."
Ashley Ater Kranov

Course Reminds Budding Ph.D.'s of the Damage They Can Do - Teaching - The Chronicle of ... - 1 views

chronicle.com/...48916

teaching learning assessment

shared by Ashley Ater Kranov on 30 Oct 09 - Cached
  • Ashley Ater Kranov on 30 Oct 09
     
    I think most of us are aware of and agree with what this author is positing - the purpose in sharing this is related to our work with departments in re-thinking how they think about teaching and evaluate it. ""People often think that education works either to improve you or to leave you as you were," Mr. Cahn says. "But that's not right. An unsuccessful education can ruin you. It can kill your interest in a topic. It can make you a less-good thinker. It can leave you less open to rational argument. So we do good and bad as teachers-it's not just good or nothing.""
Gary Brown

Texas Law Requires Professors to Post Details of Their Teaching Online - Faculty - The ... - 1 views

chronicle.com/...65450

student evaluations accreditation

shared by Gary Brown on 07 May 10 - Cached
  • Faculty members and administrators in Texas are speaking out about a recent state law that requires them to post specific, detailed information about their classroom assignments, curricula vitae, department budgets, and the results of student evaluations.
  • Beginning this fall, universities will have to post online a syllabus for every undergraduate course, including major assignments and examinations, reading lists, and course descriptions.
  • All of the information must be no more than three clicks away from the college's home page.
  • ...2 more annotations...
  • "the worst example of government meddling at a huge cost to the public and for zero public good that I have ever seen."
  • "You get the feeling that the government sees us as slackers," she says. By requiring professors to list every assignment, she says the law interferes with her ability to respond to students' interests and current events and shift to different topics during the semester.
  • Gary Brown on 07 May 10
     
    another to watch--the politization of the, well, everything
Nils Peterson

National Institute for Learning Outcomes Assessment - 1 views

learningoutcomesassessment.org/occasionalpapertwo.htm

NILOA Trudy Banta

shared by Nils Peterson on 18 Feb 10 - Cached
  • Of the various ways to assess student learning outcomes, many faculty members prefer what are called “authentic” approaches that document student performance during or at the end of a course or program of study.  Authentic assessments typically ask students to generate rather than choose a response to demonstrate what they know and can do.  In their best form, such assessments are flexible and closely aligned with teaching and learning processes, and represent some of students more meaningful educational experiences.  In this paper, assessment experts Trudy Banta, Merilee Griffin, Theresa Flateby, and Susan Kahn describe the development of several promising authentic assessment approaches. 
  • Educators and policy makers in postsecondary education are interested in assessment processes that improve student learning, and at the same time provide comparable data for the purpose of demonstrating accountability.
  • First, ePortfolios provide an in-depth, long-term view of student achievement on a range of skills and abilities instead of a quick snapshot based on a single sample of learning outcomes. Second, a system of rubrics used to evaluate student writing and depth of learning has been combined with faculty learning and team assessments, and is now being used at multiple institutions. Third, online assessment communities link local faculty members in collaborative work to develop shared norms and teaching capacity, and then link local communities with each other in a growing system of assessment.
    • Nils Peterson
      Nils Peterson on 18 Feb 10
       
      hey, does this sound familiar? i'm guessing the portfolios are not anywhere on the Internet, but we're otherwise in good company
  • ...1 more annotation...
  • Three Promising Alternatives for Assessing College Students' Knowledge and Skills
    • Nils Peterson
      Nils Peterson on 18 Feb 10
       
      I'm not sure they are 'alternatives' so much as 3 elements we would combine into a single strategy
Joshua Yeidel

Scholar Raises Doubts About the Value of a Test of Student Learning - Research - The Ch... - 3 views

chronicle.com/...65741

assessment accountability

shared by Joshua Yeidel on 02 Jun 10 - Cached
  • Beginning in 2011, the 331 universities that participate in the Voluntary System of Accountability will be expected to publicly report their students' performance on one of three national tests of college-level learning.
  • But at least one of those three tests—the Collegiate Learning Assessment, or CLA—isn't quite ready to be used as a tool of public accountability, a scholar suggested here on Tuesday during the annual meeting of the Association for Institutional Research.
  • Students' performance on the test was strongly correlated with how long they spent taking it.
  • ...6 more annotations...
  • Besides the CLA, which is sponsored by the Council for Aid to Education, other tests that participants in the voluntary system may use are the Collegiate Assessment of Academic Proficiency, from ACT Inc., and the Measure of Academic Proficiency and Progress, offered by the Educational Testing Service.
  • The test has sometimes been criticized for relying on a cross-sectional system rather than a longitudinal model, in which the same students would be tested in their first and fourth years of college.
  • there have long been concerns about just how motivated students are to perform well on the CLA.
  • Mr. Hosch suggested that small groups of similar colleges should create consortia for measuring student learning. For example, five liberal-arts colleges might create a common pool of faculty members that would evaluate senior theses from all five colleges. "That wouldn't be a national measure," Mr. Hosch said, "but it would be much more authentic."
  • Mr. Shavelson said. "The challenge confronting higher education is for institutions to address the recruitment and motivation issues if they are to get useful data. From my perspective, we need to integrate assessment into teaching and learning as part of students' programs of study, thereby raising the stakes a bit while enhancing motivation of both students and faculty
  • "I do agree with his central point that it would not be prudent to move to an accountability system based on cross-sectional assessments of freshmen and seniors at an institution," said Mr. Arum, who is an author, with Josipa Roksa, of Academically Adrift: Limited Learning on College Campuses, forthcoming from the University of Chicago Press
  • Gary Brown on 02 Jun 10
     
    CLA debunking, but the best item may be the forthcoming book on "limited learning on College Campuses."
  • Joshua Yeidel on 25 Jun 10
     
    "Micheal Scriven and I spent more than a few years trying to apply his multiple-ranking item tool (a very robust and creative tool, I recommend it to others when the alternative is multiple-choice items) to the assessment of critical thinking in health care professionals. The result might be deemed partially successful, at best. I eventually abandoned the test after about 10,000 administrations because the scoring was so complex we could not place it in non-technical hands."
  • Joshua Yeidel on 25 Jun 10
     
    In comments on an article about CLA, Scriven's name comes up...
Gary Brown

The Future of Wannabe U. - The Chronicle Review - The Chronicle of Higher Education - 2 views

chronicle.com/...124917

outcomes numbers

shared by Gary Brown on 18 Oct 10 - No Cached
  • Alice didn't tell me about the topics of her research; instead she listed the number of articles she had written, where they had been submitted and accepted, the reputation of the journals, the data sets she was constructing, and how many articles she could milk from each data set.
  • colleges and universities have transformed themselves from participants in an audit culture to accomplices in an accountability regime.
  • higher education has inaugurated an accountability regime—a politics of surveillance, control, and market management that disguises itself as value-neutral and scientific administration.
  • ...7 more annotations...
  • annabe administrator noted that the recipient had published well more than 100 articles. He never said why those articles mattered.
  • And all we have are numbers about teaching. And we don't know what the difference is between a [summary measure of] 7.3 and a 7.7 or an 8.2 and an 8.5."
  • The problem is that such numbers have no meaning. They cannot indicate the quality of a student's education.
  • or can the many metrics that commonly appear in academic (strategic) plans, like student credit hours per full-time-equivalent faculty member, or the percentage of classes with more than 50 students. Those productivity measures (for they are indeed productivity measures) might as well apply to the assembly-line workers who fabricate the proverbial widget, for one cannot tell what the metrics have to do with the supposed purpose of institutions of higher education—to create and transmit knowledge. That includes leading students to the possibility of a fuller life and an appreciation of the world around them and expanding their horizons.
  • But, like the fitness club's expensive cardio machines, a significant increase in faculty research, in the quality of student experiences (including learning), in the institution's service to its state, or in its standing among its peers may cost more than a university can afford to invest or would even dream of paying.
  • Such metrics are a speedup of the academic assembly line, not an intensification or improvement of student learning. Indeed, sometimes a boost in some measures, like an increase in the number of first-year students participating in "living and learning communities," may even detract from what students learn. (Wan U.'s pre-pharmacy living-and-learning community is so competitive that students keep track of one another's grades more than they help one another study. Last year one student turned off her roommate's alarm clock so that she would miss an exam and thus no longer compete for admission to the School of Pharmacy.)
  • Even metrics intended to indicate what students may have learned seem to have more to do with controlling faculty members than with gauging education. Take student-outcomes assessments, meant to be evaluations of whether courses have achieved their goals. They search for fault where earlier researchers would not have dreamed to look. When parents in the 1950s asked why Johnny couldn't read, teachers may have responded that it was Johnny's fault; they had prepared detailed lesson plans. Today student-outcomes assessment does not even try to discover whether Johnny attended class; instead it produces metrics about outcomes without considering Johnny's input.
  • Gary Brown on 18 Oct 10
     
    A good one to wrestle with.  It may be worth formulating distinctions we hold, and steering accordingly.
Nils Peterson

Change Magazine - The New Guys in Assessment Town - 0 views

www.changemag.org/...full-assessment-town.html

shared by Nils Peterson on 05 May 09 - Cached
  • if one of the institution’s general education goals is critical thinking, the system makes it possible to call up all the courses and programs that assess student performance on that outcome.
  • bringing together student learning outcomes data at the level of the institution, program, course, and throughout student support services so that “the data flows between and among these levels”
  • Like its competitors, eLumen maps outcomes vertically across courses and programs, but its distinctiveness lies in its capacity to capture what goes on in the classroom. Student names are entered into the system, and faculty use a rubric-like template to record assessment results for every student on every goal. The result is a running record for each student available only to the course instructor (and in a some cases to the students themselves, who can go to the system to  get feedback on recent assessments).
    • Nils Peterson
      Nils Peterson on 05 May 09
       
      sounds like harvesting gradebook. assess student work and roll up
    • Joshua Yeidel
      Joshua Yeidel on 05 May 09
       
      This system has some potential for formative use at the per-student leve.
  • ...7 more annotations...
  • “I’m a little wary.  It seems as if, in addition to the assessment feedback we are already giving to students, we might soon be asked to add a data-entry step of filling in boxes in a centralized database for all the student learning outcomes. This is worrisome to those of us already struggling under the weight of all that commenting and essay grading.”
    • Nils Peterson
      Nils Peterson on 05 May 09
       
      its either double work, or not being understood that the grading and the assessment can be the same activity. i suspect the former -- grading is being done with different metrics
    • Joshua Yeidel
      Joshua Yeidel on 05 May 09
       
      I am in the unusual position of seeing many papers _after_ they have been graded by a wide variety of teachers. Many of these contain little "assessment feedback" -- many teachers focus on "correcting" the papers and finding some letter or number to assign as a value.
  • “This is where we see many institutions struggling,” Galvin says. “Faculty simply don’t have the time for a deeper involvement in the mechanics of assessment.” Many have never seen a rubric or worked with one, “so generating accurate, objective data for analysis is a challenge.”  
    • Nils Peterson
      Nils Peterson on 05 May 09
       
      Rather than faculty using the community to help with assessment, they are outsourcing to a paid assessor -- this is the result of undertaking this thinking while also remaining in the institution-centric end of the spectrum we developed
  • I asked about faculty pushback. “Not so much,” Galvin says, “not after faculty understand that the process is not intended to evaluate their work.”
    • Nils Peterson
      Nils Peterson on 05 May 09
       
      red flag
  • the annual reports required by this process were producing “heaps of paper” while failing to track trends and developments over time. “It’s like our departments were starting anew every year,” Chaplot says. “We wanted to find a way to house the data that gave us access to what was done in the past,” which meant moving from discrete paper reports to an electronic database.
    • Joshua Yeidel
      Joshua Yeidel on 05 May 09
       
      It's not clear whether the "database" is housing measurements, narratives and reflections, or all of the above.
  • Can eLumen represent student learning in language? No, but it can quantify the number of boxes checked against number of boxes not checked.”
  • developing a national repository of resources, rubrics, outcomes statements, and the like that can be reviewed and downloaded by users
    • Nils Peterson
      Nils Peterson on 05 May 09
       
      in building our repository we could well open-source these tools, no need to lock them up
  • “These solutions cement the idea that assessment is an administrative rather than an educational enterprise, focused largely on accountability. They increasingly remove assessment decision making from the everyday rhythm of teaching and learning and the realm of the faculty.
    • Nils Peterson
      Nils Peterson on 06 May 09
       
      Over the wall assessment, see Transformative Assessment rubric for more detail
1 - 20 of 21 Next ›
Showing 20▼ items per page