CTLT and Friends

an elaborate database that will help them track the success (or lack thereof) of various instructional techniques.

The IDEA Center, an education research group based at Kansas State University, has been spreading its particular course-evaluation gospel since 1975. The central innovation of the IDEA system is that departments can tailor their evaluation forms to emphasize whichever learning objectives are most important in their discipline.

(Roughly 350 colleges use the IDEA Center's system, though in some cases only a single department or academic unit participates.)

The new North Texas instrument that came from these efforts tries to correct for biases that are beyond an instructor's control. The questionnaire asks students, for example, whether the classroom had an appropriate size and layout for the course. If students were unhappy with the classroom, and if it appears that their unhappiness inappropriately colored their evaluations of the instructor, the system can adjust the instructor's scores accordingly.

Elaine Seymour, who was then director of ethnography and evaluation research at the University of Colorado at Boulder, was assisting with a National Science Foundation project to improve the quality of science instruction at the college level. She found that many instructors were reluctant to try new teaching techniques because they feared their course-evaluation ratings might decline.

"So the ability to do some quantitative analysis of these comments really allows you to take a more nuanced and effective look at what these students are really saying."

Mr. Frick and his colleagues found that his new course-evaluation form was strongly correlated with both students' and instructors' own measures of how well the students had mastered each course's learning goals.

The survey instrument, known as SALG, for Student Assessment of their Learning Gains, is now used by instructors across the country. The project's Web site contains more than 900 templates, mostly for courses in the sciences.

"Students are the inventory," Mr. Crumbley says. "The real stakeholders in higher education are employers, society, the people who hire our graduates. But what we do is ask the inventory if a professor is good or bad. At General Motors," he says, "you don't ask the cars which factory workers are good at their jobs. You check the cars for defects, you ask the drivers, and that's how you know how the workers are doing."

William H. Pallett, president of the IDEA Center, says that when course rating surveys are well-designed and instructors make clear that they care about them, students will answer honestly and thoughtfully.

In Mr. Bain's view, student evaluations should be just one of several tools colleges use to assess teaching. Peers should regularly visit one another's classrooms, he argues. And professors should develop "teaching portfolios" that demonstrate their ability to do the kinds of instruction that are most important in their particular disciplines. "It's kind of ironic that we grab onto something that seems fixed and fast and absolute, rather than something that seems a little bit messy," he says. "Making decisions about the ability of someone to cultivate someone else's learning is inherently a messy process. It can't be reduced to a formula."

“Student behaviors have changed,” Mr. Kabbaz said. “The question becomes: Have we institutionally changed our habits of engaging these students?”

The NRC project's directors say that those small samples are not a problem, because the reputational scores were not converted directly into program assessments. Instead, the scores were used to develop a profile of the kinds of traits that faculty members value in doctoral programs in their field.

For one thing, Mr. Stigler says, the relationships between programs' reputations and the various program traits are probably not simple and linear.

if these correlations between reputation and citations were plotted on a graph, the most accurate representation would be a curved line, not a straight line. (The curve would occur at the tipping point where high citation levels make reputations go sky-high.)

Mr. Stigler says that it was a mistake for the NRC to so thoroughly abandon the reputational measures it used in its previous doctoral studies, in 1982 and 1995. Reputational surveys are widely criticized, he says, but they do provide a check on certain kinds of qualitative measures.

The Change Process as Problem Solving and Problem Finding

What is not useful about this framework is that it does not allow for change efforts that begin with the organization in extremis

this framework is that it gives rise to thinking about a staged approach to changing things.

Change as a “How” Problem

Change as a “What” Problem

Change as a “Why” Problem

The Approach taken to Change Management Mirrors Management's Mindset

People in core units, buffered as they are from environmental turbulence and with a history of relying on adherence to standardized procedures, typically focus on “how” questions.

To summarize: Problems may be formulated in terms of “how,” “what” and “why” questions. Which formulation is used depends on where in the organization the person posing the question or formulating the problem is situated, and where the organization is situated in its own life cycle. “How” questions tend to cluster in core units. “What” questions tend to cluster in buffer units. People in perimeter units tend to ask “what” and “how” questions. “Why” questions are typically the responsibility of top management.

One More Time: How do you manage change? The honest answer is that you manage it pretty much the same way you’d manage anything else of a turbulent, messy, chaotic nature, that is, you don’t really manage it, you grapple with it. It’s more a matter of leadership ability than management skill. The first thing to do is jump in. You can’t do anything about it from the outside. A clear sense of mission or purpose is essential. The simpler the mission statement the better. “Kick ass in the marketplace” is a whole lot more meaningful than “Respond to market needs with a range of products and services that have been carefully designed and developed to compare so favorably in our customers’ eyes with the products and services offered by our competitors that the majority of buying decisions will be made in our favor.” Build a team. “Lone wolves” have their uses, but managing change isn’t one of them. On the other hand, the right kind of lone wolf makes an excellent temporary team leader. Maintain a flat organizational team structure and rely on minimal and informal reporting requirements. Pick people with relevant skills and high energy levels. You’ll need both. Toss out the rulebook. Change, by definition, calls for a configured response, not adherence to prefigured routines. Shift to an action-feedback model. Plan and act in short intervals. Do your analysis on the fly. No lengthy up-front studies, please. Remember the hare and the tortoise. Set flexible priorities. You must have the ability to drop what you’re doing and tend to something more important. Treat everything as a temporary measure. Don’t “lock in” until the last minute, and then insist on the right to change your mind. Ask for volunteers. You’ll be surprised at who shows up. You’ll be pleasantly surprised by what they can do. Find a good “straw boss” or team leader and stay out of his or her way. Give the team members whatever they ask for — except authority. They’ll generally ask only for what they really need in the way of resources. If they start asking for authority, that’s a signal they’re headed toward some kind of power-based confrontation and that spells trouble. Nip it in the bud! Concentrate dispersed knowledge. Start and maintain an issues logbook. Let anyone go anywhere and talk to anyone about anything. Keep the communications barriers low, widely spaced, and easily hurdled. Initially, if things look chaotic, relax — they are. Remember, the task of change management is to bring order to a messy situation, not pretend that it’s already well organized and disciplined.

But the research aspirations of many large universities are in conflict with their founding principles, Ms. Wellman says, especially as undergraduate admissions has become more selective

another result of the chase for research dollars is that measures for faculty assessment and promotion rely too heavily on the research output and publication and too little on the quality of classroom teaching.

"I'm not pushing for banning research," he says, but there should be more flexibility and balance in the criteria."

In recent years, however, it's become clear that daydreaming is actually an important element of the creative process, allowing the brain to remix ideas, explore counterfactuals and turn the spotlight of attention inwards.

The danger, however, is that we're teaching our kids a very narrow and stultifying model of cognition, in which conscientiousness is privileged above all.

The solution, I suppose, is rather banal: we really do need arts education in our schools,

include them in their tenure-and promotion portfolios.

As Ernest Boyer said, faculty reward systems will need to be revised in order for faculty members to truly be rewarded on the basis of their scholarship of teaching."

But the biggest barrier to improvement is the pervasive perception that academic institutions — and the prevailing rewards structure of science — value research far more than teaching

despite their beliefs that teaching was at least as important as research, many respondents said that they would choose to appoint a researcher rather than a teacher to an open tenured position.

To correct this misalignment of values, two things are required. The first is to establish a standardized system of teaching evaluation. This would give universities and professors alike the feedback they need to improve.

The second requirement is to improve the support and rewards for university-level teaching.

systematic training in how to teach well

But by showering so many rewards on research instead of on teaching, universities and funding agencies risk undermining the educational quality that is required for research to flourish in the long term.

We need designers who understand that learning comes from experience, practice, conversations and reflection, and are prepared to move away from massaging content into what they see as good instructional design. Designers need to get off the content bus and start thinking about, using, designing and exploiting learning environments full of experiences and interactivity.