CTLT and Friends

Data does not just happen; it is created in specific and deliberate ways. It is generated by sensors, not babies; and those sensors are designed to measure specific parameters for specific reasons, at certain rates, with certain resolutions. Or more correctly: it is gathered by people, for specific reasons, with a certain view of the world in mind, a certain concept of what the problem or the subject is. The people use the sensors, to gather the data, to measure a certain chosen aspect of the world.

If we come to accept that data just is, it's too easy to forget that it reflects a specific set of contexts, contingencies and choices, and that crucially, these could be (and maybe should be) different. Accepting data shaped by someone else's choices is a tacit acceptance of their view of the world, their notion of what is interesting or important or valid. Data is not inherent or intrinsic in anything: it is constructed, and if we are going to work intelligently with data we must remember that it can always be constructed some other way.

We need real, broad-based, practical and critical data skills and literacies, an understanding of how to make data and do things with it.

Levander and Mikkola (2009) describe the full complexity of managing alignment at the degree level which makes it difficult for the individual teacher and the program coordinator to keep connections between courses in mind.

Make explicit the quality model.

Build in support for quality enhancement.

Institute a process for quality feasibility.

Cohen (1987) argues that limitations in learning are not mainly caused by ineffective teaching, but are instead mostly the result of a misalignment between what teachers teach, what they intend to teach, and what they assess as having been taught.

Raban (2007) observes that the quality management systems of most universities employ procedures that are retrospective and weakly integrated with long term strategic planning. He continues to argue that the conventional quality management systems used by higher education are self-defeating as they undermine the commitment and motivation of academic staff through an apparent lack of trust, and divert resources away from the core activities of teaching and research (Raban, 2007, p. 78).

Ensure participation of formal institutional leadership and integration with institutional priorities.

Action research perspective, flexible responsive.

Having a scholarly, not bureaucratic focus.

Modifying an institutional information system.

A fundamental enabler of this project is the presence of an information system that is embedded into the everyday practice of teaching and learning (for both students and staff) that encourages and enables consideration of alignment.

"The evaluation part is critical. No matter which process we used, whether it was the [team] or hybrid model, they all did significantly worse than we hoped [in the evaluation stage]," Terwiesch says. "It's no good generating a great idea if you don't recognize the idea as great. It's like me sitting here and saying I had the idea for Amazon. If I had the idea but didn't do anything about it, then it really doesn't matter that I had the idea."

He says an online system that creates a virtual "suggestion box" can accomplish the same goal as long as it is established to achieve a particular purpose.

Imposing structure doesn't replace or stifle the creativity of employees, Ulrich adds. In fact, the goal is to establish an idea generation process that helps to bring out the best in people. "We have found that, in the early phases of idea generation, providing very specific process guideposts for individuals [such as] 'Generate at least 10 ideas and submit them by Wednesday,' ensures that all members of a team contribute and that they devote sufficient creative energy to the problem."

The results of the experiment with the students showed that average quality of the ideas generated by the hybrid process were better than those that came from the team process by the equivalent of roughly 30 percentage points.

in about three times more ideas than the traditional method.

"We find huge differences in people's levels of creativity, and we just have to face it. We're not all good singers and we're not all good runners, so why should we expect that we all are good idea generators?

They found that ideas built around other ideas are not statistically better than any random suggestion.

"In innovation, variance is your friend. You want wacky stuff because you can afford to reject it if you don't like it. If you build on group norms, the group kills variance."

"the worst example of government meddling at a huge cost to the public and for zero public good that I have ever seen."

"You get the feeling that the government sees us as slackers," she says. By requiring professors to list every assignment, she says the law interferes with her ability to respond to students' interests and current events and shift to different topics during the semester.

"If you have lower response rates, you're less inclined to make summative decisions about a faculty member's performance,"

While the majority of instructors still administer paper surveys, the number using online surveys increased from 1.08 percent in 2002 to 23.23 percent in 2008.

"I cannot provide you with the magic bullet that will help you recruit your students and make sure they are performing to the maximum of their ability," Mr. Steedle acknowledged to his audience at the Denver Convention Center. But, he said, his study results make clear that some recruitment strategies are more effective than others, and also offer some notes of caution for those examining students' scores.

The study focused on the council's Collegiate Learning Assessment, or CLA, an open-ended test of critical thinking and writing skills which is annually administered by several hundred colleges. Most of the colleges that use the test try to recruit 100 freshmen and 100 seniors to take it, but doing so can be daunting, especially for colleges that administer it in the spring, right when the seniors are focused on wrapping up their work and graduating.

The incentives that spurred students the least were the opportunity to help their college as an institution assess student learning, the opportunity to compare themselves to other students, a promise they would be recognized in some college publication, and the opportunity to put participation in the test on their resume.

The incentives which students preferred appeared to have no significant bearing on their performance. Those who appeared most inspired by a chance to earn 25 dollars did not perform better on the CLA than those whose responses suggested they would leap at the chance to help out a professor.

What accounted for differences in test scores? Students' academic ability going into the test, as measured by characteristics such as their SAT scores, accounted for 34 percent of the variation in CLA scores among individual students. But motivation, independent of ability, accounted for 5 percent of the variation in test scores—a finding that, the paper says, suggests it is "sensible" for colleges to be concerned that students with low motivation are not posting scores that can allow valid comparisons with other students or valid assessments of their individual strengths and weaknesses.

A major limitation of the study was that Mr. Steedle had no way of knowing how the students who took the test were recruited. "If many of them were recruited using cash and prizes, it would not be surprising if these students reported cash and prizes as the most preferable incentives," his paper concedes.