Group items tagged

Nothing is more compelling than data from actual workplace settings, but getting it is usually very hard."

Instead, the job without the feedback attracted more workers -- 254, compared with 76 for the job with feedback.

"This indicates that when people are great and they know it, they tend to slack off. But when they're at the bottom, and are told they're doing terribly, they are de-motivated," says Barankay.

In the second stage of the experiment

The aim was to determine whether giving people feedback affected their desire to do more work, as well as the quantity and quality of their work.

Of the workers in the control group, 66% came back for more work, compared with 42% in the treatment group. The members of the treatment group who returned were also 22% less productive than the control group. This seems to dispel the notion that giving people feedback might encourage high-performing workers to work harder to excel, and inspire low-ranked workers to make more of an effort.

it seems that people would rather not know how they rank compared to others, even though when we surveyed these workers after the experiment, 74% said they wanted feedback about their rank."

top performers move on to new challenges and low performers have no viable options elsewhere.

feedback about rank is detrimental to performance,"

it is well documented that tournaments, where rankings are tied to prizes, bonuses and promotions, do inspire higher productivity and performance.

"In workplaces where rankings and relative performance is very transparent, even without the intervention of management ... it may be better to attach financial incentives to rankings, as interpersonal comparisons without prizes may lead to lower effort," Barankay suggests. "In those office environments where people may not be able to assess and compare the performance of others, it may not be useful to just post a ranking without attaching prizes."

"The key is to devote more time to thinking about whether to give feedback, and how each individual will respond to it. If, as the employer, you think a worker will respond positively to a ranking and feel inspired to work harder, then by all means do it. But it's imperative to think about it on an individual level."

Whether the audit will actually work remains to be seen. Many of the people who attended the meeting expressed deep skepticism and unease about how effectively a rigorous and independent audit procedure could be applied.

rankings have become an intensely competitive business, and that any audit procedures would need to be clear and open enough to ensure that competitors were not pronouncing on one another's work.

I forgot that students can’t provide a legitimate opinion about their institutions. What do they know about them? I guess not much since their points of view are flatly ignored.

The NRC project's directors say that those small samples are not a problem, because the reputational scores were not converted directly into program assessments. Instead, the scores were used to develop a profile of the kinds of traits that faculty members value in doctoral programs in their field.

For one thing, Mr. Stigler says, the relationships between programs' reputations and the various program traits are probably not simple and linear.

if these correlations between reputation and citations were plotted on a graph, the most accurate representation would be a curved line, not a straight line. (The curve would occur at the tipping point where high citation levels make reputations go sky-high.)

Mr. Stigler says that it was a mistake for the NRC to so thoroughly abandon the reputational measures it used in its previous doctoral studies, in 1982 and 1995. Reputational surveys are widely criticized, he says, but they do provide a check on certain kinds of qualitative measures.

A great paradox of modern academe is that our institutions take pride in being on the cutting edge of new ideas and innovations, yet remain resistant and even hostile to the openness made possible by technology

We should not hide our departments' deficiencies in debatable rankings, but rather be honest about those limitations in order to aggressively pursue solutions that will strengthen doctoral programs and the institutions in which they play a vital role.

Besides the CLA, which is sponsored by the Council for Aid to Education, other tests that participants in the voluntary system may use are the Collegiate Assessment of Academic Proficiency, from ACT Inc., and the Measure of Academic Proficiency and Progress, offered by the Educational Testing Service.

The test has sometimes been criticized for relying on a cross-sectional system rather than a longitudinal model, in which the same students would be tested in their first and fourth years of college.

there have long been concerns about just how motivated students are to perform well on the CLA.

Mr. Hosch suggested that small groups of similar colleges should create consortia for measuring student learning. For example, five liberal-arts colleges might create a common pool of faculty members that would evaluate senior theses from all five colleges. "That wouldn't be a national measure," Mr. Hosch said, "but it would be much more authentic."

Mr. Shavelson said. "The challenge confronting higher education is for institutions to address the recruitment and motivation issues if they are to get useful data. From my perspective, we need to integrate assessment into teaching and learning as part of students' programs of study, thereby raising the stakes a bit while enhancing motivation of both students and faculty

"I do agree with his central point that it would not be prudent to move to an accountability system based on cross-sectional assessments of freshmen and seniors at an institution," said Mr. Arum, who is an author, with Josipa Roksa, of Academically Adrift: Limited Learning on College Campuses, forthcoming from the University of Chicago Press

Ms. Asher was also a champion of reviewing the financial incentives for colleges.  “We need to shift incentives for colleges to focus on outcomes for students.”

when our students attend three or four schools, the subject of these sentences is considerably weakened in terms of what happens to those students.

Who or what is one accountable to?

For what?

Why that particular “what” -- and not another “what”?

To what extent is the relationship reciprocal? Are there rewards and/or sanctions inherent in the relationship? How continuous is the relationship?

In the Socratic moral universe, one is simultaneously witness and judge. The Greek syneidesis (“conscience” and “consciousness”) means to know something with, so to know oneself with oneself becomes an obligation of institutions and systems -- to themselves.

Obligation becomes self-reflexive.

There are no external authorities here. We offer, we accept, we provide evidence, we judge. There is nothing wrong with this: it is indispensable, reflective self-knowledge. And provided we judge without excuses, we hold to this Socratic moral framework. As Peter Ewell has noted, the information produced under this rubric, particularly in the matter of student learning, is “part of our accountability to ourselves.”

But is this “accountability” as the rhetoric of higher education uses the white noise -- or something else?

in response to shrill calls for “accountability,” U.S. higher education has placed all its eggs in the Socratic basket, but in a way that leaves the basket half-empty. It functions as the witness, providing enormous amounts of information, but does not judge that information.

Every single “best practice” cited by Aldeman and Carey is subject to measurement: labor market histories of graduates, ratios of resource commitment to various student outcomes, proportion of students in learning communities or taking capstone courses, publicly-posted NSSE results, undergraduate research participation, space utilization rates, licensing income, faculty patents, volume of non-institutional visitors to art exhibits, etc. etc. There’s nothing wrong with any of these, but they all wind up as measurements, each at a different concentric circle of putatively engaged acceptees of a unilateral contract to provide evidence. By the time one plows through Aldeman and Carey’s banquet, one is measuring everything that moves -- and even some things that don’t.

Sorry, but basic capacity facts mean that consumers cannot vote with their feet in higher education.

If we glossed the Socratic notion on provision-of-information, the purpose is self-improvement, not comparison. The market approach to accountability implicitly seeks to beat Socrates by holding that I cannot serve as both witness and judge of my own actions unless the behavior of others is also on the table. The self shrinks: others define the reference points. “Accountability” is about comparison and competition, and an institution’s obligations are only to collect and make public those metrics that allow comparison and competition. As for who judges the competition, we have a range of amorphous publics and imagined authorities.

There are no formal agreements here: this is not a contract, it is not a warranty, it is not a regulatory relationship. It isn’t even an issue of becoming a Socratic self-witness and judge. It is, instead, a case in which one set of parties, concentrated in places of power, asks another set of parties, diffuse and diverse, “to disclose more and more about academic results,” with the second set of parties responding in their own terms and formulations. The environment itself determines behavior.

Ewell is right about the rules of the information game in this environment: when the provider is the institution, it will shape information “to look as good as possible, regardless of the underlying performance.”

U.S. News & World Report’s rankings

The messengers become self-appointed arbiters of performance, establishing themselves as the second party to which institutions and aggregates of institutions become “accountable.” Can we honestly say that the implicit obligation of feeding these arbiters constitutes “accountability”?

But if the issue is student learning, there is nothing wrong with -- and a good deal to be said for -- posting public examples of comprehensive examinations, summative projects, capstone course papers, etc. within the information environment, and doing so irrespective of anyone requesting such evidence of the distribution of knowledge and skills. Yes, institutions will pick what makes them look good, but if the public products resemble AAC&U’s “Our Students’ Best Work” project, they set off peer pressure for self-improvement and very concrete disclosure. The other prominent media messengers simply don’t engage in constructive communication of this type.

Ironically, a “market” in the loudest voices, the flashiest media productions, and the weightiest panels of glitterati has emerged to declare judgment on institutional performance in an age when student behavior has diluted the very notion of an “institution” of higher education. The best we can say is that this environment casts nothing but fog over the specific relationships, responsibilities, and obligations that should be inherent in something we call “accountability.” Perhaps it is about time that we defined these components and their interactions with persuasive clarity. I hope that this essay will invite readers to do so.

Clifford Adelman is senior associate at the Institute for Higher Education Policy. The analysis and opinions expressed in this essay are those of the author, and do not necessarily represent the positions or opinions of the institute, nor should any such representation be inferred.

Tenure, they argue, does anything but protect intellectual freedom

Schools get status by bringing on professors who are star researchers, star scholars. That's all we really know about Caltech or MIT or Stanford. We don't really know about the quality of undergraduate teaching at any of these places. And it's the students who suffer.

Claudia and I were up at Harvard talking to students, and they said they get nothing from their classes, but that doesn't matter. They're smart already—they can breeze through college. The point is that they're going to be Harvard people when they come out.

So tenure is, in fact, the enemy of spontaneity, the enemy of intellectual freedom.

Good teaching can't be quantified at the college level.

or instance, Evergreen College, a sweet little state school in Olympia, Washington. We spent three days there and it was fantastic. They don't give grades, and they don't have academic departments. There are no faculty rankings. Almost all the classes we saw were taught by two professors—say, one from philosophy and one from psychology, teaching jointly on Henry and William James. Even though they don't give grades, the professors write out long evaluations for students. And the students have no problem getting into graduate schools.

I like Missouri Western State. It's a third-tier university, but the faculty realize they're going to stay there, they're not going to get hired away by other colleges, so they pitch in and take teaching seriously. At a school like that, you have a decent chance of finding a mentor who will write you a strong recommendation, better than you would at Harvard.

We believe the current criteria for admissions—particularly the SAT—are just so out of whack. It's like No Child Left Behind. It really is. It's one of the biggest crimes that's ever been perpetrated.

Professor X. He argued that some students just aren't ready for college. What's your view on that? Our view is that the primary obligation belongs to the teacher. Good teaching is not just imparting knowledge, like pouring milk into a jug. It's the job of the teacher to get students interested and turned on no matter what the subject is. Every student can be turned on if teachers really engage in this way. We saw it at Evergreen and other places that have this emphasis.

This is the hand I was dealt this semester. This is my job." Some people say to me, "Your students at Queens, are they any good?" I say, "I make them good." Every student is capable of college. I know some people have had difficult high school educations. But if you have good teachers who really care, it's remarkable how you can make up the difference.

State and federal governments are better equipped to enforce consumer protections for students, say accreditors, who have traditionally focused on preserving academic quality.

Judith S. Eaton, president of the Council for Higher Education Accreditation, which represents about 3,000 colleges, said that over the past several years accrediting organizations have responded to the growing calls for accountability and transparency from the public and lawmakers. The groups, she said, have worked to better identify and judge student achievement and share more information about what they do and how well the institutions are performing.

Peter T. Ewell, vice president of the National Center for Higher Education Management Systems, said the debate boils down to whether accreditors should serve primarily as consumer protectors or continue their traditional role of monitoring academic quality more broadly.

Richard K. Vedder, director of the Center for College Affordability & Productivity and a member of the Spellings Commission

"We should be moving to more of a Consumer Reports for colleges, to provide the public with information that the college rankings do imperfectly," he said.

accreditation will have to evolve to meet not only government's expectations but also the changing college

market

Nearly two years into the Obama Administration, colleges have not gotten the relief they expected from the contentious battles over measuring quality that defined the Bush Education Department.

Bracing for the prospect of new rules and laws that could expand their responsibilities, accreditors and the institutions they monitor are defending the self-regulation colleges use to ensure academic quality. But they are also responding to the pressures from the White House and Capitol Hill by making some changes on their own, hoping to stanch the possibility of more far-reaching federal requirements.

Advocates of change say the six regional and seven national accreditors have varying standards that are sometimes too lax, allowing for limited oversight of how credits are awarded, how much learning is accomplished, and what happens to the mission of institutions that change owners.

Boards should expect to receive useful high-level information on learning outcomes, the report says, and should make comparisons over time and to other institutions. Training in how to understand academic and learning assessments should also be part of orientation for new board members.