Group items tagged

When asked the simple question of what school standards are, most students are unable to answer the question. When the concept is explained, however, they really do not know if having common standards would make a difference or not. Codie Allen, a senior in the Vail School Distract says, “I think that things will pretty much stay stagnate, people aren’t really going to change because of standards.”

Council of Chief State School Officers. Common Core State Standards Initiative, 2010.

"The Board of Regents here saw that it would be an important test because it measures analytical ability, problem-solving ability, critical thinking, and communication. Those are the skills that you want every undergraduate to walk away with." (Other large systems that have embraced the CLA include California State University and the West Virginia system.)

value added

We began by administering a retired CLA question, a task that had to do with analyzing crime-reduction strategies,

performance task that mirrors the CLA

Mr. Ernsting and Ms. McConnell are perfectly sincere about using CLA-style tasks to improve instruction on their campuses. But at the same time, colleges have a less high-minded motive for familiarizing students with the CLA style: It just might improve their scores when it comes time to take the actual test.

by 2012, the CLA scores of more than 100 colleges will be posted, for all the world to see, on the "College Portrait" Web site of the Voluntary System of Accountability, an effort by more than 300 public colleges and universities to provide information about life and learning on their campuses.

If familiarizing students with CLA-style tasks does raise their scores, then the CLA might not be a pure, unmediated reflection of the full range of liberal-arts skills. How exactly should the public interpret the scores of colleges that do not use such training exercises?

Trudy W. Banta, a professor of higher education and senior adviser to the chancellor for academic planning and evaluation at Indiana University-Purdue University at Indianapolis, believes it is a serious mistake to publicly release and compare scores on the test. There is too much risk, she says, that policy makers and the public will misinterpret the numbers.

most colleges do not use a true longitudinal model: That is, the students who take the CLA in their first year do not take it again in their senior year. The test's value-added model is therefore based on a potentially apples-and-oranges comparison.

freshman test-takers' scores are assessed relative to their SAT and ACT scores, and so are senior test-takers' scores. For that reason, colleges cannot game the test by recruiting an academically weak pool of freshmen and a strong pool of seniors.

students do not always have much motivation to take the test seriously

seniors, who are typically recruited to take the CLA toward the end of their final semester, when they can already taste the graduation champagne.

Of the few dozen universities that had already chosen to publish CLA data on that site, roughly a quarter of the reports appeared to include erroneous descriptions of the year-to-year value-added scores.

It is clear that CLA scores do reflect some broad properties of a college education.

Students' CLA scores improved if they took courses that required a substantial amount of reading and writing. Many students didn't take such courses, and their CLA scores tended to stay flat.

Colleges that make demands on students can actually develop their skills on the kinds of things measured by the CLA.

Mr. Shavelson believes the CLA's essays and "performance tasks" offer an unusually sophisticated way of measuring what colleges do, without relying too heavily on factual knowledge from any one academic field.

Politicians and consumers want easily interpretable scores, while colleges need subtler and more detailed data to make internal improvements.

The CLA is used at more than 400 colleges

Since its debut a decade ago, it has been widely praised as a sophisticated alternative to multiple-choice tests

The Change Process as Problem Solving and Problem Finding

What is not useful about this framework is that it does not allow for change efforts that begin with the organization in extremis

this framework is that it gives rise to thinking about a staged approach to changing things.

Change as a “How” Problem

Change as a “What” Problem

Change as a “Why” Problem

The Approach taken to Change Management Mirrors Management's Mindset

People in core units, buffered as they are from environmental turbulence and with a history of relying on adherence to standardized procedures, typically focus on “how” questions.

To summarize: Problems may be formulated in terms of “how,” “what” and “why” questions. Which formulation is used depends on where in the organization the person posing the question or formulating the problem is situated, and where the organization is situated in its own life cycle. “How” questions tend to cluster in core units. “What” questions tend to cluster in buffer units. People in perimeter units tend to ask “what” and “how” questions. “Why” questions are typically the responsibility of top management.

One More Time: How do you manage change? The honest answer is that you manage it pretty much the same way you’d manage anything else of a turbulent, messy, chaotic nature, that is, you don’t really manage it, you grapple with it. It’s more a matter of leadership ability than management skill. The first thing to do is jump in. You can’t do anything about it from the outside. A clear sense of mission or purpose is essential. The simpler the mission statement the better. “Kick ass in the marketplace” is a whole lot more meaningful than “Respond to market needs with a range of products and services that have been carefully designed and developed to compare so favorably in our customers’ eyes with the products and services offered by our competitors that the majority of buying decisions will be made in our favor.” Build a team. “Lone wolves” have their uses, but managing change isn’t one of them. On the other hand, the right kind of lone wolf makes an excellent temporary team leader. Maintain a flat organizational team structure and rely on minimal and informal reporting requirements. Pick people with relevant skills and high energy levels. You’ll need both. Toss out the rulebook. Change, by definition, calls for a configured response, not adherence to prefigured routines. Shift to an action-feedback model. Plan and act in short intervals. Do your analysis on the fly. No lengthy up-front studies, please. Remember the hare and the tortoise. Set flexible priorities. You must have the ability to drop what you’re doing and tend to something more important. Treat everything as a temporary measure. Don’t “lock in” until the last minute, and then insist on the right to change your mind. Ask for volunteers. You’ll be surprised at who shows up. You’ll be pleasantly surprised by what they can do. Find a good “straw boss” or team leader and stay out of his or her way. Give the team members whatever they ask for — except authority. They’ll generally ask only for what they really need in the way of resources. If they start asking for authority, that’s a signal they’re headed toward some kind of power-based confrontation and that spells trouble. Nip it in the bud! Concentrate dispersed knowledge. Start and maintain an issues logbook. Let anyone go anywhere and talk to anyone about anything. Keep the communications barriers low, widely spaced, and easily hurdled. Initially, if things look chaotic, relax — they are. Remember, the task of change management is to bring order to a messy situation, not pretend that it’s already well organized and disciplined.

"It is worth pausing to ask how the proponents of outcome measures can be so very confident that the actual performance of tasks deemed essential for the practice of law can be identified, measured, and evaluated," said Robert C. Post, dean of Yale Law School.

Phillip A. Bradley, senior vice president and general counsel for Duane Reade, a large drugstore chain, likened law schools to car companies that are "manufacturing something that nobody wants." Mr. Bradley said many law firms are developing core competencies they expect of their lawyers, but many law schools aren't delivering graduates who come close to meeting them.

And—as Footnote One softly but incessantly reminds us—quality, at its core, must be a measure of what students actually learn and are able to do with the knowledge and skills they gain.

and yet we seem reluctant or unable to discuss higher education's true purpose: equipping students for success in life.

Research has already shown that higher education institutions vary significantly in the value they add to students in terms of what those students actually learn. Various tools and instruments tell us that some institutions add much more value than others, even when looking at students with similar backgrounds and abilities.

The idea with tuning is to take various programs within a specific discipline—chemistry, history, psychology, whatever—and agree on a set of learning outcomes that a degree in the field represents. The goal is not for the various programs to teach exactly the same thing in the same way or even for all of the programs to offer the same courses. Rather, programs can employ whatever techniques they prefer, so long as their students can demonstrate mastery of an agreed-upon body of knowledge and set of skills. To use the musical terminology, the various programs are not expected to play the same notes, but to be "tuned" to the same key.

when our students attend three or four schools, the subject of these sentences is considerably weakened in terms of what happens to those students.

Who or what is one accountable to?

For what?

Why that particular “what” -- and not another “what”?

To what extent is the relationship reciprocal? Are there rewards and/or sanctions inherent in the relationship? How continuous is the relationship?

In the Socratic moral universe, one is simultaneously witness and judge. The Greek syneidesis (“conscience” and “consciousness”) means to know something with, so to know oneself with oneself becomes an obligation of institutions and systems -- to themselves.

Obligation becomes self-reflexive.

There are no external authorities here. We offer, we accept, we provide evidence, we judge. There is nothing wrong with this: it is indispensable, reflective self-knowledge. And provided we judge without excuses, we hold to this Socratic moral framework. As Peter Ewell has noted, the information produced under this rubric, particularly in the matter of student learning, is “part of our accountability to ourselves.”

But is this “accountability” as the rhetoric of higher education uses the white noise -- or something else?

in response to shrill calls for “accountability,” U.S. higher education has placed all its eggs in the Socratic basket, but in a way that leaves the basket half-empty. It functions as the witness, providing enormous amounts of information, but does not judge that information.

Every single “best practice” cited by Aldeman and Carey is subject to measurement: labor market histories of graduates, ratios of resource commitment to various student outcomes, proportion of students in learning communities or taking capstone courses, publicly-posted NSSE results, undergraduate research participation, space utilization rates, licensing income, faculty patents, volume of non-institutional visitors to art exhibits, etc. etc. There’s nothing wrong with any of these, but they all wind up as measurements, each at a different concentric circle of putatively engaged acceptees of a unilateral contract to provide evidence. By the time one plows through Aldeman and Carey’s banquet, one is measuring everything that moves -- and even some things that don’t.

Sorry, but basic capacity facts mean that consumers cannot vote with their feet in higher education.

If we glossed the Socratic notion on provision-of-information, the purpose is self-improvement, not comparison. The market approach to accountability implicitly seeks to beat Socrates by holding that I cannot serve as both witness and judge of my own actions unless the behavior of others is also on the table. The self shrinks: others define the reference points. “Accountability” is about comparison and competition, and an institution’s obligations are only to collect and make public those metrics that allow comparison and competition. As for who judges the competition, we have a range of amorphous publics and imagined authorities.

There are no formal agreements here: this is not a contract, it is not a warranty, it is not a regulatory relationship. It isn’t even an issue of becoming a Socratic self-witness and judge. It is, instead, a case in which one set of parties, concentrated in places of power, asks another set of parties, diffuse and diverse, “to disclose more and more about academic results,” with the second set of parties responding in their own terms and formulations. The environment itself determines behavior.

Ewell is right about the rules of the information game in this environment: when the provider is the institution, it will shape information “to look as good as possible, regardless of the underlying performance.”

U.S. News & World Report’s rankings

The messengers become self-appointed arbiters of performance, establishing themselves as the second party to which institutions and aggregates of institutions become “accountable.” Can we honestly say that the implicit obligation of feeding these arbiters constitutes “accountability”?

But if the issue is student learning, there is nothing wrong with -- and a good deal to be said for -- posting public examples of comprehensive examinations, summative projects, capstone course papers, etc. within the information environment, and doing so irrespective of anyone requesting such evidence of the distribution of knowledge and skills. Yes, institutions will pick what makes them look good, but if the public products resemble AAC&U’s “Our Students’ Best Work” project, they set off peer pressure for self-improvement and very concrete disclosure. The other prominent media messengers simply don’t engage in constructive communication of this type.

Ironically, a “market” in the loudest voices, the flashiest media productions, and the weightiest panels of glitterati has emerged to declare judgment on institutional performance in an age when student behavior has diluted the very notion of an “institution” of higher education. The best we can say is that this environment casts nothing but fog over the specific relationships, responsibilities, and obligations that should be inherent in something we call “accountability.” Perhaps it is about time that we defined these components and their interactions with persuasive clarity. I hope that this essay will invite readers to do so.

Clifford Adelman is senior associate at the Institute for Higher Education Policy. The analysis and opinions expressed in this essay are those of the author, and do not necessarily represent the positions or opinions of the institute, nor should any such representation be inferred.

I just heard from a faculty member at Cornell who has increased the amount of experiential learning she requires for class projects. More students are asking for it, and she's using every opportunity to get people out in the community or talking to people so they can engage in real-world experience.

Maryland has created a Center for Social Value Creation

Siloed disciplines are one of our biggest challenges. The world doesn't operate in disciplines—its problems and organizations are cross-cutting. The more interdisciplinary people can think and learn, the more equipped they will be to deal with the complexity of the real world.