Group items tagged

"Measuring College-Teacher Quality January 13, 2011, 10:40 am By Kevin Carey David Glenn's Chronicle article on using course sequence grades to estimate teacher quality in higher education illustrates a crucial flaw in the way education researchers often think about the role of evidence in education practice. The article cites a recent study of Calculus grades in the Air Force Academy. All students there are required to take Calculus I and II. They're randomly assigned to instructors who use the same syllabus. Students all take the same final, which is collectively graded by a pool of instructors. These unusual circumstances control for many external factors that might otherwise complicate an analysis of teacher quality. The researchers found that students taught by permanent faculty got worse grades in Calculus I than students taught by short-term faculty. But the pattern reversed when those students went on to Calculus II-those taught by full-time faculty earned better grades in the more advanced course, suggesting that short-term faculty might have been "teaching to the test" at the expense of deeper conceptual understanding. Students taught by full-time faculty were also more likely to enroll in upper-level math in their junior and senior years. In addition, the study found that student course evaluations were positively correlated with grades in Calculus I but negatively correlated with grades in Calculus II."

"Apparently, there's a company that employs people in India with graduate degrees to grade papers for American professors. For twelve bucks a paper, they'll give not just a letter grade, but comments. The idea is to free up faculty to focus on instruction (or, more accurately, research), rather than grading. It also saves the university money, since outsourcing the grading allows you to run classes at much larger sizes. From the comments to the article, you'd think that this had never been done before. You'd think that professors have always done their own grading, and that the grading was a form of deep examination of each student's soul, resulting in unparalleled insight and bonding. Um, no. And I have the scantron invoice to prove it."

"The Measuring Stick Finale: A Hawk and a Skeptic Walk Into a Bar … December 21, 2010, 12:38 pm By David Glenn As promised, this blog will expire with the calendar year (cue violins). In this last post, I've tried to distill some fundamental arguments about assessment and accountability in higher education. I've borrowed liberally from comments that readers have left here and elsewhere on The Chronicle's site. Many thanks to all of you for reading and arguing. Of course there are more than two sides to these debates. In that respect, what follows is pathetically reductive. But I've tried not to put my thumb on the scale on behalf of either of these characters. I've tried to convey the strongest cases on each side of an admittedly-crudely-drawn line. (If I've failed to do that, you should of course call me out in the comments.) The scene: Friday, 7:45 p.m. A bar on the outskirts of a moderately selective public university. The décor and the jukebox appeal to disillusioned 36-year-old faculty members and a few graduate students. The fraternities leave this place alone. Accountability Skeptic: Did you see that memo from the dean today? He's hired some consultant to teach us how to "design learning outcomes" for our students. I can't imagine a bigger waste of time and money. And I don't think the dean even believes in this stuff himself. I think he's just trying to keep the accreditors off his back. Accountability Hawk: Don't be so cynical. Tuition and fees here have gone up by more than 50 percent since 2000. Students are taking on miserable levels of debt to be in our classrooms. They deserve to have faculty members who are focused on their learning-and that means that we need some kind of common understandings in our departments about the knowledge and skills students are supposed to be picking up. Skeptic: Listen. I do focus on my students. I assess their learning every week. It's called grading. I get the feeling that the dea

"December 19, 2010 2 Studies Shed New Light on the Meaning of Course Evaluations By David Glenn Under the mandate of a recently enacted state law, the Web sites of public colleges and universities in Texas will soon include student-evaluation ratings for each and every undergraduate course. Bored and curious people around the planet-steelworkers in Ukraine, lawyers in Peru, clerical workers in India-will be able, if they're so inclined, to learn how students feel about Geology 3430 at Texas State University at San Marcos. But how should the public interpret those ratings? Are student-course evaluations a reasonable gauge of quality? Are they correlated with genuine measures of learning? And what about students who choose not to fill out the forms-does their absence skew the data? Two recent studies shed new light on those old questions. In one, three economists at the University of California at Riverside looked at a pool of more than 1100 students who took a remedial-mathematics course at a large university in the West (presumably Riverside) between 2007 and 2009. According to a working paper describing the study, the course was taught by 33 different instructors to 97 different sections during that period. The instructors had a good deal of freedom in their teaching and grading practices-but every student in every section had to pass a common high-stakes final exam, which they took after filling out their course evaluations. That high-stakes end-of-the-semester test allowed the Riverside economists to directly measure student learning. The researchers also had access to the students' pretest scores from the beginning of the semester, so they were able to track each student's gains. Most studies of course evaluations have lacked such clean measures of learning. Grades are an imperfect tool, as students' course ratings are usually strongly correlated with their grades in the course. Because of that powerful correlation, some studies have suggested that

"Guido Sarducci and the Purpose of Higher Education March 14, 2011, 10:38 am By Sandy Baum and Michael McPherson The way college courses generally work is that a teacher presents a group of students with some subject matter, then attempts through tests and papers to determine how well the students have mastered the subject matter. Those judgments are summarized in a letter grade. A list of those subject matters and grades constitutes the transcript that describes what the student has learned and what the student's performance was overall. The students and the teacher are focused on the subject matter, and the implied view is that the learning in college is captured in the exercises that inform those grades. The limitations of this "subject matter recall" model of higher education are hilariously captured in Don Novello's comic performance on Saturday Night Live as Father Guido Sarducci, who marketed the "Five Minute University": http://youtu.be/kO8x8eoU3L4 Sometimes, in some subjects, the mastery of specific subject matter is precisely what is at stake. Students aiming to be engineers will make pretty direct use of the principles they learn in a course in mechanics, for example. Students focused on other particular occupational qualifications, whether in certificate, associate-degree, or bachelors'-degree programs, will probably make direct use of some of the subjects they are taught. However, the learning produced in large parts of a college education is-or at least should be-different from that, and the role of tests and grades is correspondingly different. As economics professors, we have always understood that little of what students learned in advanced economics courses-neither the factual content nor the specific analytical techniques-was ever going to come up in their later lives (unless fate intervened and made them economists). We suspect that the same is true of other advanced courses, whether in mathematics or literature or psyc

"MITx: The Next Chapter for University Credentialing? December 19, 2011 - 3:36pm By Audrey Watters A big announcement from MIT today: the university is launching a new online learning initiative, MITx, one that will allow non-enrolled students to take online courses and receive certification if they successfully complete them. MIT has long been known for being on the leading edge of higher education experimentation, most notably with MIT OpenCourseware. The decision by MIT faculty to make all of their course materials freely and openly available online is now a decade old, and some 100 million people have downloaded and accessed that content. But even with a catalog that boasts over 2,000 courses, MIT OCW has always been just that: courseware. All the syllabi, handouts, and quizzes, but no interaction with professors, no interaction with fellow learners, no grades, no college credit. MITx will act as a middle tier, of sorts, something between the traditional, on-campus experience of formally enrolled MIT students and the open and informal learning opportunities afforded by open courseware. But "this is not MIT light," insists Provost L. Rafael Reif. What MITx is is still very much under construction. The first class should be available in the spring of 2012, and it's not clear what course(s) will be offered (although it's a probably a safe bet that it's a science or engineering class). MIT describes MITx as a self-paced course, one with "interactivity, online laboratories and student-to-student communication." While the course itself will be free (and the custom-created course materials will be openly licensed, as with all MIT courses), students who wish to be graded will be able to pay for certification. There's no indication yet of what that fee will be. Nor is it clear how assessment for MITx will work -- will all students take quizzes and submit homework or just those who are paying for certification? Will students have access to instructors? If so, how?

"Explaining Gaps in Readiness for College-Level Math: The Role of High School Courses Issue/Topic: Curriculum--Mathematics; High School--College Readiness; Postsecondary Participation--Access Author(s): Long, Mark; Iatarola, Patrice; Conger, Dylan Organization(s): Evans School, University of Washington; Department of Educational Leadership and Policy, Florida State University; Trachtenberg School of Public Policy and Public Administration, George Washington University Publication: Education Finance and Policy Season: Winter 2009 Background: Despite substantial increases in high school graduation requirements, almost one-third of US college freshmen are unprepared for college-level math. Students who are low income, Hispanic and black are least likely to be prepared for college-level coursework, and females are slightly less likely to be ready than males. Little of the prior research in this area has focused on the contribution of high school courses to demographic and socioeconomic gaps in secondary and postsecondary success. Purpose: To estimate how much of the racial, socioeconomical and gender gaps in high school and postsecondary outcomes are determined by the courses that students take while in high school. Findings/Results: Less than half of the black and poor students and just over half of the Hispanic students in the sample were prepared for college-level math - far below the readiness rates for whites, Asian and nonpoor students. 1. The courses students take in high school contribute significantly to their college readiness, with the largest gains occurring at Algebra 2 (the minimum level of math required to graduate even with a college preparatory diploma in Florida is Algebra 1). 2. The slight male advantage in college math readiness is not explained by course-taking differentials because female students already take more advanced math courses than males. This finding is consistent with other work on gender disparities. 3. The results suggest

"Can Students Learn to Learn? January 31, 2011 SAN FRANCISCO -- Why do some students in a course perform better than others of roughly equal ability? The answers, of course, are as varied as are students. Some spend more time studying, or study more efficiently; some have other priorities; some don't connect with the instructor. Some of these factors relate to metacognition, defined variously as knowing about knowing or being able to understand why we learn the way we do. A student with metacognition may realize after a disappointing test that she didn't study hard enough, and needs to devote more time to academics. The student operating without metacognition may respond to the same setback by trashing his instructor on RateMyProfessors.com. While some colleges have long taught study skills, some institutions are experimenting with efforts to teach much more than how to study: they are looking for ways to grow their students' metacognition. Many of these projects are still small and don't have years of data to report, but on Friday, several of those involved in the efforts shared their enthusiasm for the approach in a session at the annual meeting of the Association of American Colleges and Universities. The projects discussed here were from members of the Associated Colleges of the Midwest, which received support from the Teagle Foundation to coordinate the efforts. So how does this work? Kristin E. Bonnie, an assistant professor of psychology at Beloit College, said that on her tests, she has always let students pick a few questions on the multiple-choice portion (say 3 of 25) that won't be graded. It's a way to show students that she understands they may not grasp everything right away. In the past, she just let students cross out the questions they didn't want to answer. Now, she makes them answer all the questions -- and to exempt a question from grading, students must pick from a list she provides of the reasons they are selecting that question. Students

Completing assignments and sitting through exams can be stressful. But when it comes to being graded the waiting is often the hardest part. This is perhaps most true at the end of a semester, as students wait for their instructors to reduce months of work into a series of letter grades that will stay on the books forever. But at Austin Peay State University, students do not have to wait for the end of a semester to learn their grade averages. Thanks to a new technology, pioneered by the university's provost, they do not even have to wait for the semester to start. Tristan Denley, the provost, has built software, called Degree Compass, that analyzes an individual student's academic record, along with the past grades of hundreds of Austin Peay State students in various courses, and predicts how well a particular student is likely to do in a particular course long before the first day of class. (That includes first-year students; the software draws on their high school transcripts and standardized test scores.)

"January 22, 2012 MIT Mints a Valuable New Form of Academic Currency James Yang for The Chronicle By Kevin Carey The Massachusetts Institute of Technology has invented or improved many world-changing things-radar, information theory, and synthetic self-replicating molecules, to name a few. Last month the university announced, to mild fanfare, an invention that could be similarly transformative, this time for higher education itself. It's called MITx. In that small lowercase letter, a great deal is contained. MITx is the next big step in the open-educational-resources movement that MIT helped start in 2001, when it began putting its course lecture notes, videos, and exams online, where anyone in the world could use them at no cost. The project exceeded all expectations-more than 100 million unique visitors have accessed the courses so far. Meanwhile, the university experimented with using online tools to help improve the learning experience for its own students in Cambridge, Mass. Now MIT has decided to put the two together-free content and sophisticated online pedagogy­-and add a third, crucial ingredient: credentials. Beginning this spring, students will be able to take free, online courses offered through the MITx initiative. If they prove they've learned the materi­al, MITx will, for a small fee, give them a credential certifying as much. In doing this, MIT has cracked one of the fundamental problems retarding the growth of free online higher education as a force for human progress. The Internet is a very different environment than the traditional on-campus classroom. Students and employers are rightly wary of the quality of online courses. And even if the courses are great, they have limited value without some kind of credential to back them up. It's not enough to learn something-you have to be able to prove to other people that you've learned it. The best way to solve that problem is for a world-famous university with an unimpeachable reputat

"January 23, 2012 Invisible Gorillas Are Everywhere By William Pannapacker By now most everyone has heard about an experiment that goes something like this: Students dressed in black or white bounce a ball back and forth, and observers are asked to keep track of the bounces to team members in white shirts. While that's happening, another student dressed in a gorilla suit wanders into their midst, looks around, thumps his chest, then walks off, apparently unseen by most observers because they were so focused on the bouncing ball. Voilà: attention blindness. The invisible-gorilla experiment is featured in Cathy Davidson's new book, Now You See It: How the Brain Science of Attention Will Transform the Way We Live, Work, and Learn (Viking, 2011). Davidson is a founder of a nearly 7,000-member organization called Hastac, or the Humanities, Arts, Sciences, and Technology Advanced Collaboratory, that was started in 2002 to promote the use of digital technology in academe. It is closely affiliated with the digital humanities and reflects that movement's emphasis on collaboration among academics, technologists, publishers, and librarians. Last month I attended Hastac's fifth conference, held at the University of Michigan at Ann Arbor. Davidson's keynote lecture emphasized that many of our educational practices are not supported by what we know about human cognition. At one point, she asked members of the audience to answer a question: "What three things do students need to know in this century?" Without further prompting, everyone started writing down answers, as if taking a test. While we listed familiar concepts such as "information literacy" and "creativity," no one questioned the process of working silently and alone. And noticing that invisible gorilla was the real point of the exercise. Most of us are, presumably, the products of compulsory educational practices that were developed during the Industrial Revolution. And the way most of us teach is a relic of the s

"January 8, 2012 'Badges' Earned Online Pose Challenge to Traditional College Diplomas 'Badges' Earned Online Pose Challenge to Traditional College Diplomas 1 Photo illustration by Bob McGrath for The Chronicle Enlarge Image By Jeffrey R. Young The spread of a seemingly playful alternative to traditional diplomas, inspired by Boy Scout achievement patches and video-game power-ups, suggests that the standard certification system no longer works in today's fast-changing job market. Educational upstarts across the Web are adopting systems of "badges" to certify skills and abilities. If scouting focuses on outdoorsy skills like tying knots, these badges denote areas employers might look for, like mentorship or digital video editing. Many of the new digital badges are easy to attain-intentionally so-to keep students motivated, while others signal mastery of fine-grained skills that are not formally recognized in a traditional classroom. At the free online-education provider Khan Academy, for instance, students get a "Great Listener" badge for watching 30 minutes of videos from its collection of thousands of short educational clips. With enough of those badges, paired with badges earned for passing standardized tests administered on the site, users can earn the distinction of "Master of Algebra" or other "Challenge Patches." Traditional colleges and universities are considering badges and other alternative credentials as well. In December the Massachusetts Institute of Technology announced that it will create MITx, a self-service learning system in which students can take online tests and earn certificates after watching the free lecture materials the university has long posted as part of its OpenCourseWare project. MIT also has an arrangement with a company called OpenStudy, which runs online study groups, to give online badges to students who give consistently useful answers in discussion forums set up around the university's free course materials. But the b

"Technology Is at Least 3 Years Away From Improving Student Success January 13, 2012, 7:34 am By Josh Fischman Las Vegas-At the very start of the Higher Ed Tech Summit here this week, James Applegate threw out a challenge. Mr. Applegate, vice president for program development at the Lumina Foundation, told an overflow crowd that the United States needed 60 percent of its adults to hold high-quality degrees and credentials by the year 2025. During the rest of the day, technology executives described programs that could improve graduation rates and learning, but won't be able to do so for several years. They collect many points of data on what professors and students do, but can't yet say what results in better grades and graduation rates. "We're beginning to get lots of data on things like time of task, but we don't have the outcomes yet to say what leads to a true learning moment. I think we are three to five years away from being about to do that," said Troy Williams, vice president and general manager of Macmillan New Ventures, which makes the classroom polling system called I-clicker. "These are really early days," agreed Matthew Pittinsky, who runs a digital transcript company called Parchment and was one of the founders of Blackboard. There's lots of technology out there that's outcome-related. For instance, at the meeting, which is part of the international Consumer Electronics Show, the interactive textbook publisher Kno announced a suite of new features. One of them, a performance gauge callled Kno Me, gives students information about how much time they spend on different sections of a book, the results of quizzes, and the kinds of notes they took. "With thousands of students using these books, we can show them which of these variables are related to students-anonymous, of course-who get A's, or B's, or C's, so students learn what kind of activity leads to the best results," said Osman Rashid, the company's chief ex

"A recent Inside Higher Ed article discussed the experimental work of Duke University's Cathy Davidson, involving students grading themselves. According to Davidson, when students are held responsible for assessing their own - and their peers' - writing performances and products, they learn to take more responsibility for their own learning, and consequently apply themselves much more energetically to their work. In response, Leonard Cassuto of Fordham University points to the fact that at least 15 of Davidson's 16 students in this experiment earned As for the course. Cassuto sees that as a problem and argues that professors need to be the ones saying "You did good work, but not the best in the class." I think I may have somewhat of a compromise when it comes to assessing student written work. I was in the same situation as many writing instructors for years. Students write, write, write. Then I would spend about five minutes per page supplying written commentary individually on each of their papers. But about a year ago I started doing things differently. And I don't plan on going back any time soon."

"Lori Whisenant knows that one way to improve the writing skills of undergraduates is to make them write more. But as each student in her course in business law and ethics at the University of Houston began to crank out-often awkwardly-nearly 5,000 words a semester, it became clear to her that what would really help them was consistent, detailed feedback. Her seven teaching assistants, some of whom did not have much experience, couldn't deliver. Their workload was staggering: About 1,000 juniors and seniors enroll in the course each year. "Our graders were great," she says, "but they were not experts in providing feedback." That shortcoming led Ms. Whisenant, director of business law and ethics studies at Houston, to a novel solution last fall. She outsourced assignment grading to a company whose employees are mostly in Asia."

Innovative system for grading that encourages collaboration and a high degree of creativity on the part of students.

"'Jaw-Dropping' Data on Black Male Student Achievement The Council of Great City Schools is today releasing what it calls "jaw-dropping" data on the achievement of black male students, The New York Times reported. While the data are primarily about K-12 achievement, they suggest strong links to subsequent gaps noted for black male students in college enrollment levels. The report argues that the achievement gaps start at young ages, noting that while 12 percent of black boys in the fourth grade are proficient in reading, 38 percent of white boys are proficient. In eighth grade, 12 percent of black boys and 44 percent of white boys are proficient in math. According to the report, poverty levels are only part of the equation because poor white boys (defined by eligibility for subsidized school lunches) are doing as well as black boys who do not live in poverty."

"April 3, 2011 A Perfect Storm in Undergraduate Education, Part 2 By Thomas H. Benton What is keeping undergraduates from learning? Last month, I speculated from my perspective as a college teacher about a set of interlocking factors that have contributed to the problem. In that column (The Chronicle, February 25), I referred to the alarming data presented by Richard Arum and Josipa Roksa in Academically Adrift: Limited Learning on College Campuses (University of Chicago Press, 2011) in the context of President Obama's call for more students to attend college in order to prepare for the economy of the future. Why, I asked, should we send more students to college-at an ever greater cost-when more than a third of them, according to Arum and Roksa, demonstrate "no improvement in critical thinking, complex reasoning, and writing skills" after four years of education? This month I want to speculate on why students (and, to a lesser extent, their parents) are not making choices that support educational success. What could they possibly be thinking? The student as consumer. Surely adolescent expectations of Animal House debaucheries have been with us since the decline of college as preparation for the ministry. But, in the past few generations, the imagery and rhetoric of academic marketing have cultivated a belief that college will be, if not decadent, at least primarily recreational: social activities, sporting events, and travel. Along the way, there may be some elective cultural enrichment and surely some preprofessional training and internships, the result of which will be access to middle-class careers. College brochures and Web sites may mention academic rankings, but students probably won't read anything about expectations of rigor and hard work: On the contrary, "world-renowned professors" will provide you with a "world-class education." Increasingly, students are buying an "experience" instead of earning an education, and, in the competition to attract cu

"February 12, 2012 'Academically Adrift': The News Gets Worse and Worse 'Academically Adrift': The News Gets Worse and Worse 1 Michael Morgenstern for The Chronicle Enlarge Image By Kevin Carey In the last few months of 2010, rumors began circulating among higher-education policy geeks that the University of Chicago Press was about to publish a new book written by a pair of very smart sociologists who were trying to answer a question to which most people thought they already knew the answer: How much do students learn while they're in college? Their findings, one heard, were ... interesting. The book, Academically Adrift, by Richard Arum and Josipa Roksa, fulfilled that promise-and then some. It was no surprise that The Chronicle gave prominent coverage to the conclusion that "American higher education is characterized by limited or no learning for a large proportion of students," but few people anticipated that the book would become the rare piece of serious academic scholarship that jumps the fence and roams free into the larger culture. Vanity Fair used space normally allotted to Kennedy hagiography to call it a "crushing exposé of the heretofore secret society known as 'college.'" The gossip mavens at Gawker ran the book through their patented Internet cynicism machine and wrote that "To get a college degree, you must go into a soul-crushing amount of debt. And what do you get for all that money? Not learning." The New Yorker featured Academically Adrift in a typically brilliant essay by Louis Menand. In one of her nationally syndicated columns, Kathleen Parker called the book a "dense tome" while opining that the failure of higher education constituted a "dot-connecting exercise for Uncle Shoulda, who someday will say-in Chinese-'How could we have let this happen?'" Her response proved that Kathleen Parker has a gift for phrasing and did not actually read the book, whose main text runs to only 144 concise and well-argued pages. But the definitive

"The Year Ahead in IT, 2012 January 6, 2012 - 3:00am By Lev Gonick This series of annual Year Ahead articles on technology and education began on the eve of what we now know is one of the profound downturns in modern capitalism. When history is written, the impact of the deep economic recession of 2008-2012 will have been pivotal in the shifting balance of economic and political power around the world. Clear, too, is the reality that innovation and technology as it is applied to education is moving rapidly from its Anglo-American-centered roots to a now globally distributed dynamic generating disruptive activities that affect learners and institutions the world over. Seventy years ago, the Austrian-born Harvard lecturer and conservative political economist Joseph Schumpeter popularized the now famous description of the logic of capitalism, Capitalism, Socialism, and Democracy. The opening of new markets, foreign or domestic … illustrate(s) the same process of industrial mutation - if I may use that biological term - that incessantly revolutionizes the economic structure from within, incessantly destroying the old one, incessantly creating a new one. This process of Creative Destruction is the essential fact about capitalism. Our colleges and universities, especially those in the United States, are among the most conservative institutions in the world. The rollback of public investment in, pressure for access to, and indeterminate impact of globalization on postsecondary education all contribute to significant disorientation in our thinking about the future of the university. And then there are the disruptive impacts of information technology that only exacerbate the general set of contradictions that we associate with higher education. The faculty are autonomous and constrained, powerful and vulnerable, innovative at the margins yet conservative at the core, dedicated to education while demeaning teaching devoted to liberal arts and yet powerfully vocatio