Group items tagged

But, one-off experiments can often seem much more promising than reality, once they are brought to scale.

Anant Agarwal, an electrical engineer who is president of EdX,

Les Perelman, has drawn national attention several times for putting together nonsense essays that have fooled software grading programs into giving high marks. He has also been highly critical of studies that purport to show that the software compares well to human graders. “My first and greatest objection to the research is that they did not have any valid statistical test comparing the software directly to human graders,” said Mr. Perelman, a retired director of writing and a current researcher at M.I.T.

Two start-ups, Coursera and Udacity, recently founded by Stanford faculty members to create “massive open online courses,” or MOOCs, are also committed to automated assessment systems because of the value of instant feedback. “It allows students to get immediate feedback on their work, so that learning turns into a game, with students naturally gravitating toward resubmitting the work until they get it right,” said Daphne Koller, a computer scientist and a founder of Coursera.

Mark D. Shermis, a professor at the University of Akron in Ohio, supervised the Hewlett Foundation’s contest on automated essay scoring and wrote a paper about the experiment. In his view, the technology — though imperfect — has a place in educational settings. With increasingly large classes, it is impossible for most teachers to give students meaningful feedback on writing assignments, he said. Plus, he noted, critics of the technology have tended to come from the nation’s best universities, where the level of pedagogy is much better than at most schools. “Often they come from very prestigious institutions where, in fact, they do a much better job of providing feedback than a machine ever could,” Dr. Shermis said. “There seems to be a lack of appreciation of what is actually going on in the real world.”

A number of the professors in the survey said they hoped to use MOOCs to increase their visibility, both among colleagues within their discipline (39 percent) and with the media and the general public (34 percent).

In May 2012, when the presidents of Harvard University and the Massachusetts Institute of Technology announced that they would enter the MOOC fray with $60-million to start edX, they were emphatic that their agenda was to improve, not supplant, classroom education.

"Online education is not an enemy of residential education," said Susan Hockfield, president of MIT at the time, from a dais at a hotel in Cambridge, "but an inspiring and liberating ally."

Typically a professor spent over 100 hours on his MOOC before it even started, by recording online lecture videos and doing other preparation. Others laid that groundwork in a few dozen hours.

Once the course was in session, professors typically spent eight to 10 hours per week on upkeep. Most professors managed not to be inundated with messages from their MOOC students—they typically got five e-mails per week

In all, the extra work took a toll. Most respondents said teaching a MOOC distracted them from their normal on-campus duties.

"It's out of 'my own' time, which is quite limited," Mr. Owens reported. "So, yes, other areas of my job suffered."

In lieu of credit toward a degree, most professors offer certificates to students who complete massive online courses. Three-quarters of the professors surveyed said they offered some sort of document certifying that a student had completed a MOOC. It remains unclear, however, how seriously those certificates are being taken by employers. College degrees are still seen as the coin of the realm.

Most professors who responded to The Chronicle's survey said they believed that MOOCs would drive down the cost of college; 85 percent said the free courses would make traditional degrees at least marginally less expensive, and half of that group said it would lower the cost "significantly."

A copy of that internal presentation, which Junn repeatedly emphasized was preliminary, was obtained this week by Inside Higher Ed from the California Faculty Association. According to the preliminary presentation, 74 percent or more of the students in traditional classes passed, while no more than 51 percent of Udacity students passed any of the three courses.

The courses were also put together in a rush. That’s apparently because of the timing of the deal with Udacity. The pilot project was announced a fortnight before classes started. (Like other similar deals, it was also the result of a no-bid contract.) The deal came together at the highest levels: On June 16, 2012, Brown e-mailed and called Thrun to talk about how Udacity could help California's higher education systems. “We need your help,” Brown said, according to Thrun. But, because of the haste, faculty were building the courses on the fly. Not only was this a “recipe for insanity,” Junn said, but faculty did not have a lot of time to watch how students were doing in the courses because the faculty were busy trying to finish them. It took about 400 hours to build a course, though the courses are designed to be reused.

Another factor in the disappointing outcomes may have been the students themselves. The courses included at-risk students, high school students and San Jose State students who had already failed a remedial math course.

Student performance data from the San Jose State/Udacity courses are expected to be released in coming weeks.