Center of Educational Technologies

Will MOOCs be Flukes?

06.11.2014
Source: The New Yourker

Online education is a technology with potentially revolutionary implications—but without a precise plan for realizing that potential.

Online education is a technology with potentially revolutionary implications—but without a precise plan for realizing that potential.

Credit Photograph by Melanie Stetson Freeman / The Christian Science Monitor via Getty

On July 23rd, 1969, Geoffrey Crowther addressed the inaugural meeting of the Open University, a British institution that had just been created to provide an alternative to traditional higher education. Courses would be conducted by mail and live radio. The basic mission, Crowther declared, was a simple one: to be open to people from all walks of life. “The first, and most urgent task before us is to cater for the many thousands of people, fully capable of a higher education, who, for one reason or another, do not get it, or do not get as much of it as they can turn to advantage, or as they discover, sometimes too late, that they need,” he told his audience. “Men and women drop out through failures in the system,” he continued, “through disadvantages of their environment, through mistakes of their own judgment, through sheer bad luck. These are our primary material.” He then invoked the message emblazoned on the Statue of Liberty: Open University wanted the tired, the poor, the huddled masses. To them, most of all, it opened its doors.

The mission Crowther described is the same one that has driven the proliferation of massive open online courses, or MOOCs, during the past few years. (Open University has often served as a sort of inspirational model for such ventures, which Nathan Heller wrote about last year in the magazine.) The premise of the MOOC movement is as commendable as it is democratic: quality education should not be a luxury good. MOOCs are flexible and they can be free; if people want an education, MOOCs can give it to them.

MOOCs started with a bang of optimism. In 2011, Sebastian Thrun, a star professor at Stanford, made his introductory course on artificial intelligence available via an online broadcast. Within three months, he garnered over a hundred and sixty thousand viewers. Soon after, he started Udacity, one of the largest MOOC platforms. Other platforms that have sprung up in recent years include edX, which was developed through a collaboration between Harvard and the Massachusetts Institute of Technology; Coursera, which was co-founded by two of Thrun’s colleagues at Stanford; and Khan Academy, the brainchild of the former hedge-fund analyst Salman Khan. Overseas, there are ventures like iversity and FutureLearn. And the number of available classes has been growing at an impressive pace. This month, there are four hundred and ninety-five MOOCs listed as in-progress on the MOOC aggregator Class Central, five times as many as there were a year ago.

And yet, despite the steady spread of the MOOC movement and the growing acceptance among university administrators that quality online education doesn’t have to be an oxymoron, enthusiasm for MOOCs has waned in the past year. Last winter, Thrun himself expressed some doubts. “We were on the front pages of newspapers and magazines, and at the same time, I was realizing, we don’t educate people as others wished, or as I wished,” he told Fast Company in an interview. “We have a lousy product.” (He later told the Times that his enthusiasm had not actually declined as much as he might have indicated.)

On one hand, MOOCs have achieved some worthy goals: they make top educational resources available, for free or for very low prices, to people who wouldn’t have access to them otherwise, including older populations, people with unpredictable schedules, and international audiences. And the quality of the education they offer can be quite high: the Open University, for instance, which began to offer online courses in the nineties, has consistently ranked among the top universities in the U.K. in measures of student satisfaction.

On the other hand, there are the numbers that gave Thrun pause. MOOC enrollment has soared, but completion rates are abysmal. According to a 2013 study, an average of only five per cent of the students in seventeen Coursera classes offered through the University of Pennsylvania actually finished their classes. Other estimated completion rates hover below thirteen per cent. And not all of the students who completed their courses necessarily passed.

The problem with MOOCs begins with the fact that, as their name says, they’re massive and open, which means that it can be easy to get lost in them. There are tens or even hundreds of thousands of students in some classes. Often, the students receive no personal acknowledgment or contact to hold them to account. And they can generally drop out the second they’re unhappy, frustrated, or overwhelmed. The datasuggest, in fact, that the students who succeed in the MOOC environment are those who don’t particularly need MOOCs in the first place: they are the self-motivated, self-directed, and independent individuals who would push to succeed anywhere.

Last year, a team from the University of Pennsylvania, led by Gayle Christensen, found that the majority of the people enrolled in thirty-two of the courses offered through Penn’s Coursera platform were young, already well educated, from developed countries, and, for the most part, employed—precisely the traditional students that Crowther held in contrast to the Open University’s target demographic. And most of the students weren’t taking classes in order to gain an essential education that they wouldn’t receive otherwise; predominantly, they said they had enrolled to satisfy curiosity or advance in a current job—both of which are worthy pursuits, to be sure, but they are not the main needs that MOOCs were created to meet.

Earlier this year, an M.I.T. post-doctoral researcher named Jennifer DeBoer analyzed student data from an M.I.T. class offered through the edX platform to see if there were any factors that predicted success in the class. When she and her colleagues looked at over seven thousand responses to an exit survey, they found that the students who did the best were the ones who came in with prior education, had the highest intellectual starting points, and collaborated outside of class. In other words, those who succeeded in the MOOC were those who had already succeeded academically.

2013 analysis of an ambitious project launched by San Jose State University in collaboration with Udacity showed similar patterns. The program was the first step of a planned push to incorporate online learning into the California’s public-university system. It was open to would-be college students as well as those who were already enrolled. The analysis found students who had already matriculated performed notably better than those who hadn’t. While overall pass rates were quite low—an average of thirty-three per cent across the three courses in the sample—those who performed the worst were the students who were not already enrolled at the university. In a remedial algebra course, for instance, only a quarter of the students passed, and less than one fifth of the non-matriculated students were among them. To put it in absolute terms, out of eighty-one students in the course, only twenty passed and only six of those were non-matriculated students. As the authors of the report acknowledged, the rates were disappointingly low. The project was suspended for reassessment.

Even students who succeed in traditional classrooms can get lost in the MOOC shuffle. In a five-year-long study of over fifty thousand students taking both online and face-to-face courses, researchers at Troy University found that online classes had higher failure rates than traditional classroom courses. When Di Xu, an economist at Columbia University’s Teachers College, analyzed data from more than forty thousand students who had enrolled in online courses at thirty-four colleges throughout Washington state, she found that, relative to face-to-face courses, online students earned lower grades and were less persistent. But not all students fared equally: she found that some subsets struggled more than others. Those subsets were male students, younger students, black students, and students who had lower G.P.A.s. What Xu found, in other words, was that MOOCs were the least effective at serving the students who needed educational resources the most.

So what’s the solution? How can MOOCs live up to their promise? One possibility is to go back in order to go forward. The MOOC movement started off in a tech whirlwind; the people who pushed it forward were so caught up in its technological possibilities that they scarcely considered decades of research into educational psychology. They might, for instance, have looked at the work of the researchers Patrick Suppes and Richard Atkinson, who in 1962 were charged with designing a course that would use the latest computer technology to teach mathematics and reading to children in kindergarten through third grade. Suppes, a Stanford psychologist and philosopher who had been trained in mathematics, decided to use something called “control theory” as the basis for his approach. Students in his computer-based class wouldn’t all receive the same instruction. Instead, their materials and the order in which those materials were presented would shift according to their past performance and other learning metrics—much like the G.R.E.’s adjusted sections, which become harder or easier depending on how you’re doing. Some students might get stopped every fifteen minutes for a reassessment and summary of materials; others might go for an hour before they reach a stopping point. The approach combined leveling (in which the same material is presented at different learning and reading levels, depending on the student) with dynamic learning (which involves playing around with the manner and order in which information is presented, so that students don’t get bored or frustrated).

The Suppes-Atkinson courses proved so successful that they were soon expanded to include multiple subjects for many age levels. A company called the Computer Curriculum Corporation (which is now a part of Pearson) started distributing them globally. In 1966, the psychologist William Estes, a pioneer of mathematical learning theory, including control theory, presented a paper at the International Congress of Psychology, held that year in Moscow. He spoke Russian. He’d learned it from a Suppes-Atkinson course.

Why don’t MOOCs structure their materials in a similar fashion? The technology to do so has only improved, as have the metrics to measure success. Some MOOCs, like mathematics or economics courses, lend themselves to this approach, but even lecture-reliant classes could incorporate the method by, for example, incorporating pauses for reviews and summaries of material, tailored to each student’s comprehension level. As it is, individualized methodology has largely gotten lost in the excitement over technological capabilities and large-group approaches. According to numerous interviews I conducted, and studies I read, very few MOOCs are using anything like control theory as part of their approach to teaching.

The other major problem is that MOOCs tend to be set up in a way that minimizes frustration for students (who might drop out at any moment). There often aren’t pop quizzes or the kinds of challenges that can alienate students in traditional settings. The problem here is that easy learning does not make good learning. In fact, the very tools that we believe make for better education may also make students more likely to quit. More frequent testing, for instance, can improve memory, learning, and retention. And, sometimes, the best test of all is the test that you fail: recent work from the cognitive psychologist Elizabeth Ligon Bjork has shown that pre-testing on never-before-seen materials helps students perform better in a subsequent course covering that material. In general, Bjork has found, speed bumps in learning are good—desirable difficulties, she calls them. MOOCs would likely be more effective if they didn’t shy away from challenging students, rather than presenting a fluid experience which gives the false impression of the learning and retention.

In 2012, Stanford president John Hennessy pronounced that the MOOC was going to be “transformative to education.” But, he added, “We don’t really understand how yet.” Two years later, the landscape remains much the same: MOOCs are a technology with potentially revolutionary implications for education, but without a precise plan for realizing that potential. One way of getting there could be for the leaders of the MOOC movement to look more closely at old methods, from when education was less massive, less open, and entirely offline.

Correction: A previous version of this article stated that Di Xu used data collected from students at Washington State University.

Maria Konnikova