One of the reasons MOOCs are interesting as a pedagogical experiment is that, being online, they generate lots of capturable data. This should create a data-rich environment which improves our understanding of learning processes, etc etc.
So why is so little data about MOOCs actually being made public?
Katy Jordan of the UK Open University just put together a nice little graphic about MOOC enrolment and success rates. While her conclusions are interesting (avg enrolment = 50,000; avg. completion rate = 10%), what I found most striking was the small size of her sample – apparently this data simply isn’t available for most courses. Duke and Georgia Tech have commendably published some data about their early experiments. But apart from that, the cupboard is pretty bare.
With such small samples, drawing conclusions about MOOCs is difficult. Are male enrolments in MOOCs high because they like the format, or because a high percentage of the courses offered to date have been in male-dominated fields like computer science? Is the high proportion of students with advanced degrees truly indicative of demand, or does it just reflect the fact that a lot of currently offered MOOCs cover quite advanced subject matter? We simply don’t know.
The reasons for this knowledge lacuna is easy enough to spot: Udacity, Coursera, and EdX, who own the data, are all private companies (even if the latter is owned by non-profit institutions), and, as a rule, commercial entities don’t open their kimonos all that wide when it comes to customer data. Until now, this hasn’t been such a big deal because most MOOCs are just free edutainment, so the consequences of not knowing were low. But if cash-strapped governments and/or universities start forcing people to take these courses for credit, the stakes will rise dramatically.
Why? Because there are some very justified fears about how using low-cost, low-support MOOCs for introductory or remedial courses – as is currently being done in an experiment at San Jose State University – might affect student success. In particular, research from Columbia University has shown that less academically-prepared students struggle even more in online classes than they do elsewhere. To properly evaluate the effects of these cost-cutting strategies, we’ll need to know how students from different backgrounds are faring, meaning this data has to be tracked and published.
So, a key question for McGill, UBC, and Toronto, who’ve signed up with EdX and Coursera: will they commit to tracking and publishing demographic data on student participation and success in their MOOCs? Or, will the results of their publicly-funded experiments in online education become the property of private, for-profit entities?
We should be told.
Tweet this post
Time span of a reliable research study would be in years not in months. There is not yet enough data on MOOCs to say anything sure or to have learned that much. How detailed data about learning processes do universities publish from normal on campus courses?
The opening lines in today’s One Thought includes the phrase “pedagogical experiment.” This raises an interesting angle for me: if MOOCs were a graduate student thesis project instead of a post-secondary phenom, they would most likely have to go through a research ethics review. Maybe this is my inherently over-cautious nature speaking, but the Colombia U study suggests that there may be some negatives associated with MOOCs. So, I agree with the call for more data and a much clearer understanding of the implications of MOOCs on students; in fact, it feels that we might be getting too far ahead of ourselves. A year ago, MOOCs were nowhere, and now early-adopter institutions are integrating them into course offerings. Seems like we’re going from drawing board to classroom very quickly. Weren’t open area classrooms a revolutionary innovation a few decades ago that turned out to be less beneficial than the theory suggested? I wonder if slowing down wouldn’t be so bad…
It’s important to remember that one reason many MOOC attendees have advanced degrees is that an awful lot of professors are signing up to see what they’re like. Look for blog reviews and Twitter mentions, and you’ll find an incredible number of profs taking MOOCs. That’s a serious skew for the data, including for completion rates, depth of engagement, and really everything interesting to be learned from these vehicles.