One pitfall for learner data from Open Education

If on campus university courses would have the drop-out rate of Moocs they would be considered an utter failure. There are of course a lot of valid reasons why you would expect the amount of drop-outs to be way higher in a MOOC then in an on campus course but still the drop-out is impressive. It has been suggested that it is maybe best “to focus on the hundreds who complete them and what their data teaches us about how people learn“.

To me that seems like a waste but that is maybe a topic for a different post, for now lets focus on the effect of focussing on students that finish a MOOC in order to gain a better understanding of “how people learn”. The first remark that should be made is that analyzing this does not learn us a lot about how people learn but tells us something about how people who are able to finish a MOOC learn. The sketchy information that is available about that suggests that MOOCs are finished by the usual suspects: “Course completers typically held a Bachelor’s degree or higher” If that is the case for most MOOCs then we should be very aware that the findings of such research cannot be translated to the general population of learners.

In their article about fully guided instructionRichard E. Clark, Paul A. Kirschner and John Sweller made perfectly clear that there is a big distinction between the learning of novices and experts. Where novice to intermediate learners learn best when there is full guidance during instruction, experts “often thrive without much guidance”. So when we analyze the (most effective) learning of course completers in a MOOC our findings are skewed because we see the learning of experts. Translating those findings to the general population of learners would again ignore the well researched fact that novice to intermediate learners learn best from fully guided instruction.

To make matters worse: In the same article by Clark et al it is also explained that less skilled learners tend to prefer less guided instruction “even though they learn less from it”. This seems to be the case because the fully guided instruction forces them to “engage in explicit attention driven effort” Which they tend to dislike. So in evaluation MOOCs there is a double pitfall first we could mistakenly think that the succesful way of learning from the course completers is transferable to the whole population of learners and secondly if we were to asks the drop-outs of the MOOCs if they like the expert approach they would most likely agree even if in reality it would negative for their learning.

To make matters even more worse: Among educators there seems to be a tendency to prefer the less guided instruction model also for novice and intermediate learners. It is not entirely clear why. One explanation could be that they “confuse constructivism which is a theory about how one learns with a prescription for how to teach”. Alternatively one could think that developing fully guided instruction is more work than developing less guided instruction and that that is the explanation why it is preferred by educators.

This also raises the question how open, open education really is. Is it only open in the sense that everybody can enrol or is it also open in the sense that everybody can complete? For me that is the real issue are MOOCs just another means to serve the same population that is already privileged or is it really something new that can also serve learners that until now have no or limited access to higher education?

Update march 19th: My discussion with Alison Seaman about this post can be found here

This entry was posted in Uncategorized and tagged . Bookmark the permalink.

4 Responses to One pitfall for learner data from Open Education

  1. Tom Medema says:

    Interesting article! It may be worthwhile to compare a number of MOOCs and their strategies to the corresponding dropout rates. Moreover, something as simple as interviewing MOOC dropouts may provide more information on what went wrong.

    Based on that Duke University report you linked to, it seems like MOOC students may simply not be as motivated as “normal” students. Why else would almost 50% of the students not even watch a single movie? For example, it may be that because the tuition fees are so low, one does not have to be very comitted to enroll for this program. Only people who feel commited to the study would be willing to pay a higher fee.

    Or, in simple demand and supply high school economics: as supply can be virtually infinite, prices will be low. Therefore people who put a relatively low amount of value on the course will enroll even though they would not have enrolled for a program that was more expensive. In other words, a MOOC may attract more uncommitted students than a regular course.

    But of course, that’s not the entire story.

    • louwarnoud says:

      Yes ofcourse such simple high school economics is part of the explanation for the high non particpation grade. On the other hand would we take a free product from the supermarket that we really don’t need? Part of the story in my view is also that participants after enrolling in a course discover that the course is not suitable for them. In the Duke case the amount of math was one of the reported reasons for non participation.
      It would indeed be worthwhile to look at the data from other MOOCs but contrary to what there name suggests a lot of MOOCs are not sharing their usage data.

  2. Well, after mulling this around and reading both your post. Great post by the way with excellent links to credit your points, I also followed your conversation with Alison. As an educator, and learner, I have to take this from both sides or maybe three as I am a parent of a 20 year old as well. I do not believe the MOOC experience is for the unexperienced learner. I know as I know my hand that we have a great deal of students at all levels of higher ed that are not truly there to well learn. They are there because that’s what there parents told them to do, they are there because society says continue your education when they may not sure what their passion is. These students need a hand, they need a push, a great instructor, a mentor. They need accoutability. They need timeframes, assignments and due dates. My own son could never take an online course at 20, he’d forget he was in it. I’m really starting to believe that any type of MOOC is based on the type of learner you are. You are right, I am a highly motivated person, and I always have been. Yet when I was just starting out, I would have needed more than a connectivist or xmooc provides. I just think it’s great that we are looking to TRY to fulfill the needs of the general population with educational options they did not have before and as you and Alison seemed to really help me realize, it’s “who you are” “what motivation you have” and “where you are in your life” as to whether you as a person will suceed in a MOOC. Thank you for stirring some great thoughts.

  3. louwarnoud says:

    Thanks for you nice reply, I saw that you wrote a whole post about it now also. Replying to both your post and your reply here I think you are right that motivation is an important concept in relationship to being succesful in a MOOC. If we interpret motivation as the effort you are willing or able to put into a MOOC we are getting close to the concepts I was discussing in my post. Crucial in how much somebody can learn is the amount of “attention driven effort” a student has to put into learning new things. There are limits how much of that a student can give to a learning task. Thats why for novice to intermediate users you have to structure/guide their learning or they get lost in the amount of concepts and information they have to process.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s