Conclusion: What have we learned?

The study asked four questions, which will be considered individually:

How does offering a choice of course content release affect the final overall completion rate of the Learn Moodle Basics MOOC?

The overall completion rates of the three MOOCs studied have risen slightly each time (14.1%, 16.0% and 17.3% if defining completion according to those who signed up.) However it is not possible to state with certainty whether this is because these three MOOCs have offered participants a choice of course content release or whether there are other factors. The MOOC has run twice yearly since 2015 and slight changes are made after each MOOC based on participant feedback. It is conceivable that these improvements, rather than offering a choice of content release, are the reason for the slightly higher completion rate. To test whether the access to materials is a contributing factor, the facilitators would have to run several MOOCs without a choice, and this would not be popular.

Which factors affect a participant’s choice of  course content release (All at once or step by step)?

Analysis of reponses in a participant survey in the January 2019 MOOC provides a number of insights. It is important to bear in mind that the data is from a single MOOC; however, 1127 participants responded, which is a fairly large cohort. Additionally not all participants chose to give free text replies.

  • The MOOC is for beginner, non technical educators wishing to teach with Moodle. It is not designed to be a regular experience, but some participants engage in it more than once either to refresh their skills or simply to join a learning community. Participants who are doing the MOOC for the first time are more likely to choose the Step by Step group than the All at once group (85% of SBS were doing the MOOC for the first time as opposed to 69% AAO)
  • Participants were asked how confident they were of completing the MOOC. There was a high level of confidence expressed by both groups: 95.85%  of the All at once group and 96.78% from the Step by Step group. However, participants took the survey at the end of their first week in the MOOC. Confidence levels, along with motivation might still be high, and it is important to bear in mind that not all those who completed the survey went on to complete.
  • The MOOC is in English, although participants can communicate in forums in their own language. The level of competency in English does not appear to be a significant factor in choice of path, since both groups expressed a high level of competency - 95% AAO and 92.76% SBS said they were "very! or "quite" confident. The AAO group consider themselves slightly more confident but only by a very small percentage.
  • An analysis of the free text responses enabled us to see how often certain key terms were used by both groups. The responses were grouped into themes and their frequency counted. From this, it is possible to deduce that more participants in the All at Once group considered themselves "Experienced" (15.68 % AAO v 0.9% SBS) while more participants from the Step by Step group felt they thought the weekly release of content might be "Easier" (2.10% AAO) v 10/16% SBS. Unsurprisingly, more from the All at Once group wanted the "Flexiblity" of a full overview (16.8% AAO v 2.9% SBS) and to be able to study at their "Own pace" (12.46% AAO v 2.9% SBS) whereas more from the Step by Step group hoped for the course to be "Manageable" (18.3% AAO v 26.3% SSB)

How does the choice of timed content release (all at once or step-by-step) affect a participant’s chance of completion of the Learn Moodle Basics MOOC?

In all three MOOCs the completion rates from participants in the All at Once group and the Step by Step group are very similar. The mean completion percentage for All at Once is 32.8% while the mean percentage for Step by Step participants is 33.3%

This is in line with an earlier similar study (Mullaney and Reich, 2013). Although the MOOCs in that study  are separate and longer, the authors also concluded that there is no appreciable difference in completion rates. Thus we can say that a participant is no more or less likely to complete based their choice of course content release, and that we must look elsewhere for factors likely to improve completion rates.

Which other factors, if any, affect full, partial or non-completion of the course?

The peer assessment 'My home country workshop' activity is the only activity with deadlines, the first of which must be met in order to complete the activity and complete the whole MOOC. This has always been considered to be the main factor why more participants do not complete the MOOC. However the results of this study suggest that this is not the case.

Originally the course offered just one completion certificate based on completing all activities including one with a deadline. However, feedback from participants encouraged the facilitators to offer an alternative certificate for those who completed every activity except the single one with the deadline.

A surprising fact revealed by the study is the low figures for participants who, unable to obtain a full Certificate of completion, succeeded instead to obtain a 'lesser' Certificate of achievement. Completing over 30 required activities is a worthwhile achievement in itself, and so it was expected that the figures for obtaining this Certificate of achivement would be much higher than the overall mean which is either 0.01% or 0.0.2% depending on whether the figure is based on the number who sign up or the number who log in at the start. The mean percentages of the All at Once group who partially completed the course is 4% while the mean percentage of the Step by Step group is 3.0%, only slightly higher from the All at Once group and perhaps related to the fact that more participants from the All at Once group were either experienced or returning Moodlers and therefore more comfortable with the course materials.

It would appear that the promise of an alternative certificate for those who do not meet the single deadline is not motivating enough to encourage either cohort to pursue the course if they are unlikely to complete everything. 

Points raised in the paired discussion lead to speculation that participants, realising they cannot obtain a full completion certificate, lose interest in completing other, non-time dependent activities. Additionally it must be remembered that some participants join simply to experience the latest version of Moodle, or to help others, and for them completion is not important.

The following section will discuss how these findings can inform and improve the design of future Learn Moodle MOOCs, with the hope of increasing the completion rates.