Do planning prompts increase educational success? Evidence from randomized controlled trials in MOOCs

Massive Open Online Courses are a promising educational innovation. Yet, they suffer from high drop-out rates. As a remedy, we propose a planning prompt and test its effect on course completion and further outcomes such as course engagement and satisfaction in four large-scale randomized controlled trials. The results reveal an overall null effect on the completion rate, ruling out effect sizes beyond the [-7%, 3%] interval. However, this overall effect masks heterogeneity across and within courses: In one course the planning prompt increases course completion by 19%, highlighting the importance of replications in slightly different contexts. Using random causal forests, we also reveal tendencies for differential effects by subgroups. Better targeting could hence improve the effectiveness of planning prompts in online learning.