Quality assessment and situational analysis of experimental E-learning designs: A case study of mobile learning

Because of the substantial development of mobile devices and educational software in recent years, the results of mobile learning–based interventions represent a popular research topic for investigation. Although experiment quality is the basis of empirical research, few studies have explored this issue. Thus, the present study investigated the shortfalls in existing experimental research designs related to mobile learning over the past decade and offers suggestions in this paper. The researchers collected data from all 197 experimental studies on mobile learning published in the Education Resources Information Center and Institute of Science Index from 2003 to 2013. The findings of the present study are described as follows: (1) Quasi-experimental designs represent the most frequently used design type (61%); however, among the quasi-experimental studies, 25% did not consider baseline equivalence. (2) Over half of the studies may not have met basic statistical assumptions, and approximately 70% used insufficient sample sizes, leading to low statistical power and imprecise effect size estimation. (3) Half of the studies did not provide information on test reliability and validity. Finally, this paper discusses the results and their implications for future research and practice.