The quality of experimental designs in mobile learning research: A systemic review and self-improvement tool

Yao Ting Sung, Han Yueh Lee, Je Ming Yang*, Kuo En Chang

*Corresponding author for this work

Research output: Contribution to journalReview articlepeer-review

37 Citations (Scopus)


The intervention effects of mobile learning have become one of the most popular topics in education research. Despite scholars critiquing the quality of mobile learning research, there is an obvious lack of quantitative studies exploring the design quality of experimental studies of mobile learning as well as tools for self-checking and improving experimental design. Based on the newly developed CREED (Checklist for the Rigor of Education-Experiment Designs), which focuses on the internal, construct, and statistical-conclusion validities of research, this study investigated the experimental research designs of mobile-learning studies. By analyzing 342 experimental studies published in refereed journals from 2006 to 2016, this study found that 72% of the mobile-learning studies exhibited experimental designs with a low or medium-low level of rigor. The mobile-learning research suffered from methodological limitations such as a lack of baseline equivalence between the experimental and control groups, not examining the fulfillment of statistical assumptions for specific statistical methods, poor quality of outcome assessment tools, and insufficient statistical powers. This study concluded that while utilizing advanced technologies, mobile-learning research employs laggard methodologies compared with standards proposed in educational and psychological research. Recommendations for overcoming these limitations are provided for use in further research and practices.

Original languageEnglish
Article number100279
JournalEducational Research Review
Publication statusPublished - 2019 Nov


  • Checklist for the Rigor of Education-Experiment Designs (CREED)
  • Experimental design
  • Mobile learning
  • Research quality
  • Study Design and Implementation Assessment Device (Study DIAD)
  • What Works Clearing House (WWC)

ASJC Scopus subject areas

  • Education


Dive into the research topics of 'The quality of experimental designs in mobile learning research: A systemic review and self-improvement tool'. Together they form a unique fingerprint.

Cite this