Quality assessment and situational analysis of experimental E-learning designs

A case study of mobile learning

Han Yueh Lee, Je Ming Yang, Yao-Ting Sung

Research output: Contribution to journalArticle

Abstract

Because of the substantial development of mobile devices and educational software in recent years, the results of mobile learning–based interventions represent a popular research topic for investigation. Although experiment quality is the basis of empirical research, few studies have explored this issue. Thus, the present study investigated the shortfalls in existing experimental research designs related to mobile learning over the past decade and offers suggestions in this paper. The researchers collected data from all 197 experimental studies on mobile learning published in the Education Resources Information Center and Institute of Science Index from 2003 to 2013. The findings of the present study are described as follows: (1) Quasi-experimental designs represent the most frequently used design type (61%); however, among the quasi-experimental studies, 25% did not consider baseline equivalence. (2) Over half of the studies may not have met basic statistical assumptions, and approximately 70% used insufficient sample sizes, leading to low statistical power and imprecise effect size estimation. (3) Half of the studies did not provide information on test reliability and validity. Finally, this paper discusses the results and their implications for future research and practice.

Original languageEnglish
Pages (from-to)31-60
Number of pages30
JournalJournal of Research in Education Sciences
Volume62
Issue number2
DOIs
Publication statusPublished - 2017 Jan 1

Fingerprint

information center
equivalence
learning
research planning
empirical research
experiment
science
resources
education
software

Keywords

  • Experimental design
  • Mobile learning
  • Research quality

ASJC Scopus subject areas

  • Education

Cite this

Quality assessment and situational analysis of experimental E-learning designs : A case study of mobile learning. / Lee, Han Yueh; Yang, Je Ming; Sung, Yao-Ting.

In: Journal of Research in Education Sciences, Vol. 62, No. 2, 01.01.2017, p. 31-60.

Research output: Contribution to journalArticle

@article{0ea1c9f3e6c64b4baac8af7a4ffee621,
title = "Quality assessment and situational analysis of experimental E-learning designs: A case study of mobile learning",
abstract = "Because of the substantial development of mobile devices and educational software in recent years, the results of mobile learning–based interventions represent a popular research topic for investigation. Although experiment quality is the basis of empirical research, few studies have explored this issue. Thus, the present study investigated the shortfalls in existing experimental research designs related to mobile learning over the past decade and offers suggestions in this paper. The researchers collected data from all 197 experimental studies on mobile learning published in the Education Resources Information Center and Institute of Science Index from 2003 to 2013. The findings of the present study are described as follows: (1) Quasi-experimental designs represent the most frequently used design type (61{\%}); however, among the quasi-experimental studies, 25{\%} did not consider baseline equivalence. (2) Over half of the studies may not have met basic statistical assumptions, and approximately 70{\%} used insufficient sample sizes, leading to low statistical power and imprecise effect size estimation. (3) Half of the studies did not provide information on test reliability and validity. Finally, this paper discusses the results and their implications for future research and practice.",
keywords = "Experimental design, Mobile learning, Research quality",
author = "Lee, {Han Yueh} and Yang, {Je Ming} and Yao-Ting Sung",
year = "2017",
month = "1",
day = "1",
doi = "10.6209/JORIES.2017.62(2).02",
language = "English",
volume = "62",
pages = "31--60",
journal = "Journal of Research in Education Sciences",
issn = "2073-753X",
publisher = "國立臺灣師範大學",
number = "2",

}

TY - JOUR

T1 - Quality assessment and situational analysis of experimental E-learning designs

T2 - A case study of mobile learning

AU - Lee, Han Yueh

AU - Yang, Je Ming

AU - Sung, Yao-Ting

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Because of the substantial development of mobile devices and educational software in recent years, the results of mobile learning–based interventions represent a popular research topic for investigation. Although experiment quality is the basis of empirical research, few studies have explored this issue. Thus, the present study investigated the shortfalls in existing experimental research designs related to mobile learning over the past decade and offers suggestions in this paper. The researchers collected data from all 197 experimental studies on mobile learning published in the Education Resources Information Center and Institute of Science Index from 2003 to 2013. The findings of the present study are described as follows: (1) Quasi-experimental designs represent the most frequently used design type (61%); however, among the quasi-experimental studies, 25% did not consider baseline equivalence. (2) Over half of the studies may not have met basic statistical assumptions, and approximately 70% used insufficient sample sizes, leading to low statistical power and imprecise effect size estimation. (3) Half of the studies did not provide information on test reliability and validity. Finally, this paper discusses the results and their implications for future research and practice.

AB - Because of the substantial development of mobile devices and educational software in recent years, the results of mobile learning–based interventions represent a popular research topic for investigation. Although experiment quality is the basis of empirical research, few studies have explored this issue. Thus, the present study investigated the shortfalls in existing experimental research designs related to mobile learning over the past decade and offers suggestions in this paper. The researchers collected data from all 197 experimental studies on mobile learning published in the Education Resources Information Center and Institute of Science Index from 2003 to 2013. The findings of the present study are described as follows: (1) Quasi-experimental designs represent the most frequently used design type (61%); however, among the quasi-experimental studies, 25% did not consider baseline equivalence. (2) Over half of the studies may not have met basic statistical assumptions, and approximately 70% used insufficient sample sizes, leading to low statistical power and imprecise effect size estimation. (3) Half of the studies did not provide information on test reliability and validity. Finally, this paper discusses the results and their implications for future research and practice.

KW - Experimental design

KW - Mobile learning

KW - Research quality

UR - http://www.scopus.com/inward/record.url?scp=85022180752&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85022180752&partnerID=8YFLogxK

U2 - 10.6209/JORIES.2017.62(2).02

DO - 10.6209/JORIES.2017.62(2).02

M3 - Article

VL - 62

SP - 31

EP - 60

JO - Journal of Research in Education Sciences

JF - Journal of Research in Education Sciences

SN - 2073-753X

IS - 2

ER -