The effectiveness of automated writing evaluation in EFL/ESL writing: a three-level meta-analysis

Thuy Thi Nhu Ngo, Howard Hao Jan Chen*, Kyle Kuo Wei Lai

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

13 Citations (Scopus)

Abstract

The present study performs a three-level meta-analysis to investigate the overall effectiveness of automated writing evaluation (AWE) on EFL/ESL student writing performance. 24 primary studies representing 85 between-group effect sizes and 34 studies representing 178 within-group effect sizes found from 1993 to 2021 were separately meta-analyzed. The results indicated a medium overall between-group effect size (g = 0.59) and a large overall within-group effect size (g = 0.98) of AWE on student writing performance. Analyses of moderators show that: (1)- AWE is more effective in improving vocabulary usage but less effective in improving grammar in students’ writing; (2)- Grammarly shows potential in being a highly effective tool, though Pigai did not demonstrate such effectiveness; (3)- Medium to long duration of AWE usage leads to a higher effect, but short duration leads to a lower effect in writing outcome compared to non-AWE treatment; (4)- Studying with peers in AWE condition potentially produces a large effect; (5)- AWE is beneficial to students at the undergraduate level, students in the EFL context, and students with intermediate English proficiency. Directions for future research are also discussed in the present study. Overall, AWE is a beneficial application and is recommended for integration in the writing classroom.

Original languageEnglish
Pages (from-to)727-744
Number of pages18
JournalInteractive Learning Environments
Volume32
Issue number2
DOIs
Publication statusPublished - 2024

Keywords

  • AWE
  • Automated writing evaluation
  • effectiveness
  • meta-analysis
  • writing

ASJC Scopus subject areas

  • Education
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'The effectiveness of automated writing evaluation in EFL/ESL writing: a three-level meta-analysis'. Together they form a unique fingerprint.

Cite this