Assessing creative problem-solving with automated text grading

Hao Chuan Wang*, Chun Yen Chang, Tsai Yen Li


研究成果: 雜誌貢獻期刊論文同行評審

69 引文 斯高帕斯(Scopus)


The work aims to improve the assessment of creative problem-solving in science education by employing language technologies and computational-statistical machine learning methods to grade students' natural language responses automatically. To evaluate constructs like creative problem-solving with validity, open-ended questions that elicit students' constructed responses are beneficial. But the high cost required in manually grading constructed responses could become an obstacle in applying open-ended questions. In this study, automated grading schemes have been developed and evaluated in the context of secondary Earth science education. Empirical evaluations revealed that the automated grading schemes may reliably identify domain concepts embedded in students' natural language responses with satisfactory inter-coder agreement against human coding in two sub-tasks of the test (Cohen's Kappa = .65-.72). And when a single holistic score was computed for each student, machine-generated scores achieved high inter-rater reliability against human grading (Pearson's r = .92). The reliable performance in automatic concept identification and numeric grading demonstrates the potential of using automated grading to support the use of open-ended questions in science assessments and enable new technologies for science learning.

頁(從 - 到)1450-1466
期刊Computers and Education
出版狀態已發佈 - 2008 12月

ASJC Scopus subject areas

  • 電腦科學(全部)
  • 教育


深入研究「Assessing creative problem-solving with automated text grading」主題。共同形成了獨特的指紋。