Automated scoring for Creative Problem Solving ability with ideation-explanation modeling

Hao Chuan Wang, Chun-Yen Chang, Tsai Yen Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

This paper describes an automated scorer for assessing students' Creative Problem-Solving (CPS) abilities via modeling the intra-structure of students' essays describing their thoughts on solving particular problems. The automated scorer aims to grade students' open-ended responses to an essay-question-type CPS ability test, instead of using typical Likert-type or multiple-choice questions that may be imperfect to assess the creative perspective of human problem-solving. The scorer is distinguishable to most generic automated essay scoring systems that a bipartite graphbased representation is explicitly built for the pair-wise relation between a student's ideas and self-explained reasons for a CPS task. This design will enable several analytical approaches for CPS, such as quantitative scoring and qualitative diagnoses. The preliminary empirical evaluation with 20 students' data shows that the scoring results of the scorer is satisfactory and highly correlated with those of human experts (Pearson's r=.67∼.82) in terms of quantitative scoring task. The approach provides a promising solution to support large-scaled studies on human creativity and may further enable CPS-aware personalization systems.

Original languageEnglish
Title of host publicationProc. Int. Conf. on Computers in Education 2005
Subtitle of host publication"Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE
Pages522-529
Number of pages8
Publication statusPublished - 2005 Dec 1
Event13th International Conference on Computers in Education, ICCE 2005 - Singapore, Singapore
Duration: 2005 Nov 282005 Dec 2

Publication series

NameProc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE

Other

Other13th International Conference on Computers in Education, ICCE 2005
CountrySingapore
CitySingapore
Period05/11/2805/12/2

Fingerprint

ability
Students
student
personalization
Modeling
Scoring
Problem solving
Ideation
creativity
expert
evaluation

ASJC Scopus subject areas

  • Management of Technology and Innovation
  • Education

Cite this

Wang, H. C., Chang, C-Y., & Li, T. Y. (2005). Automated scoring for Creative Problem Solving ability with ideation-explanation modeling. In Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE (pp. 522-529). (Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE).

Automated scoring for Creative Problem Solving ability with ideation-explanation modeling. / Wang, Hao Chuan; Chang, Chun-Yen; Li, Tsai Yen.

Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE. 2005. p. 522-529 (Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, HC, Chang, C-Y & Li, TY 2005, Automated scoring for Creative Problem Solving ability with ideation-explanation modeling. in Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE. Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE, pp. 522-529, 13th International Conference on Computers in Education, ICCE 2005, Singapore, Singapore, 05/11/28.
Wang HC, Chang C-Y, Li TY. Automated scoring for Creative Problem Solving ability with ideation-explanation modeling. In Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE. 2005. p. 522-529. (Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE).
Wang, Hao Chuan ; Chang, Chun-Yen ; Li, Tsai Yen. / Automated scoring for Creative Problem Solving ability with ideation-explanation modeling. Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE. 2005. pp. 522-529 (Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE).
@inproceedings{e276998f77d8416cb45089c73357de65,
title = "Automated scoring for Creative Problem Solving ability with ideation-explanation modeling",
abstract = "This paper describes an automated scorer for assessing students' Creative Problem-Solving (CPS) abilities via modeling the intra-structure of students' essays describing their thoughts on solving particular problems. The automated scorer aims to grade students' open-ended responses to an essay-question-type CPS ability test, instead of using typical Likert-type or multiple-choice questions that may be imperfect to assess the creative perspective of human problem-solving. The scorer is distinguishable to most generic automated essay scoring systems that a bipartite graphbased representation is explicitly built for the pair-wise relation between a student's ideas and self-explained reasons for a CPS task. This design will enable several analytical approaches for CPS, such as quantitative scoring and qualitative diagnoses. The preliminary empirical evaluation with 20 students' data shows that the scoring results of the scorer is satisfactory and highly correlated with those of human experts (Pearson's r=.67∼.82) in terms of quantitative scoring task. The approach provides a promising solution to support large-scaled studies on human creativity and may further enable CPS-aware personalization systems.",
author = "Wang, {Hao Chuan} and Chun-Yen Chang and Li, {Tsai Yen}",
year = "2005",
month = "12",
day = "1",
language = "English",
isbn = "9810540051",
series = "Proc. Int. Conf. on Computers in Education 2005: {"}Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences{"}- Sharing Research Results and Exemplary Innovations, ICCE",
pages = "522--529",
booktitle = "Proc. Int. Conf. on Computers in Education 2005",

}

TY - GEN

T1 - Automated scoring for Creative Problem Solving ability with ideation-explanation modeling

AU - Wang, Hao Chuan

AU - Chang, Chun-Yen

AU - Li, Tsai Yen

PY - 2005/12/1

Y1 - 2005/12/1

N2 - This paper describes an automated scorer for assessing students' Creative Problem-Solving (CPS) abilities via modeling the intra-structure of students' essays describing their thoughts on solving particular problems. The automated scorer aims to grade students' open-ended responses to an essay-question-type CPS ability test, instead of using typical Likert-type or multiple-choice questions that may be imperfect to assess the creative perspective of human problem-solving. The scorer is distinguishable to most generic automated essay scoring systems that a bipartite graphbased representation is explicitly built for the pair-wise relation between a student's ideas and self-explained reasons for a CPS task. This design will enable several analytical approaches for CPS, such as quantitative scoring and qualitative diagnoses. The preliminary empirical evaluation with 20 students' data shows that the scoring results of the scorer is satisfactory and highly correlated with those of human experts (Pearson's r=.67∼.82) in terms of quantitative scoring task. The approach provides a promising solution to support large-scaled studies on human creativity and may further enable CPS-aware personalization systems.

AB - This paper describes an automated scorer for assessing students' Creative Problem-Solving (CPS) abilities via modeling the intra-structure of students' essays describing their thoughts on solving particular problems. The automated scorer aims to grade students' open-ended responses to an essay-question-type CPS ability test, instead of using typical Likert-type or multiple-choice questions that may be imperfect to assess the creative perspective of human problem-solving. The scorer is distinguishable to most generic automated essay scoring systems that a bipartite graphbased representation is explicitly built for the pair-wise relation between a student's ideas and self-explained reasons for a CPS task. This design will enable several analytical approaches for CPS, such as quantitative scoring and qualitative diagnoses. The preliminary empirical evaluation with 20 students' data shows that the scoring results of the scorer is satisfactory and highly correlated with those of human experts (Pearson's r=.67∼.82) in terms of quantitative scoring task. The approach provides a promising solution to support large-scaled studies on human creativity and may further enable CPS-aware personalization systems.

UR - http://www.scopus.com/inward/record.url?scp=84863056226&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863056226&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9810540051

SN - 9789810540050

T3 - Proc. Int. Conf. on Computers in Education 2005: "Towards Sustainable and Scalable Educational Innovations Informed by the Learning Sciences"- Sharing Research Results and Exemplary Innovations, ICCE

SP - 522

EP - 529

BT - Proc. Int. Conf. on Computers in Education 2005

ER -