Development and Validation of a Multimedia-based Assessment of Scientific Inquiry Abilities

Che Yu Kuo, Hsin-Kai Wu, Tsung Hau Jen, Ying-Shao Hsu

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

The potential of computer-based assessments for capturing complex learning outcomes has been discussed; however, relatively little is understood about how to leverage such potential for summative and accountability purposes. The aim of this study is to develop and validate a multimedia-based assessment of scientific inquiry abilities (MASIA) to cover a more comprehensive construct of inquiry abilities and target secondary school students in different grades while this potential is leveraged. We implemented five steps derived from the construct modeling approach to design MASIA. During the implementation, multiple sources of evidence were collected in the steps of pilot testing and Rasch modeling to support the validity of MASIA. Particularly, through the participation of 1,066 8th and 11th graders, MASIA showed satisfactory psychometric properties to discriminate students with different levels of inquiry abilities in 101 items in 29 tasks when Rasch models were applied. Additionally, the Wright map indicated that MASIA offered accurate information about students’ inquiry abilities because of the comparability of the distributions of student abilities and item difficulties. The analysis results also suggested that MASIA offered precise measures of inquiry abilities when the components (questioning, experimenting, analyzing, and explaining) were regarded as a coherent construct. Finally, the increased mean difficulty thresholds of item responses along with three performance levels across all sub-abilities supported the alignment between our scoring rubrics and our inquiry framework. Together with other sources of validity in the pilot testing, the results offered evidence to support the validity of MASIA.

Original languageEnglish
Pages (from-to)2326-2357
Number of pages32
JournalInternational Journal of Science Education
Volume37
Issue number14
DOIs
Publication statusPublished - 2015 Sep 22

Keywords

  • Computer-based Assessment
  • Inquiry Ability
  • Item Response Theory
  • Multimedia-based Assessment
  • Validation

ASJC Scopus subject areas

  • Education

Fingerprint Dive into the research topics of 'Development and Validation of a Multimedia-based Assessment of Scientific Inquiry Abilities'. Together they form a unique fingerprint.

  • Cite this