What makes an item more difficult? Effects of modality and type of visual information in a computer-based assessment of scientific inquiry abilities

Hsin Kai Wu, Che Yu Kuo, Tsung Hau Jen, Ying Shao Hsu

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

Effects of multiple external representations on teaching and learning have been widely researched; however, relatively little is understood about how different types of representations used for item presentation influence students' performances in computer-based assessments, particularly in those evaluating complex abilities. We designed a multimedia-based assessment for secondary school students that focused on scientific inquiry abilities (i.e., questioning, experimenting, analyzing, and explaining). Through the participation of 1218 students (561 8th graders and 657 11th graders), the balanced arrangement of test booklets, and the use of a generalized partial credit Rasch model, this study aimed at investigating how the modality of representations for item presentation (i.e., dynamic and static) and type of visual information (i.e., context and content) conveyed by the representations affected the item difficulty of the multimedia-based assessment. The results showed that overall items became slightly more difficult for students when the item presentation was static. Also, the interaction effect between modality and grade was significant; when item presentation was changed from dynamic to static, the items became easier to the 8th graders, whereas they were more difficult to the 11th graders. The results suggested that older students might have more cognitive resources to retrieve information from dynamic displays to solve the assessment tasks successfully. Additionally, while all interactions between modality and type of information on items with context visuals were not significant, there were significant interactions on four items with content visuals. A further examination of the items suggested that the tasks and the phenomena involved in the items may influence how modality affected the item difficulties when content visuals were used. These results implied that researchers need to pay special attention on the design of content visuals because they appear to be more influential than context ones to students' performances of science assessments.

Original languageEnglish
Pages (from-to)35-48
Number of pages14
JournalComputers and Education
Volume85
DOIs
Publication statusPublished - 2015 Jul

Keywords

  • Applications in subject areas
  • Architectures for educational technology system
  • Secondary education

ASJC Scopus subject areas

  • Computer Science(all)
  • Education

Fingerprint Dive into the research topics of 'What makes an item more difficult? Effects of modality and type of visual information in a computer-based assessment of scientific inquiry abilities'. Together they form a unique fingerprint.

  • Cite this