Nonparametric CAT for CD in Educational Settings With Small Samples

Yuan Pei Chang, Chia Yi Chiu, Rung Ching Tsai

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Cognitive diagnostic computerized adaptive testing (CD-CAT) has been suggested by researchers as a diagnostic tool for assessment and evaluation. Although model-based CD-CAT is relatively well researched in the context of large-scale assessment systems, this type of system has not received the same degree of research and development in small-scale settings, such as at the course-based level, where this system would be the most useful. The main obstacle is that the statistical estimation techniques that are successfully applied within the context of a large-scale assessment require large samples to guarantee reliable calibration of the item parameters and an accurate estimation of the examinees’ proficiency class membership. Such samples are simply not obtainable in course-based settings. Therefore, the nonparametric item selection (NPS) method that does not require any parameter calibration, and thus, can be used in small educational programs is proposed in the study. The proposed nonparametric CD-CAT uses the nonparametric classification (NPC) method to estimate an examinee’s attribute profile and based on the examinee’s item responses, the item that can best discriminate the estimated attribute profile and the other attribute profiles is then selected. The simulation results show that the NPS method outperformed the compared parametric CD-CAT algorithms and the differences were substantial when the calibration samples were small.

Original languageEnglish
Pages (from-to)543-561
Number of pages19
JournalApplied Psychological Measurement
Volume43
Issue number7
DOIs
Publication statusPublished - 2019 Oct 1
Externally publishedYes

Keywords

  • cognitive diagnosis
  • computerized adaptive testing
  • nonparametric classification
  • nonparametric item selection

ASJC Scopus subject areas

  • Social Sciences (miscellaneous)
  • Psychology (miscellaneous)

Fingerprint Dive into the research topics of 'Nonparametric CAT for CD in Educational Settings With Small Samples'. Together they form a unique fingerprint.

  • Cite this