Modeling Rapid Guessing Behaviors in Computer-Based Testlet Items

Kuan Yu Jin*, Chia Ling Hsu, Ming Ming Chiu, Po Hsi Chen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

In traditional test models, test items are independent, and test-takers slowly and thoughtfully respond to each test item. However, some test items have a common stimulus (dependent test items in a testlet), and sometimes test-takers lack motivation, knowledge, or time (speededness), so they perform rapid guessing (RG). Ignoring the dependence in responses to testlet items can negatively bias standard errors of measurement, and ignoring RG by fitting a simpler item response theory (IRT) model can bias the results. Because computer-based testing captures response times on testlet responses, we propose a mixture testlet IRT model with item responses and response time to model RG behaviors in computer-based testlet items. Two simulation studies with Markov chain Monte Carlo estimation using the JAGS program showed (a) good recovery of the item and person parameters in this new model and (b) the harmful consequences of ignoring RG (biased parameter estimates: overestimated item difficulties, underestimated time intensities, underestimated respondent latent speed parameters, and overestimated precision of respondent latent estimates). The application of IRT models with and without RG to data from a computer-based language test showed parameter differences resembling those in the simulations.

Original languageEnglish
Pages (from-to)19-33
Number of pages15
JournalApplied Psychological Measurement
Volume47
Issue number1
DOIs
Publication statusPublished - 2023 Jan

Keywords

  • mixture model
  • multidimensional item response theory
  • rapid guessing, testlet
  • response time

ASJC Scopus subject areas

  • Social Sciences (miscellaneous)
  • Psychology (miscellaneous)

Fingerprint

Dive into the research topics of 'Modeling Rapid Guessing Behaviors in Computer-Based Testlet Items'. Together they form a unique fingerprint.

Cite this