New IRT models for examinee-selected items

Examinee-selected-item (ESI) design, in which examinees are required to respond to a fixed number of items in a given set of items (e.g., responding to two items in five given items; leading to ten selection patterns), has the advantages of enhancing students’ learning motivation and reducing their testing anxiety. The ESI design yields incomplete data (i.e., only those selected items are answered and the others have missing data). It has been argued that missing data in the ESI design are missing not at random, making standard item response theory (IRT) models inappropriate. Recently, Wang et al. (Journal of Educational Measurement 49(4):419–445, 2012) propose an IRT model for examinee-selected items by adding an additional latent trait to standard IRT models to account for the selection effect. This latent trait could correlate with the intended-to-be-measured latent trait, and the correlation quantifies how stronger the selection effect and how serious the violation of the assumption of missing at random are. In this study, we developed a framework to incorporate this model as a special case and generate several new models. We conducted an experiment to collect real data, in which 501 fifth graders took two mandatory items and four pairs of mathematic (dichotomous) items. In each pair of items, students were first asked to indicate which item they preferred to answer and then answered both items. This is referred to as the “Choose one, Answer all” approach. These new IRT models were fit to the real data and the results were discussed.