Effects of item type on student mathematics performance: similar figures as an example

This study investigated the influence of item type on the difficulty and discriminatory power of items and on ninth graders’ mathematics performance. Researchers tested similar figures as competence indicators; three tests were designed with multiple-choice items, completion items, and essay items that corresponded to the same stem, and six common items were equated for each test. The tests were administered to 1,150 students, with 361, 411, and 378 receiving different kind of three tests. The tests were equated using a common-item, nonequivalent group design and the concurrent-calibration method. The difficulty and discriminatory power of items and the ability parameters of the students taking the tests with different item types were estimated with a unidimensional two-parameter partial-credit model and a multidimensional latent regression model. The estimates of the students’ ability parameters were converted to percentile ranks (PRs). The following are the results of the study: (1) The average difficulty of the multiple-choice items was the lowest, and that of the essay items was the highest. (2) The average discriminatory power of the multiple-choice items was the lowest, and that of the essay items was the highest. (3) For 69.04% of the students, the values of three item types differed by 20 or more in PR. (4) Finally, the multidimensional latent regression model revealed that, for 12.61% of the students, the values of three item types differed by 15 or more in PR. The results indicate that item type is related to student mathematics performance through two main types of effect. The researchers further investigated why some students’ performance on the completion and essay items was superior to that on the multiple-choice items and why some students’ performance on the multiple-choice items was superior to that on the essay items.