Development of Chinese computerized adaptive test system based on higher-order item response theory

The purpose of this empirical study was to develop a higher-order computerized adaptive test (HCAT) system based on higher-order Item Response Theory (HOIRT) for a Chinese proficiency test (CPT). This study contains four steps, namely item categorization, ability estimation, item selection and system development. The development of the HCAT system succeeds in making four major contributions; the first is the original CPTs system developed for computerized adaptive test, while the second is the application of one factor within-item HO-IRT structure which enables the item selection procedure adopted next item simultaneously based on the individual test-taker’s three domain abilities (linguistic, sociolinguistic and pragmatic), thirdly is the application of the maximum a posteriori (MAP) estimator enables test-takers averagely responded 10 out of 43 items to complete the test, and last, but not least, the HCAT system is able to directly reduce the administration costs associated with the test compared with either the current traditional paper-and-pencil or computer-based CPTs. These results make CPT a multi-functional and practical test, as the domain abilities can be used in a formative assessment for diagnostic purposes, and the overall ability can be provided on the overall level of performance for a summative assessment.