Effects of Score Transformation on the Composite Scores under the Multivariate Proficiency Distribution Using IRT

This study was designed to examine the composite scores that combine the individual components of the test battery using scale scores resulting from different raw-toscale score transformations under the multivariate proficiency distribution using IRT. The purpose was to evaluate the impact of the three conversion approaches of the linear, the normalizing, and the arcsine transformation on the composite scores obtained by using IRT with the consideration that correlations existed among the examinees' proficiencies. The effects of the transformation were explored via the special case of Kolen, Wang, and Lee's (2012) IRT modeling using empirical data. The five tests of the Basic Competence Test were employed, with a random sample of 5,000 examinees drawn from the data in 2008. The analyses included the summary statistics and frequency distributions of the composite scores, overall SEMs, reliability values and the CSEMs. The results showed that the different transformations led to diverse outcomes of the composite score attributes. Their descriptive statistics, frequency distributions as well as CSEMs were not the same among the various conversion procedures. Composite scores are often formed based on scale scores attained from different forms of conversion; the role of the transformation can be critical in influencing the test results of the examinees' composite scores. Assessing the effects of the score transformation through Kolen et al.'s modeling has helped to understand more about the characteristics of the composite scores under the multivariate environment via IRT.