TY - GEN
T1 - Using cluster analysis to validate the angoff standard setting method in mixed-format assessments
AU - Tseng, Fen Lan
AU - Chiou, Jia Min
AU - Sung, Yao Ting
N1 - Publisher Copyright:
© 2017 IEEE.
PY - 2018/6/21
Y1 - 2018/6/21
N2 - Test validity is a property of the interpretation assigned to test scores. To provide an objective validating evidence for a standard-referenced assessment is especially important. In this study we utilize a statistical technique, cluster analysis, to explore the validity of one of the expert judgement technique-Yes/No Angoff standard setting method. We first segregated each examinee ability cluster using the hierarchical clustering (HC). Assume that each ability cluster is a Gaussian distribution and that the distribution of each test subject data can be modeled by mixture of Gaussians (MoG), where the mean, variance and the proportion of each cluster were initialized by the HC results. Finally, the ability clustering was implemented by the expectation maximization (EM) method. The results from the traditional standard-setting procedure and cluster analysis were compared. The study concludes that cluster analysis appears useful for helping to set standards on educational tests. In addition, it suggested that cluster analysis could be applied as a support tool to provide validating information in the process of standard setting for high-stakes achievement tests.
AB - Test validity is a property of the interpretation assigned to test scores. To provide an objective validating evidence for a standard-referenced assessment is especially important. In this study we utilize a statistical technique, cluster analysis, to explore the validity of one of the expert judgement technique-Yes/No Angoff standard setting method. We first segregated each examinee ability cluster using the hierarchical clustering (HC). Assume that each ability cluster is a Gaussian distribution and that the distribution of each test subject data can be modeled by mixture of Gaussians (MoG), where the mean, variance and the proportion of each cluster were initialized by the HC results. Finally, the ability clustering was implemented by the expectation maximization (EM) method. The results from the traditional standard-setting procedure and cluster analysis were compared. The study concludes that cluster analysis appears useful for helping to set standards on educational tests. In addition, it suggested that cluster analysis could be applied as a support tool to provide validating information in the process of standard setting for high-stakes achievement tests.
KW - cluster analysis
KW - hierarchical clustering
KW - mixture of Gaussian model
KW - standard setting
UR - http://www.scopus.com/inward/record.url?scp=85050218821&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85050218821&partnerID=8YFLogxK
U2 - 10.1109/FSKD.2017.8393175
DO - 10.1109/FSKD.2017.8393175
M3 - Conference contribution
AN - SCOPUS:85050218821
T3 - ICNC-FSKD 2017 - 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery
SP - 2540
EP - 2546
BT - ICNC-FSKD 2017 - 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery
A2 - Zhao, Liang
A2 - Wang, Lipo
A2 - Cai, Guoyong
A2 - Li, Kenli
A2 - Liu, Yong
A2 - Xiao, Guoqing
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, ICNC-FSKD 2017
Y2 - 29 July 2017 through 31 July 2017
ER -