Abstract
Test validity is a property of the interpretation assigned to test scores. To provide an objective validating evidence for a standard-referenced assessment is especially important. In this study we utilize a statistical technique, cluster analysis, to explore the validity of one of the expert judgement technique- Yes/No Angoff standard setting method. We first segregated each examinee ability cluster using the hierarchical clustering (HC). Assume that each ability cluster is a Gaussian distribution and that the distribution of each test subject data can be modeled by mixture of Gaussians (MoG), where the mean, variance and the proportion of each cluster were initialized by the HC results. Finally, the ability clustering was implemented by the expectation maximization (EM) method. The results from the traditional standard-setting procedure and cluster analysis were compared. The study suggested that cluster analysis could be applied as a support tool to provide validating information in the process of standard setting for high-stakes achievement tests.
| Original language | English |
|---|---|
| Title of host publication | 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2015 |
| Editors | Zhuo Tang, Jiayi Du, Shu Yin, Renfa Li, Ligang He |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 727-731 |
| Number of pages | 5 |
| ISBN (Electronic) | 9781467376822 |
| DOIs | |
| Publication status | Published - 2016 Jan 13 |
| Event | 12th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2015 - Zhangjiajie, China Duration: 2015 Aug 15 → 2015 Aug 17 |
Publication series
| Name | 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2015 |
|---|
Other
| Other | 12th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2015 |
|---|---|
| Country | China |
| City | Zhangjiajie |
| Period | 15/8/15 → 15/8/17 |
Fingerprint
Keywords
- cluster analysis
- hierarchical clustering
- mixture of Gaussian models
- standard setting
ASJC Scopus subject areas
- Discrete Mathematics and Combinatorics
- Modelling and Simulation
- Computer Networks and Communications
- Control and Systems Engineering
Cite this
A validity study for Yes/No Angoff standard setting method using cluster analysis. / Tseng, Fen Lan; Chiou, Jia Min; Sung, Yao-Ting.
2015 12th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2015. ed. / Zhuo Tang; Jiayi Du; Shu Yin; Renfa Li; Ligang He. Institute of Electrical and Electronics Engineers Inc., 2016. p. 727-731 7382032 (2015 12th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2015).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - A validity study for Yes/No Angoff standard setting method using cluster analysis
AU - Tseng, Fen Lan
AU - Chiou, Jia Min
AU - Sung, Yao-Ting
PY - 2016/1/13
Y1 - 2016/1/13
N2 - Test validity is a property of the interpretation assigned to test scores. To provide an objective validating evidence for a standard-referenced assessment is especially important. In this study we utilize a statistical technique, cluster analysis, to explore the validity of one of the expert judgement technique- Yes/No Angoff standard setting method. We first segregated each examinee ability cluster using the hierarchical clustering (HC). Assume that each ability cluster is a Gaussian distribution and that the distribution of each test subject data can be modeled by mixture of Gaussians (MoG), where the mean, variance and the proportion of each cluster were initialized by the HC results. Finally, the ability clustering was implemented by the expectation maximization (EM) method. The results from the traditional standard-setting procedure and cluster analysis were compared. The study suggested that cluster analysis could be applied as a support tool to provide validating information in the process of standard setting for high-stakes achievement tests.
AB - Test validity is a property of the interpretation assigned to test scores. To provide an objective validating evidence for a standard-referenced assessment is especially important. In this study we utilize a statistical technique, cluster analysis, to explore the validity of one of the expert judgement technique- Yes/No Angoff standard setting method. We first segregated each examinee ability cluster using the hierarchical clustering (HC). Assume that each ability cluster is a Gaussian distribution and that the distribution of each test subject data can be modeled by mixture of Gaussians (MoG), where the mean, variance and the proportion of each cluster were initialized by the HC results. Finally, the ability clustering was implemented by the expectation maximization (EM) method. The results from the traditional standard-setting procedure and cluster analysis were compared. The study suggested that cluster analysis could be applied as a support tool to provide validating information in the process of standard setting for high-stakes achievement tests.
KW - cluster analysis
KW - hierarchical clustering
KW - mixture of Gaussian models
KW - standard setting
UR - http://www.scopus.com/inward/record.url?scp=84966642441&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84966642441&partnerID=8YFLogxK
U2 - 10.1109/FSKD.2015.7382032
DO - 10.1109/FSKD.2015.7382032
M3 - Conference contribution
AN - SCOPUS:84966642441
T3 - 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2015
SP - 727
EP - 731
BT - 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2015
A2 - Tang, Zhuo
A2 - Du, Jiayi
A2 - Yin, Shu
A2 - Li, Renfa
A2 - He, Ligang
PB - Institute of Electrical and Electronics Engineers Inc.
ER -