Is Using a SRI to determine the fate of teachers or commencement of work suitable? A mixture IRT analysis

Ming Ci Tseng, Haw Jeng Chiou, Te Sheng Chang, Pao Feng Lo

Research output: Contribution to journalArticle

Abstract

This study examines the effects of variability in student ratings regarding instruction on decision-making for faculty teaching evaluation. A total of 6,111 undergraduate students from 173 classes in a university on the east coast of Taiwan were included in the research sample. This study is different from previous studies regarding student ratings for instruction that are constructed in classical test theory. We use item response theory to analyze the heterogeneity of students, to rigorously examine the effects on student ratings regarding instruction. The results show that teachers may easily exceed the teaching criterion score set by the university when not considering the heterogeneity of the student ratings. However, the different latent types of the variability of student ratings may be important for interpreting the results of different student rating scores. The recommendations for university teaching and student ratings regarding instruction are created based on the results from this study.

Original languageEnglish
Pages (from-to)91-116
Number of pages26
JournalJournal of Research in Education Sciences
Volume58
Issue number1
DOIs
Publication statusPublished - 2013 Jan 1

Fingerprint

rating
teacher
student
instruction
university teaching
test theory
university
Teaching
Taiwan
decision making
evaluation

Keywords

  • Mixture IRT analysis
  • Student ratings of instruction

ASJC Scopus subject areas

  • Education

Cite this

Is Using a SRI to determine the fate of teachers or commencement of work suitable? A mixture IRT analysis. / Tseng, Ming Ci; Chiou, Haw Jeng; Chang, Te Sheng; Lo, Pao Feng.

In: Journal of Research in Education Sciences, Vol. 58, No. 1, 01.01.2013, p. 91-116.

Research output: Contribution to journalArticle

@article{bdc37c4b414b44fcb63eba15dfd9548c,
title = "Is Using a SRI to determine the fate of teachers or commencement of work suitable? A mixture IRT analysis",
abstract = "This study examines the effects of variability in student ratings regarding instruction on decision-making for faculty teaching evaluation. A total of 6,111 undergraduate students from 173 classes in a university on the east coast of Taiwan were included in the research sample. This study is different from previous studies regarding student ratings for instruction that are constructed in classical test theory. We use item response theory to analyze the heterogeneity of students, to rigorously examine the effects on student ratings regarding instruction. The results show that teachers may easily exceed the teaching criterion score set by the university when not considering the heterogeneity of the student ratings. However, the different latent types of the variability of student ratings may be important for interpreting the results of different student rating scores. The recommendations for university teaching and student ratings regarding instruction are created based on the results from this study.",
keywords = "Mixture IRT analysis, Student ratings of instruction",
author = "Tseng, {Ming Ci} and Chiou, {Haw Jeng} and Chang, {Te Sheng} and Lo, {Pao Feng}",
year = "2013",
month = "1",
day = "1",
doi = "10.3966/2073753X2013035801004",
language = "English",
volume = "58",
pages = "91--116",
journal = "Journal of Research in Education Sciences",
issn = "2073-753X",
publisher = "國立臺灣師範大學",
number = "1",

}

TY - JOUR

T1 - Is Using a SRI to determine the fate of teachers or commencement of work suitable? A mixture IRT analysis

AU - Tseng, Ming Ci

AU - Chiou, Haw Jeng

AU - Chang, Te Sheng

AU - Lo, Pao Feng

PY - 2013/1/1

Y1 - 2013/1/1

N2 - This study examines the effects of variability in student ratings regarding instruction on decision-making for faculty teaching evaluation. A total of 6,111 undergraduate students from 173 classes in a university on the east coast of Taiwan were included in the research sample. This study is different from previous studies regarding student ratings for instruction that are constructed in classical test theory. We use item response theory to analyze the heterogeneity of students, to rigorously examine the effects on student ratings regarding instruction. The results show that teachers may easily exceed the teaching criterion score set by the university when not considering the heterogeneity of the student ratings. However, the different latent types of the variability of student ratings may be important for interpreting the results of different student rating scores. The recommendations for university teaching and student ratings regarding instruction are created based on the results from this study.

AB - This study examines the effects of variability in student ratings regarding instruction on decision-making for faculty teaching evaluation. A total of 6,111 undergraduate students from 173 classes in a university on the east coast of Taiwan were included in the research sample. This study is different from previous studies regarding student ratings for instruction that are constructed in classical test theory. We use item response theory to analyze the heterogeneity of students, to rigorously examine the effects on student ratings regarding instruction. The results show that teachers may easily exceed the teaching criterion score set by the university when not considering the heterogeneity of the student ratings. However, the different latent types of the variability of student ratings may be important for interpreting the results of different student rating scores. The recommendations for university teaching and student ratings regarding instruction are created based on the results from this study.

KW - Mixture IRT analysis

KW - Student ratings of instruction

UR - http://www.scopus.com/inward/record.url?scp=84940300358&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940300358&partnerID=8YFLogxK

U2 - 10.3966/2073753X2013035801004

DO - 10.3966/2073753X2013035801004

M3 - Article

AN - SCOPUS:84940300358

VL - 58

SP - 91

EP - 116

JO - Journal of Research in Education Sciences

JF - Journal of Research in Education Sciences

SN - 2073-753X

IS - 1

ER -