Investigating the performance of level-specific fit indices in multilevel confirmatory factor analysis with dichotomous indicators: A Monte Carlo study

John J.H. Lin*, Hsien Yuan Hsu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

We conducted a Monte Carlo study to examine the performance of level-specific χ2 test statistics and fit regarding their capacity to determine model fit at specific levels in multilevel confirmatory factor analysis with dichotomous indicators. Five design factors—numbers of groups (NG), group size (GS), intra-class correlation (ICC), thresholds of dichotomous indicators (THR), and factor loadings (FL)—were considered in this study. According to our simulation results, we recommend that practitioners should be aware that the performance of between-level-specific (b-l-s) χ2 and fit indices was mainly influenced by ICC and FL, followed by NG. At the same time, THR could slightly weigh in the performance of b-l-s fit indices in some conditions. Both b-l-s χ2 and fit indices were more promising indicators to correctly indicate model fit when ICC or FL increased. A small to medium NG (50–100) might be sufficient for b-l-s χ2 and fit indices only if both ICC and factor loadings were high, while in remaining conditions, an NG of 200 was needed. Moreover, practitioners could use within-level-specific (w-l-s) χ2 and fit indices (except for RMSEAW) along with traditional cut-off values to evaluate within-level models comprising dichotomous indicators. W-l-s χ2 and fit indices were more promising to determine model fit when FL increased. THR had a slight impact and could weigh in the performance of χW2, RMSEAW, CFIW, and TLIW. Unfortunately, RMSEAW was heavily affected by FL and THR and could determine model fit only when FL was high and THR was symmetric.

Original languageEnglish
JournalBehavior Research Methods
DOIs
Publication statusAccepted/In press - 2022

Keywords

  • Confirmatory factor analysis
  • Fit index
  • Intraclass correlation
  • Model evaluation
  • Multilevel structural equation modeling

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Developmental and Educational Psychology
  • Arts and Humanities (miscellaneous)
  • Psychology (miscellaneous)
  • Psychology(all)

Fingerprint

Dive into the research topics of 'Investigating the performance of level-specific fit indices in multilevel confirmatory factor analysis with dichotomous indicators: A Monte Carlo study'. Together they form a unique fingerprint.

Cite this