The Impact of Intraclass Correlation on the Effectiveness of Level-Specific Fit Indices in Multilevel Structural Equation Modeling: A Monte Carlo Study

Hsien Yuan Hsu*, Jr Hung Lin, Oi Man Kwok, Sandra Acosta, Victor Willson

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

24 Citations (Scopus)

Abstract

Several researchers have recommended that level-specific fit indices should be applied to detect the lack of model fit at any level in multilevel structural equation models. Although we concur with their view, we note that these studies did not sufficiently consider the impact of intraclass correlation (ICC) on the performance of level-specific fit indices. Our study proposed to fill this gap in the methodological literature. A Monte Carlo study was conducted to investigate the performance of (a) level-specific fit indices derived by a partially saturated model method (e.g., (Formula presented.) and (Formula presented.)) and (b) (Formula presented.) and (Formula presented.) in terms of their performance in multilevel structural equation models across varying ICCs. The design factors included intraclass correlation (ICC: ICC1 = 0.091 to ICC6 = 0.500), numbers of groups in between-level models (NG: 50, 100, 200, and 1,000), group size (GS: 30, 50, and 100), and type of misspecification (no misspecification, between-level misspecification, and within-level misspecification). Our simulation findings raise a concern regarding the performance of between-level-specific partial saturated fit indices in low ICC conditions: the performances of both (Formula presented.) and (Formula presented.) were more influenced by ICC compared with (Formula presented.) and SRMRB. However, when traditional cutoff values (RMSEA≤ 0.06; CFI, TLI≥ 0.95; SRMR≤ 0.08) were applied, (Formula presented.) and (Formula presented.) were still able to detect misspecified between-level models even when ICC was as low as 0.091 (ICC1). On the other hand, both (Formula presented.) and (Formula presented.) were not recommended under low ICC conditions.

Original languageEnglish
Pages (from-to)5-31
Number of pages27
JournalEducational and Psychological Measurement
Volume77
Issue number1
DOIs
Publication statusPublished - 2017 Jan 1
Externally publishedYes

Keywords

  • intraclass correlation
  • level-specific fit index
  • model evaluation
  • multilevel structural equation modeling

ASJC Scopus subject areas

  • Education
  • Developmental and Educational Psychology
  • Applied Psychology
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'The Impact of Intraclass Correlation on the Effectiveness of Level-Specific Fit Indices in Multilevel Structural Equation Modeling: A Monte Carlo Study'. Together they form a unique fingerprint.

Cite this