Document Type
Article
Publication Date
3-23-2016
Keywords
cross-classified multilevel data, measurement invariance, multilevel confirmatory factor analysis, cross-classified MIMIC, non-hierarchical structure data, simulations, MonteCarlo
Digital Object Identifier (DOI)
https://doi.org/10.3389/fpsyg.2016.00328
Abstract
In educational settings, researchers are likely to encounter multilevel data with cross-classified structure. However, due to the lack of familiarity and limitations of statistical software for cross-classified modeling, most researchers adopt less optimal approaches to analyze cross-classified multilevel data in testing measurement invariance. We conducted two Monte Carlo studies to investigate the performances of testing measurement invariance with cross-classified multilevel data when the noninvarinace is at the between-level: (a) the impact of ignoring crossed factor using conventional multilevel confirmatory factor analysis (MCFA) which assumes hierarchical multilevel data in testing measurement invariance and (b) the adequacy of the cross-classified multiple indicators multiple causes (MIMIC) models with cross-classified data. We considered two design factors, intraclass correlation (ICC) and magnitude of non-invariance. Generally, MCFA demonstrated very low statistical power to detect non-invariance. The low power was plausibly related to the underestimated factor loading differences and the underestimated ICC due to the redistribution of the variance component from the ignored crossed factor. The results demonstrated possible incorrect statistical inferences with conventional MCFA analyses that assume multilevel data as hierarchical structure for testing measurement invariance with cross-classified data (non-hierarchical structure). On the contrary, the cross-classified MIMIC model demonstrated acceptable performance with cross-classified data.
Rights Information
This work is licensed under a Creative Commons Attribution 4.0 License.
Was this content written or created while at USF?
Yes
Citation / Publisher Attribution
Frontiers in Psychology, v. 7, art. 328
Scholar Commons Citation
Im, Myung H.; Kim, Eun Sook; Kwok, Oi-Man; Yoon, Myeongsun; and Willson, Victor L., "Impact of Not Addressing Partially Cross-Classified Multilevel Structure in Testing Measurement Invariance: A Monte Carlo Study" (2016). Educational and Psychological Studies Faculty Publications. 195.
https://digitalcommons.usf.edu/esf_facpub/195
