A Monte Carlo investigation of robustness to nonnormal incomplete data of multilevel modeling

Due to its increasing popularity, hierarchical linear modeling (HLM) has beenused along with structural equation modeling (SEM) to analyze data with nestedstructure. In spite of the extensive research on commonly encountered problems such asviolation of normality and missing data treatment within the framework of SEM, theseareas have been much less explored in HLM. The present study compared HLM andmultilevel SEM through a Monte Carlo study from the perspectives of the influence ofnonnormality and performance of multiple imputation based on the expectationmaximization(EM) algorithm under various combinations of sample sizes at two levels.The statistical power, parameter estimates, standard errors, and estimation bias for themain effects and cross-level interaction in a two- level model were compared across thefour design factors: analysis method, normality condition, missing data proportion, andsample size. HLM and multilevel SEM appeared to have similar power detecting themain effect, while HLM had better power for the cross- level interaction. Neither seemedto be sensitive to violation of the normality assumption. A higher proportion of missingdata resulted in larger standard errors and estimation bias. Sample sizes at both the individual and cluster levels played a role in the statistical power for parameterestimates. The two-way interactions for the four factors were generally nonzero. Overall,both HLM and multilevel SEM were quite robust to violation of normality. SEM appearsmore useful in more complex path models while HLM is superior in detecting maineffects. Multiple imputation based on the EM algorithm performed well in producingstable parameter estimates for up to 30% missing data. Sample size design should takeinto account the level at which the research is most focused.