An evaluation of parameter estimation when using multilevel structural equation modeling for mediation analysis

Handling of clustered or nested data structures requires the use of multilevel modeling techniques. One such multilevel modeling technique is multilevel structural equation modeling (MLSEM). While estimation of indirect effect parameters and standard errors based on the conventional multilevel model (MMM) has been assessed, this is not the case for the use of the MLSEM model for estimating indirect effects. This simulation study was designed to investigate the use of the MLSEM for estimating mediated effects for the “upper-level” mediation model as compared with the MMM. The following conditions were manipulated: number of clusters (G), within-cluster sample size (nj ), intra-class correlation, measurement error in the mediator, and the true value of the mediated effect derived from various patterns of true values for a and b. The generating model entailed an upper-level mediation model for a cluster-randomized trial that included a dichotomous level two independent variable, a cluster-level latent mediator and an individual-level latent dependent variable both with four indicators. Relative parameter and standard error bias, obtained using the MLSEM and the MMM were evaluated and compared. Percent coverage was calculated and compared when PRODCLIN was used to calculate the confidence interval estimates of the ab effect. Finally, Type I error rates for conditions when ab = 0 were assessed and compared. In addition, statistical power for detecting a truly non-zero mediated effect was tallied and compared across models. Results showed that use of the MMM provided inaccurate and misleading parameter and standard error estimates for the estimates of the mediated effect, especially when the true values of a, b and ab were not zero and the measurement error for M was large. However, the MLSEM estimates were also unacceptable in some of the conditions with small values for G and nj. Researchers are encouraged to use the MLSEM for assessing the multilevel mediated effects when either or both paths a and b are expected to be non-zero, if G is at least 40 and nj is also greater than 40. Results are presented and discussed along with implications for applied researchers intending to assess mediated effect with clustered data.