A Monte Carlo Analysis of Alternative Meta-Analysis Estimators in the Presence of Publication Bias

This study uses Monte Carlo analysis to investigate the performances of five different meta-analysis (MA) estimators: the Fixed Effects (FE) estimator, the Weighted Least Squares (WLS) estimator, the Random Effects (RE) estimator, the Precision Effect Test (PET) estimator, and the Precision Effect Estimate with Standard Errors (PEESE) estimator. We consider two types of publication bias: publication bias directed against statistically insignificant estimates, and publication bias directed against wrong-signed estimates. Finally, we consider three cases concerning the distribution of the “true effect”: the Fixed Effects case, where there is only estimate per study, and all studies have the same true effect; the Random Effects case, where there is only one estimate per study, and there is heterogeneity in true effects across studies; and the Panel Random Effects case, where studies have multiple estimates, and true effects are random both across and within studies. Our simulations produce a number of findings that challenge results from previous research.