This paper provides concise, nontechnical, step-by-step guidelines on how to conduct a modern meta-analysis, especially in social sciences. We treat publication bias, p-hacking, and heterogeneity as phenomena meta-analysts must always confront. To this end, we provide concrete methodological...

This paper provides concise, nontechnical, step-by-step guidelines on how to conduct a modern meta-analysis, especially in social sciences. We treat publication bias, p-hacking, and heterogeneity as phenomena meta-analysts must always confront. To this end, we provide concrete methodological...

This study uses Monte Carlo analysis to investigate the performances of five different meta-analysis (MA) estimators: the Fixed Effects (FE) estimator, the Weighted Least Squares (WLS) estimator, the Random Effects (RE) estimator, the Precision Effect Test (PET) estimator, and the Precision...

A meta-analysis (MA) aggregates estimated effects from many studies to calculate a single, overall effect. There is no one, generally accepted procedure for how to do this. Several estimators are commonly used, though little is known about their relative performance. A complication arises when...

A meta-analysis is a tool for aggregating estimates of a similar "effect" across many studies. Publication bias is the phenomenon where literature is sample selected in favor of studies having statistically significant results and/or having estimates that satisfy pre-conceived expectations. A...

In this article Robert W. Reed replies to Hong's replication study published earlier this year (IREE, 2019-4).

In mid-2016, all journals of the American Economic Association (AEA) stopped including significance stars in their regression tables. This policy aimed to reduce the emphasis on statistical significance and shift focus toward the broader economic importance of research findings. This study...

Amazon's Mechanical Turk is a very widely-used tool in business and economics research, but how trustworthy are results from well-published studies that use it? Analyzing the universe of hypotheses tested on the platform and published in leading journals between 2010 and 2020 we find evidence of...

Randomized controlled trials (RCTs) are increasingly prominent in economics, with pre-registration and pre-analysis plans (PAPs) promoted as important in ensuring the credibility of findings. We investigate whether these tools reduce the extent of p-hacking and publication bias by collecting and...

Amazon Mechanical Turk is a very widely-used tool in business and economics research, but how trustworthy are results from well-published studies that use it? Analyzing the universe of hypotheses tested on the platform and published in leading journals between 2010 and 2020 we find evidence of...