Replication and the Manufacture of Scientific Inferences : A Formal Approach

The field of replication studies remains a controversial, misunderstood, and unappreciated piñata of 18 replication typologies, spanning 79 replication types. To help bring order into chaos I contribute a theory of manufactured inferences. The theory is built on three pillars: (i) replication causal diagrams (or r-dags, for short); (ii) a formal conceptualization of study procedures; and (iii) the use of Bayesian inference to update our beliefs about the natural phenomenon under investigation, and the operating characteristics of the study procedures used to study it. Next, I use this theory to motivate a formal typology of replications types; explaining how they are done, and for what purpose. Finally, I discuss some implications of this theory, including the importance of an analytical approach to robustness and generalizability replications; the need to avoid conceptual replications; the possibility of legitimate (unplanned) specifications searches; the limitations of meta-analysis; and the false dichotomy between “successful” and “failed” replications