The Meaning of Failed Replications: A Review and Proposal

The welcome rise of replication tests in economics has not been accompanied by a single, clear definition of replication. A discrepant replication, in current usage of the term, can signal anything from an unremarkable disagreement over methods to scientific incompetence or misconduct. This paper proposes an unambiguous definition of replication, one that reflects currently common but unstandardized use. It contrasts this definition with decades of unsuccessful attempts to standardize terminology, and argues that many prominent results described as replication tests – in labor, development, and other fields of economics – should not be described as such. Adopting this definition can improve incentives for researchers, encouraging more and better replication tests.