We Agree That Statistical Significance Proves Essentially Nothing: A Rejoinder to Thomas Mayer

In several dozen journal reviews and in many other comments we have received—from, for example, four Nobel laureates, the statistician Dennis Lindley (2012), the statistician Arnold Zellner (2004), the mathematician Olle Häggström (2010), the sociologist Steve Fuller (2008), and the historian Theodore Porter (2008)—no one has ever tried to defend null hypothesis significance testing and its numerous errors. Recent articles by Thomas Mayer (2012, 2013), commenting on our book The Cult of Statistical Significance, are no exception. Of the five major claims we make in our book about the theory and practice of significance testing in economics, Mayer strongly agrees with four. On the fifth claim our disagreement is a matter of degree, not of kind, with no substantive change in results. Overall, Mayer agrees with us and with the new and growing consensus that statistical significance proves essentially nothing and has to change.