How Big Is Big? A Caveat Regarding Difference-in-Differences Analysis

Difference-in-differences, or “D-in-D,” is perhaps the most broadly applied econometric technique in retrospective analyses of competition matters. We discuss a possible pitfall regarding this procedure. We argue that a positive and significant event variable coefficient is not a sufficient condition for concluding there have been anticompetitive price effects. We use simulations to demonstrate that even in cases where the alleged anticompetitive activity had no anticompetitive effect, the D-in-D procedure can still produce positive and significant event variables. This paper does not take issue with D-in-D in principle but rather as it is often practiced. Our results imply that while D-in-D is an important tool, the researcher must conduct additional analyses to put the D-in-D result into context before concluding a significant event variable is indicative of anticompetitive effects. We suggest a specific approach. We note that our results may have important implications for the current state of the academic literature regarding retrospectives in antitrust as well as for practitioners