Marketing pitfalls of statistical significance testing

Purpose – To explore the appropriateness of statistical significance testing to measure the practical, managerial significance of outcomes in marketing programmes. Design/methodology/approach – An in‐depth analysis of SST's scientific roots is coupled with delineation of a set of general objectives of marketing‐programme measurement to identify the applicability limits of significance testing. Findings – In particular, it is shown that the relatively well known sample‐size dependence of SST and its somewhat lesser known replicability, representativeness and impact fallacies can severely affect the robustness of significance tests. Statistical significance is not the same concept as practical significance. Practical implications – Comprehensive discussion of principles and practice leads to a set of prescriptive usage recommendations, directed at the goal of establishing much‐needed applicability rules and limits for the use of significance‐testing methodologies in an applied marketing context. Originality/value – This robust challenge to the efficacy of significance testing in marketing practice should be of interest to any marketing planner concerned with the collection and use of marketing intelligence.