Evenly Sensitive KS-type Inference on Distributions

In its favor, the common Kolmogorov–Smirnov test: 1) is distribution-free and non- parametric, 2) provides uniform confidence bands for the CDF by inversion, 3) may be interpreted as a family of pointwise tests controlling the familywise error rate, 4) can be calibrated to have exact finite-sample size, and 5) computes quickly. However, its low sensitivity in the tails is well known. One recent suggestion uses the probability integral transform to retain (1)–(4), but not (5), while evening the sensitivity (i.e., pointwise type I error rate) across the entire distribution. We make three main contributions. First, we provide a new formula that obviates just-in-time simulation while still achiev- ing (1)–(5) above, for any sample size and familywise error rate. Second, we identify an equivalence with credible sets from a particular nonparametric Bayesian approach. Third, we propose a new two-sample version of the test that allows pre-computation of a lookup table (which we also provide), again obviating just-in-time simulation. The two-sample test may be interpreted as a test of first-order stochastic dominance with the least favorable null. Simulations, empirical examples, and fully operational code are also provided. Classification-JEL: C21