The Size and Power of Bootstrap Tests

Bootstrap tests are tests for which the significance level is calculated by some sort of bootstrap procedure, which may be parametric or nonparametric. We show that, in many circumstances, the size distortion of a bootstrap P value for a test will be one whole order of magnitude smaller than that of the corresponding asymptotic P value. We also show that, at least in the parametric case, the magnitude of the distortion will depend on the shape of what we call the P value function. As regards the power of bootstrap tests, we show that the size-corrected power of a bootstrap test differs from that of the corresponding asymptotic test only by an amount of the same order of magnitude as the size distortion, and of arbitrary sign. Monte Carlo results are presented for two cases of interest: tests for serial correlation and nonnested hypothesis tests. These results confirm and illustrate the utility of our theoretical results, and they also suggest that bootstrap tests will often work extremely well in practice.