A practical ad hoc adjustment to the Simes P-value

For testing a set of null hypotheses, the chance of at least one significant result is much higher than the nominal size of each test. The P-value of Simes (1986) controls type-1 error under weak conditions and is far less conservative that the Bonferroni P-value when the tests are correlated. However, it can still be quite conservative. In this paper, I perform a large numerical experiment to measure this conservatism as a function of the correlation of the component P-values and the skewness of the underlying test statistics. The results are modelled, and they produce an adjustment to the Simes P-value which is close to exact for a wide range of correlations and distributional shapes.