The Strength of Statistical Evidence for Composite Hypotheses with an Application to Multiple Comparisons

The strength of the statistical evidence in a sample of data that favors one composite hypothesis over another may be quantified by the likelihood ratio using the parameter value consistent with each hypothesis that maximizes the likelihood function. Unlike the p-value and the Bayes factor, this measure of evidence is coherent in the sense that it cannot support a hypothesis over any hypothesis that it entails. Further, when comparing the hypothesis that the parameter lies outside a non-trivial interval to the hypotheses that it lies within the interval, the proposed measure of evidence almost always asymptotically favors the correct hypothesis under mild regularity conditions. Even at small sample sizes, replacing a simple hypothesis with an interval hypothesis substantially reduces the probability of observing misleading evidence. Sensitivity of the strength of evidence to hypotheses' specification is mitigated by making them fuzzy. The methodology is illustrated in the multiple comparisons setting of gene expression microarray data analysis.