Bayesian Model Selection for Small Datasets of Measurement Results

In the Cochrane Database of Systematic Reviews (CDSR) 75% of reported meta-analyses contain five or fewer studies. For a small dataset a reasonable goodness-of-fit test on a statistical model cannot be performed since either it requires a large sample size for the validity of asymptotic approximation or it might be not powerful enough to detect a deviation from the target model. Random effects model under the assumption of normality is commonly used in many fields of science. It also appears to be a classical approach for data reduction in interlaboratory studies in metrology and in meta-analysis in medicine. However, the assumption of normality might not be fulfilled in many practical applications. If a data set is small, then no statistical test on distribution will perform well. The intrinsic Bayes factor is used for selecting an appropriate probability model among several competitors, which not necessarily have to be nested. We apply the proposed methodology to the measurement results used to determine the Newtonian constant of gravitation and the Planck constant.