Comparative Reliability of Verdicts

We consider the problem of evaluating the reliability of a verdict given by a panel of judges. Given no information other than the number of panelists for and against, we address the question of when is a verdict that was obtained by a majority of k₁ vs. j₁ more or less reliable than one reached by k₂ vs. j₂. We define criteria and investigate which verdicts are comparable and which are not. Consequences of this study may have bearing on choice of panel size and decision rule for decision-making bodies, such as courts, juries, committees, and boards. As implied by the above, our perspective is a posterior view of reliability, though it also entails prior concern regarding how the reliability of a verdict will be perceived after being delivered. As an example, we apply our results to comparing the reliability of different verdicts handed down by the Supreme Court of the State of Israel and assessing the option of expanding a hearing on a case from a bench of three judges to a larger panel.