Classification loss function for parameter ensembles in Bayesian hierarchical models

In this note, we consider the problem of classifying the elements of a parameter ensemble from a Bayesian hierarchical model as above or below a given threshold, C. Two threshold classification losses (TCLs)–termed balanced TCL and p-weighted TCL, respectively–are formulated. The p-weighted TCL can be used to prioritize the minimization of false positives over false negatives or the converse. We prove that, as a special case of a more general result, the p-weighted and balanced TCLs are optimized by the ensembles of unit-specific posterior (1−p)-quantiles and posterior medians, respectively. In addition, we also relate these classification loss functions on parameter ensembles to the concepts of posterior sensitivity and specificity. Finally, we discuss the potential applications of balanced and p-weighted TCLs in Bayesian hierarchical models, and how TCLs could be used to extend existing loss functions currently used for point estimation in parameter ensembles.