Assessing Whether Decisions Are More Sensitive to Preference or Prediction Uncertainty with a Value of Information Approach

In many decisions, we are not only uncertain about the predicted outcomes of decision alternatives but also about stakeholder preferences regarding these outcomes. Further information collection may reduce uncertainties, but is costly. We present and apply a framework to identify the most decisive uncertainties and prioritize data collection efforts based on value of information (VoI) sensitivity analysis. Preference uncertainty is usually not explicitly considered in VoI analysis or in standard utility theory. Based on the expected expected utility (EEU) concept, we consider uncertain predictions and preferences jointly in decisions and subsequent VoI analysis. We focus on the expected value of partially perfect information (EVPPI) and adapt a fast, given-data algorithm for estimating this metric. The framework is motivated by complex environmental decision problems and we apply it to a hypothetical multi-criteria decision regarding coral reef management with conflicting stakeholder perspectives. The results show that better understanding of stakeholder positions can be as relevant as improving system understanding. For one perspective, preference model parameters had the highest EVPPI, while for another predictive uncertainties of the reef system attributes were more relevant. For two perspectives, the decision was largely insensitive. By considering predictive and preferential uncertainty on an equal footing in VoI analysis, we open up possibilities to design data collection for decision support processes more efficiently