Relative risk of a shuffled deck: a generalizable logical consistency criterion for sample selection in health state valuation studies

In a health state valuation study, respondents may be asked to rank a deck of cards, with each card representing a particular health state. A logical inconsistency occurs when a more severe health state card is ranked higher than a less severe card. Occasional inconsistencies may be justified by errors in judgment or measurement. However, when respondents return shuffled decks, their responses must be removed from the sample; otherwise, valuation estimates will be biased toward the median. In this paper, we present a logical consistency criterion for sample selection in health state valuation studies. This statistical criterion is based on the relative risk of a shuffled deck and generalizable to all health state classification systems, subsets (or decks) of health states, and valuation techniques. We applied the criterion to secondary data collected from 4048 United States and 3395 United Kingdom respondents. In both studies, respondents evaluated 12-card decks of EQ-5D health states using time trade-off and visual analog scale techniques. Among the UK respondents, a small portion (approximately 5%) did not satisfy the criterion; their exclusion significantly changed the sample characteristics and the mean value estimates of the EQ-5D health states. Similar results were found among the US respondents. Copyright © 2006 John Wiley & Sons, Ltd.