Use of Expert Ratings as Sampling Strata for a More Cost-Effective Probability Sample of a Rare Population

We consider situations in which externally observable characteristics allow experts to quickly categorize individual households as likely or unlikely to contain a member of a rare target population. This classification can form the basis of disproportionate stratified sampling such that households classified as “unlikely” are sampled at a lower rate than those classified as “likely,” thereby reducing screening costs. Design weights account for this approach and allow unbiased estimates for the target population. We demonstrate that with sensitivity and specificity of expert classification at least 70 percent, and ideally at least 80 percent, our approach can economically increase effective sample size for a rare population. We develop heuristics for implementing this approach and demonstrate that sensitivity drives design effects and screening costs whereas specificity only drives the latter. We demonstrate that the potential gains from this approach increase as the target population becomes rarer. We further show that for most applications, unlikely strata should be sampled at 1/6 to 1/2 the rate of likely strata. This approach was applied to a survey of Cambodian immigrants in which the 82 percent of households rated “unlikely” were sampled at 1/4 the rate as “likely” households, reducing screening from 9.4 to 4.0 approaches per complete. Sensitivity and specificity were 86 percent and 91 percent, respectively. Weighted estimation had a design effect of 1.26, so screening costs per effective sample size were reduced by 47 percent. We also note that in this instance, expert classification appeared to be uncorrelated with survey outcomes of interest among eligibles