Comparing sample surveys of health with official population statistics: some methodological issues and empirical findings

This article reports an examination of factors that might affect the interpretation of computed relative risks of medical illness obtained when estimates of morbidity derived from a specific subpopulation sample are compared with estimates obtained from official statitical bureau surveys of the parent population. Interpreting a significant or nonsignificant relative risk depends upon how fair the comparison is. Three types of error are considered: (1) those arising from subjects, including coverage, selection, response and comparability; (2) those arising from measurement processes, including data collection instruments, interviewers and data processing; and (3) those arising from confounding, which occurs when a risk factor for outcome is differentially distributed between exposure groups. An example is given from a comparison of two sets of morbidity estimates, one obtained from an epidemiological cohort study of a national random list sample of Australian Vietnam veterans, the other obtained by an Australian Bureau of Statistics national area probability sample. Variables measuring prevalence of 37 recent and chronic illness conditions were derived from both studies and their ratio computed as the relative risk of illness in the veteran sample compared with the Australian population sample. This risk was greater than 1.0 for all except one condition, and 18 of 36 recent conditions and 30 of 37 chronic conditions carried 99% confidence intervals excluding 1.0. Adjustment for variables that were thought to have potentially biased the relative risks gave varied results. The issue of when an adjustment becomes an overadjustment depends upon the meaning of the variable and its place in a putative causal pathway.