The Inconvenience of Haphazard Sampling

Targeting specific populations can be an arduous task in survey research. While the ability of researchers to use intricate sampling techniques to build a geographically representative population has increased over the last fifty years, the ability to parse together a representative subsample of certain social characteristics has had less success. For instance, cluster samples can decrease costs of interviews by using fewer locations while yielding a representative sample. However, constructing a sampling frame of hidden populations, such as homeless youths or prostitutes, will still consist of using non-random techniques. Many people who share certain characteristics will likely fall in between these extremes. This study looks at a data set, compiled by Greenburg Quinlan Rosner and named Religion and Americas Role in the World (RARW), which compared a nationally representative sample with a subsample of young evangelical Christians. The researchers used a random digit dial to recruit one-fourth of these young evangelicals into a telephone survey, and an internet opt-in, or convenience sample, to recruit the other three-fourths of the evangelicals into a web survey. This mixed-method, mixed-selection strategy poses many interesting methodological questions about representativeness, bias, and the use of post-adjustment strategies. By looking for skew in demographic variables between sample types, and then looking at the sample types in predicting outcome variables to detect bias, it will be possible to look into the first two issues. The use of post-adjustment strategies for dealing with non-random bias is analyzed. Finally, conclusions are presented about the veracity of results of non-probability samples and the advantages and disadvantages of these samples and their post-collection adjustment methods.