Measuring non-response bias in a cross-country enterprise survey

Non-response is a common issue affecting the vast majority of surveys, and low non-response is usually associated with higher quality. However, efforts to convince unwilling respondents to participate in a survey might not necessarily result in a better picture of the target population. It can lead to higher, rather than lower, non-response bias, for example if incentives are effective only for particular groups, e.g. in a business survey, if the incentives tend to attract mainly larger companies or enterprises encountering financial difficulties. We investigate the impact of non-response in the European Commission and European Central Bank Survey on the Access to Finance of Enterprises (SAFE), which collects evidence on the financing conditions faced by European small and medium-sized enterprises compared with those of large firms. This survey, which has been conducted by telephone biannually since 2009 by the European Central Bank and the European Commission, provides a valuable means of searching for this kind of bias, given the high heterogeneity of response propensities across countries. The study relies on so-called “Representativity Indicators” developed within the Representativity Indicators of Survey Quality (RISQ) project, which measure the distance to a fully representative response. On this basis, we examine the quality of the SAFE at different stages of the fieldwork as well as across different survey waves and countries. The RISQ methodology relies on rich sampling frame information, although this is partially limited in the case of the SAFE. To enrich this frame information, we also assess the representativity of a SAFE subsample created by linking the survey responses with the companies’ financial information as obtained from a business register. This subsampling is another potential source of bias which we also attempt to quantify. Finally, we suggest possible ways of improving the monitoring of the non-response bias in future rounds of the survey.