Cross-sectional impact analysis : bias from dropouts

Several microfinance organizations have begun using a management tool, developed by Assessing the Impact of Microenterprise Services (AIMS) at the United States Agency for International Development (USAID), to assess impact. This tool recommends comparing veteran members to new members of a microcredit program, and attributes any difference to the impact of the program. The tool introduces a potential source of bias into estimates of impact by not instructing organizations to include program dropouts in their calculations. This paper uses data from a longitudinal study in Peru of Mibanco borrowers and non-borrowers to quantify some, but not all, of the biases in the cross-sectional approach. In these data, not including dropouts overestimates the impact of the credit program. Furthermore, we find that the sample composition shifted over the two years (i.e., the characteristics of those who join), introducing further bias into a cross-sectional impact assessment. Note that the “reestimates” here are themselves biased and thus not a recommended procedure. They are calculated merely to assess the attrition and sample composition biases in a cross-sectional approach that compares veterans to new entrants to assess impact. microfinance ; impact methodologies ; program evaluation ; attrition ; Peru