The Linkage between Outcome Differences in Cotton Production and Rural Roads Improvements - A Matching Approach

This paper tests the linkage between a binary treatment (rural road improvement project) and a continuous outcome (cotton productivity) in Zambia’s agro-based Eastern Province as measured by repeated cross-sections of farm-level data from the Zambian post-harvest survey (PHS). We use this PHS dataset, which covers the period from 1996/1997 to 2001/2002 across two phases, the pre-treatment phase (1996/1998) and the treatment phase when the Eastern Province Feeder Road Project (EPFRP) was being implemented (1998/2002). The identification strategy relies on the implementing of matching estimators for all three treatment parameters: Average Treatment Effect (ATE); Treatment on the Treated (TT) and Treatment on the Untreated (TUT), which is crucial in terms of policy relevance (Arcand, 2012). Matching ensures a sub-set of non-project areas that best represents the counterfactual and is done at the same geographic level of aggregation (van de Walle, 2009). Since treatment participation is not by random asignment we use the propensity score as a method to reduce the bias in the estimation of these treatment effects with observational PHS data sets in order to reduce the dimensionality of the matching problem. We find the ATT estimation results are not the same when implementing various matching using ‘the logarithm of (cotton) yield’ compared to using ‘cotton productivity’ as variable. In the latter case the following matching methods all have negative difference between treated and controls: 1-to-1 propensity score matching; k-nearest neighbours matching; radius matching; and 'spline-smoothing'. However, the Kernel matching has positive difference between treated and controls for the ‘productivity’ variable: Finally, some of the local linear regression and the Mahalanobis matching specifications yields positive difference between treated and controls for the ‘logyield’ variable, but not for the ‘productivity’ variable and not for all specifications either. Through our robustness checks of the Matching Assumpion and Sensitivity of Estimates we find that the matching doesn’t reduce the starting unbalancing. The comparison of the simulated ATT and the baseline ATT tells us that the latter is robust. We conclude that the application of various non-parametric matching methods didn’t enable us to identify a robust linkage, most likely due to the PHS data source and the evaluation design. Future rigorous rural roads impact evaluation requires panel (with pre-intervention) data for project and appropriate non-project areas, which allows for an evaluation design that combines a double difference (DID) with controls for initial conditions either through propensity score matching, regression controls or an IV (van de Walle, 2009). Regression discontinuity designs would offer an alternative method for impact evaluation (ADB, 2011; see Arcand, 2012).