Restrictions of empirical policy analyses: the example of the evaluation of rural development policies

The present paper asks under what circumstances a standardisation of evaluations would be feasible in order to enable a comprehensible aggregation of results for the European administration. We argue that in the complex environment of rural development the adequate definition of system boundaries is a precondition for the successful application of empirical methods and the identification of causal effects. If macro effects and self-enforcing effects are important, the objects of inquiry have to be defined on a higher observational level. In this case, the statistical identification may not be possible because there might be hardly any comparable (“counterfactual”) observations. We conclude that evaluators need definite theoretical guidance in order to define consistently their field of inquiry. Only then, the goal of comparable and aggregable quantified results might be achievable to a certain degree.