Enhancing objectivity and decision relevance : a better framework for evaluating cohesion policies

By international comparison as well as compared to other EU policies, the EU’s Cohesion Policy (CP) evaluation system is far developed and institutionalized. This paper analyses the remaining gaps and shortcomings in the CP evaluation system against principles established by the OECD and others and provides recommendations on how to further improve it. The presence of a broad and imprecise CP objective function emerges as a key challenge for evaluations. The evaluation culture is not equally developed among all Member States and regions. In quite some cases, an unfavorable equilibrium is found which is characterized by limited evaluation capacities, poor methods, and a formalistic approach to evaluations. Program evaluations in the Member States are usually commissioned by national or regional managing authorities who have a vested interest in promoting the success of their programs. Evaluations are carried out by evaluators who are functionally independent, but often lack factual independence. There is also limited international competition in the market for evaluations commissioned by national or regional authorities. Evaluation methods applied in CP program evaluations mostly lag behind academic advancements and evaluation reports often do not transparently describe their methodological limitations. As the EU body responsible for implementing CP across all 27 Member States, the Commission may also have an overly optimistic perspective on CP. Finally, there is little evidence that evaluation findings are used for decision-making processes, funding allocation and the design of programs. The paper offers a number of recommendations how to advance the evaluation system: (1) Reorient CP reforms towards a more focused set of objectives; (2) Specify evaluation obligations more precisely in the Common Provision Regulation and set out a ‘charter for evaluators’; (3) Introduce an ‘evaluate first’ requirement when preparing or updating programs; (4) Promote the use of counterfactual methods; (5) Explicitly link funding decisions at program and policy level to evaluation results; (6) Implement measures to stimulate a European market for CP evaluations; and (7) establish a standing European Advisory Panel on CP evaluation to foster independent third-party reviews.