The limits to and effective use of evaluation methods

This article is about the limits to the usefulness of evaluation methods. Ultimately, those limits are set by lack of knowledge about the ramifications of a plan or project and about the value placed on those ramifications by individuals and communities. Within these limits, evaluation studies frequently fail to reach the potential allowed by the available information. Five key causes of this shortfall in performance are addressed in this study. The first is failure to appreciate the inherent limits of the evaluation exercise, set by disagreement over values and lack of knowledge. The second is lack of attention to the demands of decisionmakers. Here the problem is that many of the established techniques provide neither the range of information nor the type of assessment that is of most use to those involved in decisionmaking. The third is failure to integrate the evaluation exercise into the planning process as a whole so as to ensure a good balance between the use of evaluation as a feedback mechanism to devise and revise alternatives and evaluation used in making judgements by decisionmakers. Overemphasis on the former can lead to a perfunctory attitude toward the latter, whereas overemphasis on the latter may result in a poor selection of options from which to choose. Factors four and five concern more-specific issues. These are the development of strategies to aid in selecting appropriate evaluation techniques and the benefits to be gained by revising those chosen so as to clarify and fit them to the problem at hand. The selection process requires a systematic means of screening and categorisation that at least sorts the methods on the basis of the assumptions they use and the purpose for which they are designed. Finally, the user should be prepared to revise a chosen technique so as to adapt it to the particular problem at hand and to jettison any inappropriate or unreliable features.