Systematic Reviews: Questions, Methods and Usage

While their application in the context of development aid is quite new, systematic reviews have been used and debated in other fields for various decades. Although evidence gathering and synthesis in the context of development aid inarguably faces challenges distinct from other fields, it is expected that experiences about systematic review methodology from other contexts can provide useful inputs to the field of development aid evaluation. Systematic reviews operate in a complex multidisciplinary environment, which requires acknowledging the influence of institutions and social interaction. The Evaluation Study suggests that the scarcity of comparable evidence about the effects of development interventions necessitates that authors of systematic reviews change their strategies when assessing the strength of evidence or synthesizing data. The focus on asking the ‘right’ questions in international development reviews is important precisely because no review process is immune to bias. The study emphasizes that since systematic reviews in international development may be vulnerable to a range of biases. Systematic reviews should not aim, at all cost, at pursuing the classical approach suitable for traditional, ‘easy-to-measure’ situations.