Improving development effectiveness through R&D: Dynamic learning and evaluation

Research and development (R&D) is a common process in for-profit organizations. Despite the benefits, it is not routinely practiced in nonprofit organizations, in part because it is difficult to identify the effects of programs that are designed to involve individuals over long periods of time. This paper presents a process by which organizations looking to affect social outcomes can learn from their programs in both the short- and long-run in order to develop the most cost- and impact-effective programs. We call it Dynamic Learning and Evaluation (DLE). DLE is a multi-arm experimental approach to program development that encompasses all stages of the design and implementation process. It combines a clear model of the causal chain of a program with high quality monitoring and impact evaluation. During the initial program development, organizations randomly apply multiple implementation designs and test them against each other using qualitative and administrative data. Once the organization determines a combination of designs that hold the most potential, they then implement these designs in the field and estimate impacts using participant data collection processes. The organization then uses the results to inform the next round of program implementation. They repeat this process over multiple designs for the life of the program and organization. At no point in the lifespan of the organization is this learning process stopped: programs are continually updated using systematic and objective methods to improve their design and impact. We present this process in detail.