Chapter 2. The Politics and Practice of Social Experiments : Seeds of a Revolution

Between 1970 and the early 2000s, there was a revolution in support for the use of field experiments to evaluate social programs. Focusing on the welfare reform studies that helped to speed that transformation in the United States, this chapter describes the major challenges to implementing randomized controlled trials (RCTs) in real world conditions, how they emerged and were overcome, and how initial conclusions about conditions necessary to success—strong financial incentives, tight operational control, and small scale—proved to be wrong. Drawing on specific evaluations that were turning points in this 40-year story, the chapter highlights how these studies demonstrated the feasibility of using high-quality RCTs to address important problems, produced results that many considered unusually persuasive, and contributed to the conclusion that alternatives were less credible. It also examines how experimental methods evolved (for example, in the reliance on administrative records and multiarm designs) to meet political, financial, and research needs. The final section discusses lessons from this experience for other fields.