This chapter is intended as an introduction to laboratory experiments, when to use, how to evaluate them, why they matter and what are the pitfalls when designing them. I hope that users as well as consumers will find Sections that broaden their views. I start with when an economist might want...

This paper analyzes the design of tests to distinguish human from artificial intelligence through the lens of information design. We identify a fundamental asymmetry: while AI systems can strategically underperform to mimic human limitations, they cannot overperform beyond their capabilities....

In recent years, field experiments have reshaped policy worldwide, but scaling ideas remains a thorny challenge. Perhaps the most important issue facing policymakers today is deciding which ideas to scale. One approach to attenuate this information problem is to augment traditional A/B...

An extensive experimental literature has documented miscoordination in establishing cooperative relationships when they can be supported in indefinitely repeated games: some people systematically try to cooperate, while others do not. The literature has had little success in finding personal...

Lab-in-the-field experiments, in which lab experiments are conducted in more naturalistic settings, are increasingly being implemented in developing country contexts. In this chapter, we outline the conceptual and logistical challenges typically associated with lab-in-the-field experiments in...

This introductory chapter outlines key criteria for evaluating experimental measures, and connects these criteria to the selection of experimental parameters across various contexts. We aim for this chapter to serve as a framework for assessing the different measures, elicitations, and designs...

Preferences for status are typically attributed to two distinct channels: self-image, in which individuals derive utility from being richer than others, and social-image, in which individuals value being seen as richer by others. While both channels are believed to be at play, little is known...

There is much disagreement about the extent to which financial incentives motivate study participants. We elicit preferences for being paid for completing a survey, including a one-in-twenty chance of winning a $100 electronic gift card, a guaranteed electronic gift card with the same expected...

The traditional approach in experimental economics is to use a between-subject design: the analyst places each unit in treatment or control simultaneously and recovers treatment effects via differencing conditional expectations. Within-subject designs represent a significant departure from this...

This paper investigates the weekly evolution of skills as measured by unique data from a widely-emulated early childhood home-visiting program in rural China. The design of the study avoids input endogeneity issues and lack of comparable measures of skills that plague previous studies. Skills,...

This paper presents a framework for how to incorporate prior sources of information into the design of a sequential experiment. These sources can include previous experiments, expert opinions, or the experimenter's own introspection. We formalize this problem using a multi-prior Bayesian...

We measure individual-level loss aversion using three incentivized, representative surveys of the U.S. population (combined N = 3,000). We find that around 50% of the U.S. population is loss tolerant, with many participants accepting negative-expected-value gambles. This is counter to earlier...

Formal analysis plans limit false discoveries by registering and multiplicity adjusting statistical tests. As each registered test reduces power on other tests, researchers prune hypotheses based on prior knowledge, often by combining related indicators into evenly-weighted indices. We propose...

We measure individual-level loss aversion using three incentivized, representative surveys of the U.S. population (combined N = 3,000). We find that around 50% of the U.S. population is loss tolerant, with many participants accepting negative-expected-value gambles. This is counter to earlier...

Field experiments provide the clearest window into the true impact of many policies, allowing us to understand what works, what does not, and why. Yet, their widespread use has not been accompanied by a deep understanding of the political economy of their adoption in policy circles. This study...

We study the problem of estimating the average causal effect of treating every member of a population, as opposed to none, using an experiment that treats only some. We consider settings where spillovers have global support and decay slowly with (a generalized notion of) distance. We derive the...

We use a longitudinal dataset measuring beliefs and behaviors to study the dynamics of model - or narrative - adoption during the Covid-19 pandemic. We show that individuals switch beliefs about the effectiveness of preventive behaviors following changes in perceived risk. The adoption of...

This paper examines the scalability of the results from the Tennessee Student-Teacher Achievement Ratio (STAR) Project, a prominent educational experiment. We explore how the misalignment between the experimental design and the econometric model affects researchers' ability to learn about the...

Economic policies often involve dynamic interventions, where individuals receive repeated interventions over multiple periods. This dynamics makes past responses informative to predict future responses and ultimate outcomes depend on the history of interventions. Despite these phenomena,...

We provide twelve best practices and discuss how each practice can help researchers accurately, credibly, and ethically use Generative AI (GenAI) to enhance experimental research. We split the twelve practices into four areas. First, in the pre-treatment stage, we discuss how GenAI can aid in...

Ethics in social science experimentation and data collection are often discussed but rarely articulated in writing as part of research outputs. Although papers typically reference human subjects research approvals from relevant institutional review boards, most recognize that such boards are not...

We investigate the potential for Large Language Models (LLMs) to enhance scientific practice within experimentation by identifying key areas, directions, and implications. First, we discuss how these models can improve experimental design, including improving the elicitation wording, coding...

"This insightful book examines all aspects of the design process and implementation of questionnaire surveys on the activities of business, public sector, and non-profit organizations. Anthony Arundel discusses how different aspects of the survey method and planned statistical analysis can...

We examine the combined effects of popularity and feelings of being important to reaching a goal by testing how people react to situations in which their own behavior is pivotal or not, as well as the popularity of the action. We conduct a laboratory experiment to cleanly fix beliefs about the...

How should researchers design panel data experiments? We analytically derive the variance of panel estimators, informing power calculations in panel data settings. We generalize Frison and Pocock (1992) to fully arbitrary error structures, thereby extending McKenzie (2012) to allow for...

How should researchers design panel data experiments? We analytically derive the variance of panel estimators, informing power calculations in panel data settings. We generalize Frison and Pocock (1992) to fully arbitrary error structures, thereby extending McKenzie (2012) to allow for...

How should researchers design panel data experiments? We analytically derive the variance of panel estimators, informing power calculations in panel data settings. We generalize Frison and Pocock (1992) to fully arbitrary error structures, thereby extending McKenzie (2012) to allow for...

This study investigates how to use regression adjustment to reduce variance in experimental data. We show that the estimators recommended in the literature satisfy an orthogonality property with respect to the parameters of the adjustment. This observation greatly simplifies the derivation of...

We measure individual-level loss aversion using three incentivized, representative surveys of the U.S. population (combined N=3,000). We find that around 50% of the U.S. population is loss tolerant, with many participants accepting negative-expected-value gambles. This is counter to earlier...

In this paper we study estimation of and inference for average treatment effects in a setting with panel data. We focus on the setting where units, e.g., individuals, firms, or states, adopt the policy or treatment of interest at a particular point in time, and then remain exposed to this...

In this paper we study estimation of and inference for average treatment effects in a setting with panel data. We focus on the setting where units, e.g., individuals, firms, or states, adopt the policy or treatment of interest at a particular point in time, and then remain exposed to this...

Understanding the mechanisms through which treatment effects come about is crucial for designing effective interventions. The identification of such causal mechanisms is challenging and typically requires strong assumptions. This paper discusses identification and estimation of natural direct...

Understanding the mechanisms through which treatment effects come about is crucial for designing effective interventions. The identification of such causal mechanisms is challenging and typically requires strong assumptions. This paper discusses identification and estimation of natural direct...

A modern, decision-theoretic framework can help clarify important practical questions of experimental design. Building on our recent work, this chapter begins by summarizing our framework for understanding the goals of experimenters, and applying this to re-randomization.We then use this...

We propose a new hypothesis, the Power of Certainty, to help explain agents' difficulties in making choices when there are multiple possible payoff-relevant states. In the probabilistic ‘Acquiring-a-Company' problem an agent submits a price to a firm before knowing whether the firm is of low...

How should researchers design experiments to detect treatment effects with panel data? In this paper, we derive analytical expressions for the variance of panel estimators under non-i.i.d. error structures, which inform power calculations in panel data settings. Using Monte Carlo simulation, we...

We propose a new hypothesis, the Power of Certainty, to help explain agents' difficulties in making choices when there are multiple possible payoff-relevant states. In the probabilistic 'Acquiring-a-Company' problem an agent submits a price to a firm before knowing whether the firm is of low or...

We propose a technique for assessing robustness of behavioral measures and treatment effects to experimenter demand effects. The premise is that by deliberately inducing demand in a structured way we can measure its influence and construct plausible bounds on demand-free behavior. We provide...

A modern, decision-theoretic framework can help clarify important practical questions of experimental design. Building on our recent work, this chapter begins by summarizing our framework for understanding the goals of experimenters, and applying this to re-randomization. We then use this...

A modern, decision-theoretic framework can help clarify important practical questions of experimental design. Building on our recent work, this chapter begins by summarizing our framework for understanding the goals of experimenters, and applying this to re-randomization. We then use this...

"Statistical Toolbox for Industrial Applications and Product Development aims to be a reference for practitioners in statistics with an interactive and involving way of exposing contents by making use of both written material and practical examples with specific target on industrial...