Testing Behavioral Simulation Models by Direct Experiment

The decision rules in simulation models of human behavior purport to describe decision-making behavior as it is and not as it should optimally be. Without the criterion of optimality to judge the appropriateness of a decision rule, simulation modelers must rely on empirical confirmation of the structure of their models. In models of small organizations, traditional social science methods may be used. But these methods are infeasible in models of larger systems such as industries or the macroeconomy. This paper shows how direct experiment can be used to confirm or disconfirm the decision rules in simulation models. Direct experiment uses interactive gaming in which people play a role in the system being modeled. Subjects play the game in an institutional context corresponding to that of the model to be tested, and are given the same information set, but are free to make decisions any way they wish. The behavior of subjects can then be directly compared against the behavior produced by the assumed decision rules of the model. Differences between experiments to test simulation models and experiments in psychology and economics are discussed. The decision rule for capital investment in a simple macroeconomic model is tested as an example of the experimental approach. The model considers interactions between the multiplier and accelerator, a well-known structure underlying popular theories of business fluctuations, but one which has not been tested experimentally. Through their investment decisions, subjects attempt to balance demand and supply over time. The results strongly support the decision rule of the original model. Both model and subjects produce dysfunctional oscillations. The behavior of the subjects is qualitatively and quantitatively similar to that of the original model. Explanations for the bounded rationality of the subjects' behavior are considered, and the correspondence of the experiment to reality is discussed. The results demonstrate the promise of experimental methods as a complementary approach to econometrics in testing simulation models.