Bayesian Estimation of Finite-Horizon Discrete Choice Dynamic Programming Models

We develop a Bayesian Markov chain Monte Carlo (MCMC) algorithm for estimating finite-horizon discrete choice dynamic programming (DDP) models. The proposed algorithm has the potential to reduce the computational burden significantly when some of the state variables are continuous. In a conventional approach to estimating such a finite-horizon DDP model, researchers achieve a reduction in estimation time by evaluating value functions at only a subset of state points and applying an interpolation method to approximate value functions at the remaining state points (e.g., Keane and Wolpin 1994). Although this approach has proven to be effective, the computational burden could still be high if the model has multiple continuous state variables or the number of periods in the time horizon is large. We propose a new estimation algorithm to reduce the computational burden for estimating this class of models. It extends the Bayesian MCMC algorithm for stationary infinite-horizon DDP models proposed by Imai, Jain and Ching (2009) (IJC). In our algorithm, we solve value functions at only one randomly chosen state point per time period, store those partially solved value functions period by period, and approximate expected value functions nonparametrically using the set of those partially solved value functions. We conduct Monte Carlo exercises and show that our algorithm is able to recover the true parameter values well. Finally, similar to IJC, our algorithm allows researchers to incorporate flexible unobserved heterogeneity, which is often computationally infeasible in the conventional two-step estimation approach (e.g., Hotz and Miller 1993; Aguirregabiria and Mira 2002)