Sequential Estimation of Dynamic Programming Models with Unobserved Heterogeneity

This paper develops a new computationally attractive procedure for estimating dynamic discrete choice models that is applicable to a wide range of dynamic programming models. The proposed procedure can accommodate unobserved state variables that (i) are neither additively separable nor follow generalized extreme value distribution, (ii) are serially correlated, and (iii) affect the choice set. Our estimation algorithm sequentially updates the parameter estimate and the value function estimate. It builds upon the idea of the iterative estimation algorithm proposed by Aguirregabiria and Mira (2002, 2007) but conducts iteration using the value function mapping rather than the policy iteration mapping. Its implementation is straightforward in terms of computer programming; unlike the Hotz-Miller type estimators, there is no need to reformulate a fixed point mapping in the value function space as that in the space of probability distributions. It is also applicable to estimate models with unobserved heterogeneity. We analyze the convergence property of our sequential algorithm and derive the conditions for its convergence. We develop an approximated procedure which reduces computational cost substantially without deteriorating the convergence rate. We further extend our sequential procedure for estimating dynamic programming models with an equilibrium constraint, which include dynamic game models and dynamic macroeconomic models.