Chapter 14 Numerical dynamic programming in economics

This chapter explores the numerical methods for solving dynamic programming (DP) problems. The DP framework has been extensively used in economics because it is sufficiently rich to model almost any problem involving sequential decision making over time and under uncertainty. The chapter focuses on continuous Markov decision processes (MDPs) because these problems arise frequently in economic applications. Although, complexity theory suggests a number of useful algorithms, the theory has relatively little to say about important practical issues, such as determining the point at which various exponential-time algorithms such as Chebyshev approximation methods start to blow up, making it optimal to switch to polynomial-time algorithms. In future work, it will be essential to provide numerical comparisons of a broader range of methods over a broader range of test problems, including problems of moderate to high dimensionality.