From simple growth to numerical simulations: a primer in dynamic programming

These notes provide an intuitive introduction to dynamic programming. The first two Sections present the standard deterministic Ramsey model using the Lagrangian approach. These can be skipped by whom is already acquainted with this framework. Section 3 shows how to solve the well understood Ramsey model by means of a Bellman equation, while Section 4 shows how to "guess" the solution (when this is possible). Section 5 is devoted to applications of the envelope theorem. Section 6 provides a "paper and pencil" introduction to the numerical techniques used in dynamic programming, and can be skipped by the uninterested reader. Sections 7 to 9 are devoted to stochastic modelling, and to stochastic Bellman equations. Section 10 extends the discussion of numerical techniques. An Appendix provides details about the Matlab routines used to solve the examples.