Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter

1. Introduction and summary -- I. Countable state space -- 2. Decision models and definition of the problem -- 3. The principle of optimality and the optimality equation -- 4. Value iteration -- 5. Criteria of optimality and existence of $$\bar{p} $$-optimal plans -- 6. Sufficient statistics, Markovian and stationary models -- 7. Models with incomplete information -- 8. Examples of special models -- 9. Randomized plans -- 10. Dynamic programming under uncertainty -- II. General state space -- 11. Decision models -- 12. Measure-theoretic and topological preparations -- 13. Universal measurability of the maximal conditional expected reward -- 14. The optimality equation -- 15. Substitution of randomized plans by deterministic plans -- 16. A generalization of the fixed point theorem for contractions -- 17. Criteria of optimality and existence of $$\bar{p} $$-optimal plans -- 18. Sufficient statistics, Markovian and stationary models -- 19. Validity of the optimality equation without topological assumptions on state space and action space -- 20. Supplementary remarks -- Appendix 1. List of symbols and conventions -- 2. Some notions and auxiliary results from probability theory -- 3. Conditional distributions and expectations -- Literature -- Index of definitions.