1. Introduction and summary -- I. Countable state space -- 2. Decision models and definition of the problem -- 3. The principle of optimality and the optimality equation -- 4. Value iteration -- 5. Criteria of optimality and existence of $$\bar{p} $$-optimal plans -- 6. Sufficient statistics,...