1. Introduction and summary -- I. Countable state space -- 2. Decision models and definition of the problem -- 3. The principle of optimality and the optimality equation -- 4. Value iteration -- 5. Criteria of optimality and existence of $$\bar{p} $$-optimal plans -- 6. Sufficient statistics,...

We consider several sequential search problems for an object which is hidden in a discrete interval with an arbitrary prior distribution. The searcher decomposes the momentary search interval into a fixed number of subintervals and obtains the information in which of the intervals the object is...

We consider several sequential search problems for an object which is hidden in a discrete interval with an arbitrary prior distribution. The searcher decomposes the momentary search interval into a fixed number of subintervals and obtains the information in which of the intervals the object is...

This paper deals with a Markovian decision process with an absorbing set J <Subscript>0</Subscript>. We are interested in the largest number β<Superscript>*</Superscript>≥1, called the critical discount factor, such that for all discount factors β smaller than β<Superscript>*</Superscript> the limit V of the N-stage value function V <Subscript>N</Subscript> for N →∞ exists and is...</subscript></superscript></superscript></subscript>

We present tools and guidelines for investigating Lipschitz continuity of the value functions in MDP’s, using the Hausdorff metric and the Kantorovich metric for measuring the influence of the constraint set and the transition law, respectively. The methods are explained by examples....

This paper deals with a Markovian decision process with an absorbing set J 0 . We are interested in the largest number β *≥1, called the critical discount factor, such that for all discount factors β smaller than β * the limit V of the N-stage value function V N for N →∞ exists and is...

We present tools and guidelines for investigating Lipschitz continuity of the value functions in MDP’s, using the Hausdorff metric and the Kantorovich metric for measuring the influence of the constraint set and the transition law, respectively. The methods are explained by examples....