We present tools and guidelines for investigating Lipschitz continuity of the value functions in MDP’s, using the Hausdorff metric and the Kantorovich metric for measuring the influence of the constraint set and the transition law, respectively. The methods are explained by examples....

This paper deals with a Markovian decision process with an absorbing set J <Subscript>0</Subscript>. We are interested in the largest number β<Superscript>*</Superscript>≥1, called the critical discount factor, such that for all discount factors β smaller than β<Superscript>*</Superscript> the limit V of the N-stage value function V <Subscript>N</Subscript> for N →∞ exists and is...</subscript></superscript></superscript></subscript>