This paper deals with a Markovian decision process with an absorbing set J <Subscript>0</Subscript>. We are interested in the largest number β<Superscript>*</Superscript>≥1, called the critical discount factor, such that for all discount factors β smaller than β<Superscript>*</Superscript> the limit V of the N-stage value function V <Subscript>N</Subscript> for N →∞ exists and is...</subscript></superscript></superscript></subscript>