Brown [3] constructed an aperiodic Markov decision chain in which no overtaking policy (stationary or nonstationary) exists. However, in his example a strong overtaking optimal policy exists in the class of all stationary policies. We provide another example of an aperiodic and geometric ergodic...

We extend a result by Cavazos-Cadena and Lasserre on the existence of strong 1-optimal stationary policies in Markov decision chains with countable state spaces, uniformly ergodic transition probabilities and bounded costs to a larger class of models with unbounded costs and the so-called...

Brown [3] constructed an aperiodic Markov decision chain in which no overtaking policy (stationary or nonstationary) exists. However, in his example a strong overtaking optimal policy exists in the class of all stationary policies. We provide another example of an aperiodic and geometric ergodic...

We extend a result by Cavazos-Cadena and Lasserre on the existence of strong 1-optimal stationary policies in Markov decision chains with countable state spaces, uniformly ergodic transition probabilities and bounded costs to a larger class of models with unbounded costs and the so-called...