This paper investigates finite horizon semi-Markov decision processes with denumerable states. The optimality is over the class of all randomized history-dependent policies which include states and also planning horizons, and the cost rate function is assumed to be bounded below. Under suitable...

In this paper we consider the convergence of a sequence {Mn} of the models of discounted continuous-time constrained Markov decision processes (MDP) to the “limit” one, denoted by M∞. For the models with denumerable states and unbounded transition rates, under reasonably mild conditions we...

In this paper, we consider a mean–variance optimization problem for Markov decision processes (MDPs) over the set of (deterministic stationary) policies. Different from the usual formulation in MDPs, we aim to obtain the mean–variance optimal policy that minimizes the variance over a set of...