. In this paper, we consider the nonstationary Markov decision processes (MDP, for short) with average variance criterion on a countable state space, finite action spaces and bounded one-step rewards. From the optimality equations which are provided in this paper, we translate the average...

This paper deals with denumerable discrete-time Markov decision processes with unbounded costs. The criteria to be minimized are both of the limsup and liminf average criteria, instead of only the limsup average criterion widely used in the previous literature. We give another set of conditions...

In this paper, we study constrained continuous-time Markov decision processes with a denumerable state space and unbounded reward/cost and transition rates. The criterion to be maximized is the expected average reward, and a constraint is imposed on an expected average cost. We give suitable...

This paper deals with a new optimality criterion consisting of the usual three average criteria and the canonical triplet (totally so-called strong average-canonical optimality criterion) and introduces the concept of a strong average-canonical policy for nonstationary Markov decision processes,...

This paper is devoted to studying continuous-time Markov decision processes with general state and action spaces, under the long-run expected average reward criterion. The transition rates of the underlying continuous-time Markov processes are allowed to be unbounded, and the reward rates may...

In this paper weighted singularly perturbed hybrid stochastic systems are discussed. Under some reasonable assumptions, it is shown that there exists a uniformly δ-optimal policy when the perturbation is sufficiently small. Copyright Springer-Verlag Berlin Heidelberg 2005

In this paper we consider the weighted reward Markov decision process, with perturbation. The “weighted reward” refers to appropriately normalized convex combination of the discounted and the long-run average reward criteria. This criterion allows the controller to trade-off short-term costs...

In this paper discounted and average Markov decision processes with finite state space and countable action set (semi-infinite MDP for short) are discussed. Without ordinary continuity and compactness conditions, for discounted semi-infinite MDP we have shown that by exploiting the results on...