Adaptive policies for time-varying stochastic systems under discounted criterion

We consider a class of time-varying stochastic control systems, with Borel state and action spaces, and possibly unbounded costs. The processes evolve according to a discrete-time equation x n + 1 =G n (x n , a n , ξ n), n=0, 1, … , where the ξ n are i.i.d. ℜ k-valued random vectors whose common density is unknown, and the G n are given functions converging, in a restricted way, to some function G ∞ as n→∞. Assuming observability of ξ n, we construct an adaptive policy which is asymptotically discounted cost optimal for the limiting control system x n+1 =G ∞ (x n , a n , ξ n). Copyright Springer-Verlag Berlin Heidelberg 2001