Adaptive control of stochastic systems with unknown disturbance distribution: discounted criteria

We consider a class of discrete-time stochastic control systems, with Borel state and action spaces, and possibly unbounded costs. The processes evolve according to the equation x t +1 =F(x t , a t , ξ t ), t=0, 1, ..., where the ξ t are i.i.d. random vectors whose common distribution is unknown. Assuming observability of {ξ t }, we use the empirical estimator of its distribution to construct adaptive policies which are asymptotically discounted cost optimal . Copyright Springer-Verlag Berlin Heidelberg 2006