Ergodic and adaptive control of hidden Markov models

A partially observed stochastic system is described by a discrete time pair of Markov processes. The observed state process has a transition probability that is controlled and depends on a hidden Markov process that also can be controlled. The hidden Markov process is completely observed in a closed set, which in particular can be the empty set and only observed through the other process in the complement of this closed set. An ergodic control problem is solved by a vanishing discount approach. In the case when the transition operators for the observed state process and the hidden Markov process depend on a parameter and the closed set, where the hidden Markov process is completely observed, is nonempty and recurrent an adaptive control is constructed based on this family of estimates that is almost optimal. Copyright Springer-Verlag Berlin Heidelberg 2005