Markovian Dynamics in Asynchronous Stochastic Models

We study Asynchronous Stochastic Dynamic Models in which a single agent chooses in each period and examine the existence of Markov-perfect equilibrium in these models. We show that if the models are stochastic then a pure strategy Markov-perfect equilibrium exists for a finite horizon game. For the infinite horizon case, a pure strategy perfect equilibrium exists in semi-Markov strategies and a mixed strategy equilibrium in Markov strategies. The equilibrium are stationary if the model is sufficiently stationary. The results are illustrated with applications to dynamic oligopoly models, dynamic principal-agent models and dynamic models of production