Existence of Pure and Behavior Strategy Stationary Markov Equilibrium in Dynamic Stochastic Games

This paper demonstrates the existence of pure strategy stationary Markov equilibrium in a class of dynamic stochastic games. Our main result implies the existence of behavior strategy stationary Markov equilibrium in any stochastic games in which the sets of actions are compact, the set of states is countable, the period payoff function to a player depends U.S.C.-L.S.C. on actions, and the transition function depends continuously on the actions. Additionally, if for each player a static best reply set is convex, then the equilibrium can be taken to be a pure strategy equilibrium. Our method of proof is simple and highlights the similarities between the static and the dynamic models. Several applications and corollaries are presented