Efficiency in Games with Markovian Private Information

We study repeated Bayesian n-player games in which the players' privately known types evolve according an irreducible Markov chain. Our main result shows that, with communication, any Pareto-efficient payoff vector above a stationary minmax value can be approximated arbitrarily closely in a perfect Bayesian equilibrium as the discount factor goes to one. As an intermediate step we construct a dynamic mechanism (without transfers) which is approximately efficient for patient players given a sufficiently long time horizon.