This paper studies the identification problem in infinite horizon Markovian games and proposes a generally applicable estimation method. Every period firms simultaneously select an action from a finite set. We characterize the set of Markov equilibria. Period profits are a linear function of equilibrium choice probabilities. The question of identification of these values is then reduced to the existence of a solution to this linear equation system. We characterize the identification conditions. We propose a simple estimation procedure which follows the steps in the identification argument. The estimator is consistent, asymptotic normally distributed, and efficient. We have collected quarterly time series data on pubs, restaurants, coffeehouses, bakeries and carpenters for two Austrian towns between 1982 and 2002. A dynamic entry game is estimated in which firms simultaneously decide whether to enter, remain active, or exit the industry. The period profit estimates are used to simulate the equilibrium behavior under a policy experiment in which a unit tax is imposed on firms deciding to enter the industry.
