The theory of learning in games studies how, which and what kind of equilibria might arise as a consequence of a long-run non-equilibrium process of learning, adaptation and/or imitation. If agents’ strategies are completely observed at the end of each round, and agents are randomly matched with a series of anonymous opponents, fairly simple rules perform well in terms of the agent’s worst-case payoffs, and also guarantee that any steady state of the system must correspond to an equilibrium. If (as in extensive-form games) players do not observe the strategies chosen by their opponents, then learning is consistent with steady states that are not Nash equilibria because players can maintain incorrect beliefs about off-path play. Beliefs can also be incorrect due to cognitive limitations and systematic inferential errors.