A Framework for Modeling Bounded Rationality: Mis-specified Bayesian-Markov Decision Processes

We provide a framework to study dynamic optimization problems where the agent is uncertain about her environment but has (possibly) an incorrectly specified model, in the sense that the support of her prior does not include the true model. The agent's actions affect both her payoff and also what she observes about the environment; she then uses these observations to update her prior according to Bayes' rule. We show that if optimal behavior stabilizes in this environment, then it is characterized by what we call an equilibrium. An equilibrium strategy $\sigma$ is a mapping from payoff relevant states to actions such that: (i) given the strategy $\sigma$, the agent's model that is closest (according to the Kullback-Leibler divergence) to the true model is $\theta(\sigma)$, and (ii) $\sigma$ is a solution to the dynamic optimization problem where the agent is certain that the correct model is $\theta(\sigma)$. The framework is applicable to several aspects of bounded rationality, where the reason why a decision maker has incorrect beliefs can be traced to her use of an incorrectly-specified model.