Naive Reinforcement Learning With Endogenous Aspiration

This risk.paper considers a simple learning process for decision problems under All behaviour change derives from the reinforcing or deterring effect of instantaneous payoff experiences. Payoff experiences are reinforcing or deterring depending on whether the payoff exceeds an aspiration level or falls short of it. The aspiration level is endogenous. Over time it is adjusted into the direction of the actually experienced payoff. This paper shows that realistic aspiration level adjustments may improve the decision maker's long run per-formance, because they may prevent him from feeling dissatisfied with even the best of the available strategies. On the other hand, the paper also shows that in a large class of decision problems endogenous aspiration levels lead to persistent deviations from expected payoff maximisation because they create "probability matching" effects.