Dynamic Scoring : Probabilistic Model Selection Based on Utility Maximization

We propose a novel approach of model selection for probability estimates that can be applied in time evolving setting. The idea is that any discrepancy between different probability estimates opens a possibility to compare them by trading on a hypothetical betting market that trades probabilities. We describe the mechanism of such a market, where agents are maximizing some utility function which determines the optimal trading volume for given odds. This procedure produces supply and demand functions, the size of the bet as a function of a trading probability. These functions are analytical for the choice of logarithmic and exponential utility functions. Having two probability estimates and the corresponding supply and demand functions, the trade matching these estimates happens at the intersection of the supply and demand functions. We show that an agent using correct probabilities will realize a profit in expectation when trading against any other set of probabilities