Real Options Models Without Threshold Behavior : A Reinforcement Learning Approach

Real options models consider an investment project where the economic environment is uncertain. The uncertainty is integrated through stochastic processes to model the evolution of underlying variables that determine the value of an investment opportunity. Establishing the uniqueness of the optimal investment threshold is critical for obtaining an analytical solution to a firm’s investment problem. It is known that fulfilling two conditions is sufficient for one threshold to exist. Conditions are that, first, the value function of waiting for one period is monotonic in the underlying state variable, and, second, there is a positive persistence of uncertainty in the sense that the cumulative probability distribution of future values shifts uniformly to the right when the current value increases. If the conditions are met, the uniqueness of the optimal investment threshold is guaranteed, and the investment behavior exhibits a threshold effect, where investing is optimal on one side and waiting with investment on the other. In this paper, we show that investment problems do not necessarily satisfy the assumption of the positive persistence of uncertainty. Therefore, the threshold need not be unique. We model the investment problem of a single firm by a finite discrete Markov decision process, which allows modeling situations where positive persistence of uncertainty does not hold. To obtain optimal strategies, we use reinforcement learning, a solution method that does not require restrictive assumptions on the underlying probability distribution