Approximate Dynamic Programming for Online Operation of Connected Hydro-Power Reservoirs

We study the operational problem of connected hydro-power reservoirs which is traditionally formulated as a stochastic dynamic program accounting for uncertainty in electricity prices and inflows. This formulation suffers from the curse of dimensionality, as the state space explodes with the number of reservoirs and the history of prices and inflows. To tackle the dimensionality issue, we propose an approximate dynamic programming (ADP) approach that relies on sampling and a linear approximation. When the time series of prices and inflows follow autoregressive processes, our approximation provides an upper bound on the future value function. We use an offline training algorithm to learn the parameters of the linear approximation and demonstrate how this facilitates computationally efficient online optimization by linear programming. We run both in-sample and out-of-sample simulations for realistic test systems of a cascade and a network of reservoirs. In particular, we assess the performance of the scheduling policy and the profit estimate obtained by online optimization. Our results show that the expected value of using the ADP policy exceeds that of using the expected value. Additionally, we compare the performance of the proposed ADP method to the stochastic dual dynamic programming approach. Our results demonstrate that as the complexity of the connected reservoir system increases, the superiority of the proposed ADP in terms of computation time becomes more significant