Optimal Management of Renewable Energy Certificates : A Reinforcement Learning Approach

Renewable Energy Certificate (REC) market plays a critical role in renewable energy compliance by facilitating the matching of REC supply and demand. Unfortunately, small-scale generators experience great challenge in REC management due to market restrictions, highly volatile market environment, and lack of expertise in risk management. Such challenges have led to the emergence of REC brokerage service providers who participate in the market by aggregating RECs from multiple generators. In this paper, we formulate the REC management problems of a single generator as well as the aggregator using a discrete time Markov decision process (MDP). We characterize useful structural properties of the optimal solution, and develop a solution framework based on a deep reinforcement learning (DRL) algorithm incorporating graph neural networks (GNN) and the structural properties. A numerical experiment based on two REC markets in the U.S. illustrates that our proposed algorithm obtains an REC management strategy that performs better than several baseline strategies used in practice. Furthermore, we identify the role and the value of the REC aggregation service providers to the small-scale generators