Energy Management Strategy Via Maximum Entropy Reinforcement Learning for an Extended Range Logistics Vehicle

Energy management strategy plays a vital role in the energy efficiency of the extended range electric vehicle. The reinforcement learning-based control strategy can be adopted as online control without being subjected to the pre-known environment information . In this study, a novel auxiliary power unit (APU) charging strategy with multi-object optimization is proposed to achieve high fuel conversion efficiency while maintaining battery charging health. The state-of-the-art algorithm, Soft Actor-Critic (SAC), is applied to achieve better exploration of the possible APU behaviour and solve the existed sensitivity and poor convergence problems from the existed RL studies. Its performance is further verified by the results of the Deep Deterministic Policy Gradient (DDPG) algorithm and DP. Three innovative targets are selected as the RL rewards for optimization: the engine fuel rate, SOC charging trajectory, and the battery charging rate (C-rate). The comparative results show that the SAC had a 36% faster convergence speed than DDPG while providing a smoother and more stable action space. The fuel consumption with SAC also outplays that of DDPG by around 3%, which achieves almost 95% of the global optimization result. It indicates the standout ability of SAC in dealing with wide-range actions and states with high randomness