Low Latency Cyberattack Detection in Smart Grids with Deep Reinforcement Learning

This paper focuses on low latency detection of cyberattacks in smart grids with deep reinforcement learning (DRL). The objective of low latency detection is to minimize detection delays while maintaining good detection accuracy, and this is different from conventional detection methods that focus mainly on detection accuracy. A lower detection delay is critical for power grid operations as it ensures timely recovery to minimize losses due to cyberattacks. Since detection delay is the main design metric, the algorithm is developed by using a dynamic AC system model that captures the power grid state transitions in real time, while most other works in the literature use a simplified DC model. The DRL detection algorithm is developed by using a continuous state space deep Q-network (DQN) on the framework of a Markov decision process (MDP). The new DQN design has two main innovations. First, the MDP state is designed as a sliding window of Rao-statistics of the residues of AC dynamic state estimations. The proposed state formulation can accurately capture dynamic power state transitions in real time. Second, a new reward function is proposed to enable flexible trade-off between detection delays and detection accuracy. Simulation results show that the proposed DQN-based DRL detection algorithm can achieve very low detection delays with a high detection accuracy