| Title |
Reinforcement Learning-based Energy Storage System Control for Optimal Virtual Power Plant Operation |
| Authors |
권경빈(Kyung-bin Kwon) ; 박종영(Jong-young Park) ; 정호성(Hosung Jung) ; 홍수민(Sumin Hong) ; 허재행(Jae-Haeng Heo) |
| DOI |
https://doi.org/10.5370/KIEE.2023.72.11.1586 |
| Keywords |
Deep Q-Network; Markov Decision Process; Energy Storage System; Reinforcement Learning; Virtual Power Plant |
| Abstract |
In this paper, we design a framework of the energy storage system (ESS) controller in virtual power plant (VPP) that maximize the profit. We consider the VPP that includes photovoltaics, wind turbines and demand along with ESSs and describe the environment based on Markov decision process (MDP). To find the best policy for ESS charging and discharging control, we implement a deep Q-network (DQN) method that trains a neural network which estimates Q-function values for each possible discrete actions. In the numerical test utilizing real-world data of Namgwangju Station, ERCOT and US government, we train the DQN and demonstrate that the proposed algorithm converges. Through the test with the trained policy, we showcase that the policy functions effectively in the scenario with uncertainty from renewable generations and load, as it responds adaptively to electricity prices. |