Reinforcement Learning Models For Power System Control
Johri, Pranjal
Reinforcement Learning Models For Power System Control by Pranjal Johri - IIT Jodhpur Department of Computer Science and Technology 2023 - vii,15p. HB
The Indian Electricity Grid is one of the largest in the world. It has a transmission system up to 765kV level and a total installed capacity of approximately 402 GW, with 50.7% share of coal-fired plants, making it 58.6% of fossil fuel-fired plants. Hydro power plants share 11.6%, and nuclear power plants share 1.7% of the total installed capacity. Non-fossil fuel-powered generation contributes 41.4% of the total installed capacity, including 28.1% from solar, wind, and other renewable energy sources [1].
The Indian grid is governed by various technical standards and guidelines issued by the Central Electricity Authority (CEA) and the Indian Electricity Grid Code (IEGC) issued by the Central Electricity Regulatory Commission (CERC). All power generators, transmission licensees, and distribution licensees must adhere to these technical standards and IEGC. According to Clause A1(c) of the "Central Electricity Authority (Technical Standards for Connectivity to the Grid) Amendment Regulations, 2013" [2], published on October 13, 2013, "The Automatic Voltage Regulator of generators of 100 MW and above shall include Power System Stabilizer (PSS)." This is applicable to generating stations other than wind and those using inverters, connected to the grid on or after the publication date of this gazette [2].
The Power System Stabilizer (PSS) plays a crucial role in improving power system stability. PSS control helps in damping generator rotor angle swings across a broad range of frequencies in the power system. These frequencies range from low-frequency intertie modes (typically 0.1 - 1.0 Hz) to local modes (typically 1 - 2 Hz) and intra-plant modes (about 2 - 3 Hz). The low-frequency modes, often called intertie or interarea modes, result from coherent groups of generators swinging against other groups in the interconnected grid system [3].
Conventional PSSs are of the "power and frequency" input type and are typically provided as software for the Automatic Voltage Regulator (AVR). In the present state-of-the-art industrial PSS, parameters are designed and studied on a 1-machine and infinite bus system model. When connected to the AVR, the PSS should act to improve the phase characteristics of the Generator, Excitation System, and Power system (GEP) to dampen power system oscillations. PSS tuning involves three steps:
PSS tuning (design stage)
PSS tuning (factory test)
PSS tuning (site test)
In the design stage, machine ratings, excitation data, and machine constants are taken into account, and PSS parameters are calculated. Then, PSS simulation is run to obtain Bode plots, frequency characteristics, step frequency responses with and without PSS. These results are evaluated based on various parameters such as waveform, damping factor, transient time, and characteristics of Bode plots. Parameters are tuned to either increase or decrease the effect of PSS. In the factory test, results are evaluated along with additional function checks and their coherence with chosen PSS parameters. During the site test, PSS performance is verified with actual machine parameters. These are general practices followed in the industry for PSS tuning.
However, this is an open-loop approach and a one-time tuning for large turbo generators. In case of system abnormalities, unwanted unit tripping, or major changes in configuration in connected substations, only PSS parameters are altered. However, few manufacturers provide adaptive PSS, limited to selecting predefined parameter values for two to three network configuration changes. Conventional PSSs face challenges in conditions such as multi low-frequency oscillation modes (especially inter-area oscillations) [4], multiple operating conditions of modern power systems, parametric uncertainties, and changes in voltage profiles due to the penetration of renewable energy generation, especially solar PV and wind turbines.
To overcome the aforementioned limitations, various Reinforcement Learning (RL) based algorithms have been discussed and developed by academia to tune PSS parameters based on prevailing grid conditions and network parameters. A literature survey has been conducted to study the current research status of RL-based methods for Power System stability. My research proposal aims to study state-of-the-art RL-based algorithms for this problem, understand the shortcomings, improve existing methods, and develop a new RL-based method to explore the possibility of replacing PSS with fully RL-based software to support AVR of large generators to adapt to dynamic grid conditions. RL model implementation in Python has been performed, and its performance has been studied in terms of rewards.
As per provisions for the renewable energy sector in Union Budget 2021-22, the Ministry of New and Renewable Energy is working towards achieving the target set by the Government of India to install 1,75,000 MW (excluding large hydro) of renewable energy capacity by 2022 and further increase it to 4,50,000 MW by 2030 [5]. Thus, there is a need to work towards grid stability
Department of Computer Science and Technology
PSS
Power System Stabilizer
MTech Theses
006.3 / J739R
Reinforcement Learning Models For Power System Control by Pranjal Johri - IIT Jodhpur Department of Computer Science and Technology 2023 - vii,15p. HB
The Indian Electricity Grid is one of the largest in the world. It has a transmission system up to 765kV level and a total installed capacity of approximately 402 GW, with 50.7% share of coal-fired plants, making it 58.6% of fossil fuel-fired plants. Hydro power plants share 11.6%, and nuclear power plants share 1.7% of the total installed capacity. Non-fossil fuel-powered generation contributes 41.4% of the total installed capacity, including 28.1% from solar, wind, and other renewable energy sources [1].
The Indian grid is governed by various technical standards and guidelines issued by the Central Electricity Authority (CEA) and the Indian Electricity Grid Code (IEGC) issued by the Central Electricity Regulatory Commission (CERC). All power generators, transmission licensees, and distribution licensees must adhere to these technical standards and IEGC. According to Clause A1(c) of the "Central Electricity Authority (Technical Standards for Connectivity to the Grid) Amendment Regulations, 2013" [2], published on October 13, 2013, "The Automatic Voltage Regulator of generators of 100 MW and above shall include Power System Stabilizer (PSS)." This is applicable to generating stations other than wind and those using inverters, connected to the grid on or after the publication date of this gazette [2].
The Power System Stabilizer (PSS) plays a crucial role in improving power system stability. PSS control helps in damping generator rotor angle swings across a broad range of frequencies in the power system. These frequencies range from low-frequency intertie modes (typically 0.1 - 1.0 Hz) to local modes (typically 1 - 2 Hz) and intra-plant modes (about 2 - 3 Hz). The low-frequency modes, often called intertie or interarea modes, result from coherent groups of generators swinging against other groups in the interconnected grid system [3].
Conventional PSSs are of the "power and frequency" input type and are typically provided as software for the Automatic Voltage Regulator (AVR). In the present state-of-the-art industrial PSS, parameters are designed and studied on a 1-machine and infinite bus system model. When connected to the AVR, the PSS should act to improve the phase characteristics of the Generator, Excitation System, and Power system (GEP) to dampen power system oscillations. PSS tuning involves three steps:
PSS tuning (design stage)
PSS tuning (factory test)
PSS tuning (site test)
In the design stage, machine ratings, excitation data, and machine constants are taken into account, and PSS parameters are calculated. Then, PSS simulation is run to obtain Bode plots, frequency characteristics, step frequency responses with and without PSS. These results are evaluated based on various parameters such as waveform, damping factor, transient time, and characteristics of Bode plots. Parameters are tuned to either increase or decrease the effect of PSS. In the factory test, results are evaluated along with additional function checks and their coherence with chosen PSS parameters. During the site test, PSS performance is verified with actual machine parameters. These are general practices followed in the industry for PSS tuning.
However, this is an open-loop approach and a one-time tuning for large turbo generators. In case of system abnormalities, unwanted unit tripping, or major changes in configuration in connected substations, only PSS parameters are altered. However, few manufacturers provide adaptive PSS, limited to selecting predefined parameter values for two to three network configuration changes. Conventional PSSs face challenges in conditions such as multi low-frequency oscillation modes (especially inter-area oscillations) [4], multiple operating conditions of modern power systems, parametric uncertainties, and changes in voltage profiles due to the penetration of renewable energy generation, especially solar PV and wind turbines.
To overcome the aforementioned limitations, various Reinforcement Learning (RL) based algorithms have been discussed and developed by academia to tune PSS parameters based on prevailing grid conditions and network parameters. A literature survey has been conducted to study the current research status of RL-based methods for Power System stability. My research proposal aims to study state-of-the-art RL-based algorithms for this problem, understand the shortcomings, improve existing methods, and develop a new RL-based method to explore the possibility of replacing PSS with fully RL-based software to support AVR of large generators to adapt to dynamic grid conditions. RL model implementation in Python has been performed, and its performance has been studied in terms of rewards.
As per provisions for the renewable energy sector in Union Budget 2021-22, the Ministry of New and Renewable Energy is working towards achieving the target set by the Government of India to install 1,75,000 MW (excluding large hydro) of renewable energy capacity by 2022 and further increase it to 4,50,000 MW by 2030 [5]. Thus, there is a need to work towards grid stability
Department of Computer Science and Technology
PSS
Power System Stabilizer
MTech Theses
006.3 / J739R