TY - JOUR
T1 - Deep-Shield
T2 - Multi-Phase Mitigation of APT via Hierarchical Deep Reinforcement Learning
AU - Cao, Yuan
AU - Lin, Yeming
AU - Han, Dong Yu
AU - Xia, Yuanqing
N1 - Publisher Copyright:
© 2014 IEEE.
PY - 2025
Y1 - 2025
N2 - With rapid development of industrial internet, industrial cyber-physical systems (ICPSs) have been widely deployed to perform and supervise industrial applications. However, ICPSs still face significant cybersecurity challenges. Traditional defense mechanisms are mostly static and passive, which may fail to provide real-time protection. To solve the aforementioned problem, moving target defense (MTD) technique has been proposed as a proactive solution. However, due to the increasing sophistication and persistence of cyberattacks, it is difficult for single-phase MTD approaches to provide effective defense by only mitigating individual phases of the attack process. Therefore, we present Deep-Shield, a novel multi-phase MTD approach based on hierarchical deep reinforcement learning (HDRL) to improve the defense performance of single-phase MTD approaches when facing advanced persistent threat (APT) in ICPSs. We consider three representative MTD countermeasures, i.e., IP address shuffling, software diversity and components redundancy, which are able to mitigate attacks at different phases of cyber kill chain and protect assets contained in critical infrastructures. Firstly, we formulate the dynamic implementation of multi-phase MTD countermeasures as a semi-Markov decision process. Secondly, we detect current attack patterns by a neural network called PerNet, which is derived from Anomaly Transformer. Then, we design a HDRL-based multi-phase MTD algorithm for defense decision-making. Finally, through extensive experiments on a platform of software defined networks, we show that our proposed approach can achieve better defense performance compared with state-of-the-art solutions when dealing with APT.
AB - With rapid development of industrial internet, industrial cyber-physical systems (ICPSs) have been widely deployed to perform and supervise industrial applications. However, ICPSs still face significant cybersecurity challenges. Traditional defense mechanisms are mostly static and passive, which may fail to provide real-time protection. To solve the aforementioned problem, moving target defense (MTD) technique has been proposed as a proactive solution. However, due to the increasing sophistication and persistence of cyberattacks, it is difficult for single-phase MTD approaches to provide effective defense by only mitigating individual phases of the attack process. Therefore, we present Deep-Shield, a novel multi-phase MTD approach based on hierarchical deep reinforcement learning (HDRL) to improve the defense performance of single-phase MTD approaches when facing advanced persistent threat (APT) in ICPSs. We consider three representative MTD countermeasures, i.e., IP address shuffling, software diversity and components redundancy, which are able to mitigate attacks at different phases of cyber kill chain and protect assets contained in critical infrastructures. Firstly, we formulate the dynamic implementation of multi-phase MTD countermeasures as a semi-Markov decision process. Secondly, we detect current attack patterns by a neural network called PerNet, which is derived from Anomaly Transformer. Then, we design a HDRL-based multi-phase MTD algorithm for defense decision-making. Finally, through extensive experiments on a platform of software defined networks, we show that our proposed approach can achieve better defense performance compared with state-of-the-art solutions when dealing with APT.
KW - cybersecurity
KW - hierarchical deep reinforcement learning
KW - Industrial cyber-physical systems
KW - moving target defense
KW - software-defined networking
UR - http://www.scopus.com/inward/record.url?scp=105006920109&partnerID=8YFLogxK
U2 - 10.1109/JIOT.2025.3573367
DO - 10.1109/JIOT.2025.3573367
M3 - Article
AN - SCOPUS:105006920109
SN - 2327-4662
JO - IEEE Internet of Things Journal
JF - IEEE Internet of Things Journal
ER -