TY - JOUR
T1 - Joint Deep Reinforcement Learning and Stackelberg Game With Intervention for Heterogeneous Edge Computing in Industrial Internet of Things
AU - Cheng, Weijun
AU - Wei, Wenjing
AU - Liu, Xiaoshi
AU - Huan, Xintao
N1 - Publisher Copyright:
© 2014 IEEE.
PY - 2025
Y1 - 2025
N2 - Edge computing, as a novel computing paradigm, is expected to significantly enhance the productivity of the Industrial Internet of Things (IIoT). However, in a practical IIoT environment consisting of multiple edge servers, the supply of computing services (edge servers) may involve different business strategies or pricing schemes, while the demand side (user terminals) can choose the service that best suits their needs based on quotes from different suppliers. The competition between the supply and demand sides for their respective interests leads to strategic games between service suppliers and users. Therefore, how to design reasonable pricing mechanisms and offloading strategies to balance service supply and resource demand has become an urgent issue. Furthermore, during these games, self-interested service providers may collude to form mutually beneficial cartels. To address the intricacies of these games, this paper integrates deep reinforcement learning (DRL) with game theory. Specifically, a heterogeneous system model is first constructed in the Stackelberg game framework. Subsequently, we leverage the Double-Dueling Deep Q-Network (D3QN) and Deep Deterministic Policy Gradient (DDPG) algorithms to formulate users’offloading decisions and service providers’ pricing strategies, respectively. Consequently, a joint D3QN-DDPG and Stackelberg algorithm is proposed to maximize the benefits for both users and servers. Moreover, to disrupt the formation of cartels among service providers, we introduce an intervention mechanism and present an intervention-based benefit maximization algorithm. Finally, simulation experiments demonstrate that the total system utility with intervention surpasses that without intervention by approximately 16%, validating the necessity of the intervention. Comparisons with other benchmark algorithms confirm that the proposed algorithm achieves superior system benefits.
AB - Edge computing, as a novel computing paradigm, is expected to significantly enhance the productivity of the Industrial Internet of Things (IIoT). However, in a practical IIoT environment consisting of multiple edge servers, the supply of computing services (edge servers) may involve different business strategies or pricing schemes, while the demand side (user terminals) can choose the service that best suits their needs based on quotes from different suppliers. The competition between the supply and demand sides for their respective interests leads to strategic games between service suppliers and users. Therefore, how to design reasonable pricing mechanisms and offloading strategies to balance service supply and resource demand has become an urgent issue. Furthermore, during these games, self-interested service providers may collude to form mutually beneficial cartels. To address the intricacies of these games, this paper integrates deep reinforcement learning (DRL) with game theory. Specifically, a heterogeneous system model is first constructed in the Stackelberg game framework. Subsequently, we leverage the Double-Dueling Deep Q-Network (D3QN) and Deep Deterministic Policy Gradient (DDPG) algorithms to formulate users’offloading decisions and service providers’ pricing strategies, respectively. Consequently, a joint D3QN-DDPG and Stackelberg algorithm is proposed to maximize the benefits for both users and servers. Moreover, to disrupt the formation of cartels among service providers, we introduce an intervention mechanism and present an intervention-based benefit maximization algorithm. Finally, simulation experiments demonstrate that the total system utility with intervention surpasses that without intervention by approximately 16%, validating the necessity of the intervention. Comparisons with other benchmark algorithms confirm that the proposed algorithm achieves superior system benefits.
KW - Deep Reinforcement Learning
KW - Edge Computing
KW - Game Theory
KW - Industrial Internet of Things
UR - http://www.scopus.com/inward/record.url?scp=105002382205&partnerID=8YFLogxK
U2 - 10.1109/JIOT.2025.3559080
DO - 10.1109/JIOT.2025.3559080
M3 - Article
AN - SCOPUS:105002382205
SN - 2327-4662
JO - IEEE Internet of Things Journal
JF - IEEE Internet of Things Journal
ER -