Joint Deep Reinforcement Learning and Stackelberg Game With Intervention for Heterogeneous Edge Computing in Industrial Internet of Things

Edge computing, as a novel computing paradigm, is expected to significantly enhance the productivity of the Industrial Internet of Things (IIoT). However, in a practical IIoT environment consisting of multiple edge servers, the supply of computing services (edge servers) may involve different business strategies or pricing schemes, while the demand side (user terminals) can choose the service that best suits their needs based on quotes from different suppliers. The competition between the supply and demand sides for their respective interests leads to strategic games between service suppliers and users. Therefore, how to design reasonable pricing mechanisms and offloading strategies to balance service supply and resource demand has become an urgent issue. Furthermore, during these games, self-interested service providers may collude to form mutually beneficial cartels. To address the intricacies of these games, this paper integrates deep reinforcement learning (DRL) with game theory. Specifically, a heterogeneous system model is first constructed in the Stackelberg game framework. Subsequently, we leverage the Double-Dueling Deep Q-Network (D3QN) and Deep Deterministic Policy Gradient (DDPG) algorithms to formulate users’offloading decisions and service providers’ pricing strategies, respectively. Consequently, a joint D3QN-DDPG and Stackelberg algorithm is proposed to maximize the benefits for both users and servers. Moreover, to disrupt the formation of cartels among service providers, we introduce an intervention mechanism and present an intervention-based benefit maximization algorithm. Finally, simulation experiments demonstrate that the total system utility with intervention surpasses that without intervention by approximately 16%, validating the necessity of the intervention. Comparisons with other benchmark algorithms confirm that the proposed algorithm achieves superior system benefits.