IoT Microservice Deployment in Edge-Cloud Hybrid Environment Using Reinforcement Learning

The edge-cloud hybrid environment requires complex deployment strategies to enable the smart Internet-of-Things (IoT) system. However, current service deployment strategies use simple, generalized heuristics and ignore the heterogeneous characteristics in the edge-cloud hybrid environment. In this article, we devise a method to find a microservice-based service deployment strategy that can reduce the average waiting time of IoT devices in the hybrid environment. For this purpose, we first propose a microservice-based deployment problem (MSDP) based on the heterogeneous and dynamic characteristics in the edge-cloud hybrid environment, including heterogeneity of edge server capacities, dynamic geographical information of IoT devices, and changing device preference for applications and complex application structures. We then propose a multiple buffer deep deterministic policy gradient (MBDDPG) to provide more preferable service deployment solutions. Our algorithm leverages reinforcement learning and neural network to learn a deployment strategy without any human instruction. Therefore, the service provider can make full use of limited resources to improve the Quality of Service (QoS). Finally, we implement MBDDPG based on real-world data sets and some synthetic data, and we also implement another two algorithms, genetic algorithm and random algorithm, as a contrast. The experimental results demonstrate that MBDDPG is able to learn a preferable strategy which, in terms of average waiting time, outperforms genetic algorithm and the random algorithm by 32% and 44%, respectively.