High-Precision Human Pose Estimation Algorithm Based on Multi-View LiDAR and Visible Light Sensors

To address the limitations in multi-person 3D pose estimation algorithms that either lack sufficient three-dimensional information when using visible light sensors or suffer from low resolution with LiDAR sensors, we have developed a system integrating multiple visible light and 3D LiDAR composite sensors. This setup facilitates the creation of a richly detailed, mutually calibrated, and synchronized human pose dataset. We propose an advanced top-down multi-person 3D pose estimation algorithm utilizing this integrated sensor system. By leveraging multi-view fused point clouds and multi-angle visible light data, our approach encompasses modules for human localization, multimodal data fusion, and joint keypoint positioning, achieving enhanced training and inference speeds alongside improved recognition accuracy. Furthermore, our network has been successfully transplanted and accelerated on NVIDIA's Jetson processors as well as Huawei's domestically produced Atlas 200 processor.