Active Non-Line-of-Sight human pose estimation based on deep learning

Non-Line-of-Sight technology is to image objects that are hidden from the camera's view. It has a wide range of application prospects in robotic vision, national defense, remote sensing, medical imaging, and unmanned driving. Active Non-Line-of-Sight imaging mainly relies on time-resolved optical impulse responses. The Non-Line-of-Sight imaging system emits ultra-short light pulses to illuminate the diffuse reflection wall, and uses ultra-fast time-resolved single-photon detectors to collect multiple reflected photon information, thereby obtaining information in the hidden scene. Finally, various reconstruction algorithms are used to reconstruct the hidden scene. However, most of the existing reconstruction algorithms have the problems of slow reconstruction speed and fuzzy reconstruction results, especially in the aspect of human pose estimation. In this article, we describe a method of active Non-Line-of-Sight human pose estimation based on deep learning. In order to solve the problem of lack of deep learning data, we simulate large amounts of pseudo-transient images for the network, including various complex actions: walking, jumping, turning, bending back and forth, rotating, using the confocal Non-Line-of-Sight imaging model. And then we train the simulated transient images using light cones Transformation and U-net coding and decoding network structure. Finally, we examine the performance of our method on synthetic and experimental datasets. The prediction results show that our method can not only estimate the pose of real measured non-view human pose data, but also significantly improve the quality of reconstruction.