Depth map stitching using compound eye cameras for wide FOV imaging

Depth information is an important tool in the computer vision field for object detection, recognition, and 3D modeling. Conventional depth sensors have a limited field of view(FOV) and provide limited depth information. Depth map stitching has been proposed to construct a large FOV image by combining depth maps from different perspectives. Due to the sparsity of features and susceptibility to lighting conditions in the depth images, stitching faces a challenge as it often relies on feature matching for obtaining mapping matrices. To overcome this issue, this paper proposes an algorithm that utilizes color image features to guide depth map stitching. Initially, the LightGlue algorithm based on SuperPoint features is employed for precise feature matching to obtain accurate homography matrices. Depth maps are then generated using stereo vision and aligned with color images, effectively transforming the depth map stitching challenge into a color image stitching task. Additionally, we designed a bionic compound eye imaging system that captures images in real time using multiple cameras. This system consists of eight cameras arranged in pairs at specific angles with a circular configuration, achieving an expanded FOV. Experimental results demonstrate that the system effectively captures depth maps with an FOV of 110°×90° and can display real-time video with 8 frames at 640 ×480 resolution. The maximum relative depth error is 5.7% @ 0.55m. In the future, this technology is expected to be applied in panoramic 3D reconstruction, underwater mapping, and autonomous driving.