Research on target recognition of multispectral streak tube imaging lidar system using multimodal convolutional neural network

A new multispectral streak tube imaging lidar is proposed to address the shortcomings of the traditional streak tube imaging lidar which can only provide single wavelength reflection information of the target. The accuracy of the depth map is improved by fusing the reconstructed depth images. To implement the target classification task, a dataset based on depth images and intensity images was first built, which consists of 240 targets with a total of 20 classes. Then, a multimodal neural network model was designed to classify the targets based on the characteristics of the dataset. The target classification ability of three methods which are depth images, depth images + intensity images and depth images + 3 intensity images are compared. The experimental results show that the proposed method can effectively improve the target recognition accuracy, which is increased from 85.19% to 90.47%.