基于多尺度时序采样的多任务感知网络

Integrating temporal multi-scale bird’s eye view (BEV) features, a novel multi-task perception network was proposed to solve the problems of insufficient temporal feature fusion and the difficulty in reliably perceiving occluded or distant targets. Firstly, modeling the depth prediction probability, a module with occlusion adaptability was established to estimate visible depth, map the image features into BEV features and carry out supervision based on the depth maps. Afterwards, in order to improve the effectiveness of long-distance obstacle detection, a temporal BEV sampling module was designed based on deformable attention mechanism to make multi-scale BEV feature weighted fusion in time sequence. Finally, expanding data augmentation strategies to multi tasks, 3D object detection and lane line segmentation were achieved according to corresponding task heads separately. The results from nuScenes dataset and real-vehicle experiment show that this solution can improve the accuracy in detecting occluded areas and distant targets, and the inference speed can meet the requirements of real-world applications.