Visual-tactile perception for a biomimetic robot in constrained environments

Environmental perception is a crucial foundation for enhancing the application potential of biomimetic robots. Motivated by the complementary roles of visual and tactile sensing observed in rats, this work proposes a visual-tactile perception for a small-scale bio-inspired robotic rat. The method leverages binocular vision to estimate depth images through an attention-based network and improve perception and localization accuracy by 14.22% based on a dynamic objects removal module. Besides, the whisker sensor is applied to enhance the robot’s ability to identify object contours and environmental boundaries in narrow spaces, with obstacle contour and environment boundary reconstruction goodness of fit exceeding 97.00% and 93.87%, respectively. In addition, by integrating the above individual perception methods, we achieve the fusion of vision and tactile sensing for complex environment perception. To the best of our knowledge, this is the first study to implement vision-tactile fusion perception on a miniature biomimetic robot through physical experiments. The experiments demonstrate that our method exhibits promising results on the robotic rat, reducing localization errors in narrow and dim scenes by an average of 29.14% compared to existing state-of-the-art methods.