Real-Time Pose Estimation of Rats Based on Stereo Vision Embedded in a Robotic Rat

In this paper, we propose a system for real-time rat pose estimation based on stereo vision. The system is dedicated to robot-rat interaction research. First, we design a lightweight, high-resolution network (RRKDNet) for keypoint detection of the rat. The network is trained on a dataset of rat images, which are captured by the robotic rat in first-person view. Second, based on the keypoint detection results, the pose of the rat is obtained by stereo vision model calculation and robot coordinate transformation. At last, we complete a real-time simulation experiment to reproduce the pose of the rat and the robotic rat. The system has been subjected to a series of experiments and the results demonstrate that our network performs better in speed and performance than similar networks. Compared to similar networks, our network has about one-third the number of parameters, while the detection rate increases by 45.25% (the detection rate is 71.57%). The inference speed (34.42 FPS with dual model simultaneous inference) is also faster. The validation error is only 13.85 pixels on the homemade dataset, which is lower than all backbones in Deeplabcut (a toolbox more frequently used for rat keypoint detection). Thus, this work is a significant step in the autonomous intelligent interaction between robots and rats.