RGB-D camera-based tracking system for an amphibious spherical robot

To execute vision-based tasks of the amphibious spherical robot, a visual tracking system was designed and constructed. A RGB-D camera was calibrated in the amphibious environments and was then used to perceive the surroundings of the robot. A RGB-D tracker, which was capable of handling occlusions and scale changes of the target, was built upon the KCF tracker to locate the target object. HoG and CN features in the color images were extracted to describe the target object. The scale and position models of the object was established using the Gaussian model and the depth histogram of the object. The occlusion event was recognized by segmenting the depth image and using an empirical formula. The online update process of the KCF tracker was temporarily stopped once an occlusion event was detected. Experimental results with various image sequences in amphibious environments demonstrated the effectiveness and robustness of the proposed tracking algorithm, which can meet the application requirements of the amphibious spherical robots.