Real-time object detection and manipulation using biomimetic musculoskeletal soft robotic grasper addressing robotic fan-handling challenge

This chapter presents a real-time object detection and manipulation strategy for fan robotic challenge using a biomimetic robotic gripper and UR5 (Universal Robots, Denmark) robotic arm. Real-time object detection is developed based on computer vision method and Kinect v2 sensor. Largest contour area algorithm and depth segmentation techniques have been introduced to identify the Spanish fan from the surrounding same color objects in the background. The hybrid robotic gripper consists of five modular fingers, which are equipped with flexible soft layer enhancements for better performance in grasping. With the rigid-soft configuration, the hybrid robotic gripper is able to grasp the fan without any support from the environment. Cost-effective techniques have been used for the overall setup. Hand-eye calibration has been performed based on experimental analysis to find the relationship between the collected dataset of Kinect in pixels and the robot coordinate system. The experiments were limited to a range of - 75 to 75 degrees rotation of wrist along the axis normal to the table surface due to mechanical limitations of the gripper. Experimental results demonstrated the effectiveness of the proposed real-time object detection and manipulation strategy for fan robotic challenge.