An Effective 6DoF Grasp Detection Framework for Human-Robot Interaction With LLMs

6DoF grasp technology is a key technology for embodied robots. Although many researchers have proposed various 6DoF grasp methods based on the GraspNet-1B dataset, many of these methods often demand significant computational resources during training and yield only moderate grasp detection results. Moreover, these methods typically overlook the challenges of robot grasp in human-robot interaction scenarios. To address these issues, in this article, we propose an efficient robot grasp detection method called EffectGrasp. First, to solve the problem of excessive computational resource consumption during network training, we develop an efficient network training strategy that reduces computational overhead while ensuring the accuracy of grasp pose prediction. Then, to address the issue of the lack of language labels for objects in the GraspNet-1B dataset, we relabel the objects in the dataset with language annotations for completing robot grasp tasks in human-robot interaction scenarios. Next, we build an efficient grasp pose prediction network, EffectGrasp, which, combined with a large language model, can effectively understand human instructions and intentions to complete the grasp of target objects. Finally, we conduct various performance tests and grasp experiments on the GraspNet-1B dataset, simulated scenarios, and real-world scenarios. The experimental results show that EffectGrasp, using approximately 8.2 GB of memory during training, achieves grasp pose prediction results comparable to current SOTA methods. It also achieves a grasp success rate of 98% in simulated scenarios and 95% in real-world applications, making it suitable for human-robot interaction tasks in embodied robots.