A Large Language Model-Driven Natural Language Instructions Control System for Robotic Arm

Industry 5.0's human-centric model shifts intelligent manufacturing from robot-driven labor substitution to human-robot collaboration. However, traditional robotic arm systems face challenges in dynamic environments due to rigid programming and limited semantic understanding, which hinders efficient collaboration. Therefore, this paper proposes a natural language control system for robotic arms based on a large language model to achieve flexible and safe human-robot collaboration in assembly scenarios. First, based on the operational characteristics of the assembly scene, the complex assembly task is deconstructed into basic action units; at the same time, the mapping mechanism from natural language instructions to robotic arm control functions is established on the basis of the DeepSeekR1-Distill-Llama-8B large language model, and integrates LoRA parameter fine-tuning technology. Second, YOLOv8 model and transfer learning technology are used to accurately identify the target object, combined with binocular camera depth vision to obtain the target position information. Finally, the effectiveness of the proposed method is verified with a case study of tools passing tasks in an assembly scene.