Abstract
Human intent recognition is crucial for effective robotic teleoperation, yet existing methods rely on a predefined set of intentions, a requirement that is manually intractable in complex open-world scenarios. We propose an unsupervised framework that integrates Bayesian nonparametric learning with multi-modal Large Language Models (LLMs) to autonomously discover and interpret human intentions. We first employ a hierarchical Dirichlet process hidden semi-Markov model (HDP-HSMM) to adaptively segment kinematic sequences into motion primitives, reducing the load on downstream models. Each segment is represented by multi-modal features extracted from front-facing video, robot kinematics, and eye fixations. We then develop a reasoning architecture based on zero-shot Chain-of-Thought (CoT) prompting, where a Vision-Language Model (VLM) translates each segment into a semantic description, and an LLM clusters these descriptions to reveal intentions and generate human-readable interpretations. Experimental results on real-world data show that our method can uncover meaningful intentions highly consistent with expert judgments, outperforming baselines in intra-intent consistency by achieving the lowest semantic outlier rate of 8.5%.
| Original language | English |
|---|---|
| Pages (from-to) | 12512-12519 |
| Number of pages | 8 |
| Journal | IEEE Robotics and Automation Letters |
| Volume | 11 |
| Issue number | 11 |
| DOIs | |
| Publication status | Published - 1 Nov 2026 |
| Externally published | Yes |
Keywords
- Bayesian nonparametric learning
- Human intent discovery
- large language models
- robotic teleoperation
Fingerprint
Dive into the research topics of 'Unsupervised Interpretable Human Intent Discovery in Teleoperation via Bayesian Nonparametric Learning and Multi-Modal LLMs'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver