TY - JOUR
T1 - LLM-guided fuzzy kinematic modeling for resolving kinematic uncertainties and linguistic ambiguities in text-to-motion generation
AU - Manjotho, Ali Asghar
AU - Tewolde, Tekie Tsegay
AU - Duma, Ramadhani Ally
AU - Niu, Zhendong
N1 - Publisher Copyright:
© 2025 Elsevier Ltd
PY - 2025/6/15
Y1 - 2025/6/15
N2 - Generating realistic and coherent human motions from text descriptions is essential for applications in computer vision, computer animations, and digital environments. However, existing text-to-motion generation models often overlook kinematic uncertainties and linguistic ambiguities, leading to unnatural and misaligned motion sequences. To address these issues, we propose a novel framework that integrates fuzzy kinematic modeling with large language model (LLM) guidance to jointly model kinematic uncertainties and resolve linguistic ambiguities. Our approach first extracts rich kinematic attributes from raw motion data and converts them into fuzzy kinematic facts (FKFs), which serve as an uncertainty-aware motion representation across different kinematic hierarchies. Simultaneously, we refine ambiguous text descriptions by extracting contextual terms using LLM-guided few-shot in-context learning, enhancing text with additional semantic clarity. These FKFs and contextual terms are then used to train a diffusion-based motion generation model, ensuring semantically accurate and physically plausible motion synthesis. To further enhance kinematic structural consistency in FKF representations, we introduce a Graph-Augmented Self-Attention (GASA) module, which injects spatio-temporal relational constraints into the diffusion process, improving motion coherence and structural integrity. Evaluations on HumanML3D and KIT-ML datasets demonstrate that our method outperforms state-of-the-art models, achieving the lowest FID scores (0.052 and 0.091) and reducing kinematic uncertainty footprint by 21.1% and 17.7%, respectively. The source code and additional resources are publicly available at https://alimanjotho.github.io/llm-fqk-t2m.
AB - Generating realistic and coherent human motions from text descriptions is essential for applications in computer vision, computer animations, and digital environments. However, existing text-to-motion generation models often overlook kinematic uncertainties and linguistic ambiguities, leading to unnatural and misaligned motion sequences. To address these issues, we propose a novel framework that integrates fuzzy kinematic modeling with large language model (LLM) guidance to jointly model kinematic uncertainties and resolve linguistic ambiguities. Our approach first extracts rich kinematic attributes from raw motion data and converts them into fuzzy kinematic facts (FKFs), which serve as an uncertainty-aware motion representation across different kinematic hierarchies. Simultaneously, we refine ambiguous text descriptions by extracting contextual terms using LLM-guided few-shot in-context learning, enhancing text with additional semantic clarity. These FKFs and contextual terms are then used to train a diffusion-based motion generation model, ensuring semantically accurate and physically plausible motion synthesis. To further enhance kinematic structural consistency in FKF representations, we introduce a Graph-Augmented Self-Attention (GASA) module, which injects spatio-temporal relational constraints into the diffusion process, improving motion coherence and structural integrity. Evaluations on HumanML3D and KIT-ML datasets demonstrate that our method outperforms state-of-the-art models, achieving the lowest FID scores (0.052 and 0.091) and reducing kinematic uncertainty footprint by 21.1% and 17.7%, respectively. The source code and additional resources are publicly available at https://alimanjotho.github.io/llm-fqk-t2m.
KW - Fuzzy qualitative kinematics
KW - Kinematic uncertainties
KW - Large language models
KW - Linguistic ambiguities
KW - Text-to-motion generation
UR - http://www.scopus.com/inward/record.url?scp=105001691670&partnerID=8YFLogxK
U2 - 10.1016/j.eswa.2025.127283
DO - 10.1016/j.eswa.2025.127283
M3 - Article
AN - SCOPUS:105001691670
SN - 0957-4174
VL - 279
JO - Expert Systems with Applications
JF - Expert Systems with Applications
M1 - 127283
ER -