Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion

Yongkang Tang; Hai An Vu; Phuc Q. Le; Daisuke Masano; Oo Han Thet; Chastine Fatichah; Zhentao Liu; Masashi Yamaguchi; Martin Leonard Tangel; Fangyan Dong; Yoichi Yamazaki; Kaoru Hirota

doi:10.20965/jaciii.2011.p0563

Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion

Yongkang Tang^*, Hai An Vu, Phuc Q. Le, Daisuke Masano, Oo Han Thet, Chastine Fatichah, Zhentao Liu, Masashi Yamaguchi, Martin Leonard Tangel, Fangyan Dong, Yoichi Yamazaki, Kaoru Hirota

^*此作品的通讯作者

科研成果: 期刊稿件 › 文章 › 同行评审

6 引用（Scopus）

摘要

A multimodal gesture recognition method for mascot robot system is proposed based on Choquet integral by fusing camera and 3D accelerometer data. The optimization of two fuzzy measures in the training phase for two recognition units, i.e., camera-based and accelerometer-based units, obtains enough recognition rate of 92.7% for 8 types of gestures by improving the recognition rate approximate 20% compared with that of each unit. The proposed method targets casual communication from humans to robots by integrating nonverbal gesture messages and verbal messages.

源语言	英语
页（从-至）	563-572
页数	10
期刊	Journal of Advanced Computational Intelligence and Intelligent Informatics
卷	15
期	5
DOI	https://doi.org/10.20965/jaciii.2011.p0563
出版状态	已出版 - 7月 2011
已对外发布	是

访问文件

10.20965/jaciii.2011.p0563

其它文件与链接

链接到 Scopus 的出版物

引用此

Tang, Y., Vu, H. A., Le, P. Q., Masano, D., Thet, O. H., Fatichah, C., Liu, Z., Yamaguchi, M., Tangel, M. L., Dong, F., Yamazaki, Y., & Hirota, K. (2011). Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion. Journal of Advanced Computational Intelligence and Intelligent Informatics, 15(5), 563-572. https://doi.org/10.20965/jaciii.2011.p0563

@article{5edfe2c4f5ee4c1da070d67de11a3db9,

title = "Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion",

abstract = "A multimodal gesture recognition method for mascot robot system is proposed based on Choquet integral by fusing camera and 3D accelerometer data. The optimization of two fuzzy measures in the training phase for two recognition units, i.e., camera-based and accelerometer-based units, obtains enough recognition rate of 92.7% for 8 types of gestures by improving the recognition rate approximate 20% compared with that of each unit. The proposed method targets casual communication from humans to robots by integrating nonverbal gesture messages and verbal messages.",

keywords = "3D accelerometer, Choquet integral, Gesture recognition, Human-robot interaction, Sensor fusion",

author = "Yongkang Tang and Vu, {Hai An} and Le, {Phuc Q.} and Daisuke Masano and Thet, {Oo Han} and Chastine Fatichah and Zhentao Liu and Masashi Yamaguchi and Tangel, {Martin Leonard} and Fangyan Dong and Yoichi Yamazaki and Kaoru Hirota",

year = "2011",

month = jul,

doi = "10.20965/jaciii.2011.p0563",

language = "English",

volume = "15",

pages = "563--572",

journal = "Journal of Advanced Computational Intelligence and Intelligent Informatics",

issn = "1343-0130",

publisher = "Fuji Technology Press",

number = "5",

}

Tang, Y, Vu, HA, Le, PQ, Masano, D, Thet, OH, Fatichah, C, Liu, Z, Yamaguchi, M, Tangel, ML, Dong, F, Yamazaki, Y & Hirota, K 2011, 'Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion', Journal of Advanced Computational Intelligence and Intelligent Informatics, 卷 15, 号码 5, 页码 563-572. https://doi.org/10.20965/jaciii.2011.p0563

TY - JOUR

T1 - Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion

AU - Tang, Yongkang

AU - Vu, Hai An

AU - Le, Phuc Q.

AU - Masano, Daisuke

AU - Thet, Oo Han

AU - Fatichah, Chastine

AU - Liu, Zhentao

AU - Yamaguchi, Masashi

AU - Tangel, Martin Leonard

AU - Dong, Fangyan

AU - Yamazaki, Yoichi

AU - Hirota, Kaoru

PY - 2011/7

Y1 - 2011/7

N2 - A multimodal gesture recognition method for mascot robot system is proposed based on Choquet integral by fusing camera and 3D accelerometer data. The optimization of two fuzzy measures in the training phase for two recognition units, i.e., camera-based and accelerometer-based units, obtains enough recognition rate of 92.7% for 8 types of gestures by improving the recognition rate approximate 20% compared with that of each unit. The proposed method targets casual communication from humans to robots by integrating nonverbal gesture messages and verbal messages.

AB - A multimodal gesture recognition method for mascot robot system is proposed based on Choquet integral by fusing camera and 3D accelerometer data. The optimization of two fuzzy measures in the training phase for two recognition units, i.e., camera-based and accelerometer-based units, obtains enough recognition rate of 92.7% for 8 types of gestures by improving the recognition rate approximate 20% compared with that of each unit. The proposed method targets casual communication from humans to robots by integrating nonverbal gesture messages and verbal messages.

KW - 3D accelerometer

KW - Choquet integral

KW - Gesture recognition

KW - Human-robot interaction

KW - Sensor fusion

UR - http://www.scopus.com/inward/record.url?scp=79960493103&partnerID=8YFLogxK

U2 - 10.20965/jaciii.2011.p0563

DO - 10.20965/jaciii.2011.p0563

M3 - Article

AN - SCOPUS:79960493103

SN - 1343-0130

VL - 15

SP - 563

EP - 572

JO - Journal of Advanced Computational Intelligence and Intelligent Informatics

JF - Journal of Advanced Computational Intelligence and Intelligent Informatics

IS - 5

ER -

Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion

摘要

访问文件

其它文件与链接

指纹

引用此