Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion

Yongkang Tang; Hai An Vu; Phuc Q. Le; Daisuke Masano; Oo Han Thet; Chastine Fatichah; Zhentao Liu; Masashi Yamaguchi; Martin Leonard Tangel; Fangyan Dong; Yoichi Yamazaki; Kaoru Hirota

doi:10.20965/jaciii.2011.p0563

Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion

Yongkang Tang^*, Hai An Vu, Phuc Q. Le, Daisuke Masano, Oo Han Thet, Chastine Fatichah, Zhentao Liu, Masashi Yamaguchi, Martin Leonard Tangel, Fangyan Dong, Yoichi Yamazaki, Kaoru Hirota

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

6 Citations (Scopus)

Abstract

A multimodal gesture recognition method for mascot robot system is proposed based on Choquet integral by fusing camera and 3D accelerometer data. The optimization of two fuzzy measures in the training phase for two recognition units, i.e., camera-based and accelerometer-based units, obtains enough recognition rate of 92.7% for 8 types of gestures by improving the recognition rate approximate 20% compared with that of each unit. The proposed method targets casual communication from humans to robots by integrating nonverbal gesture messages and verbal messages.

Original language	English
Pages (from-to)	563-572
Number of pages	10
Journal	Journal of Advanced Computational Intelligence and Intelligent Informatics
Volume	15
Issue number	5
DOIs	https://doi.org/10.20965/jaciii.2011.p0563
Publication status	Published - Jul 2011
Externally published	Yes

Keywords

3D accelerometer
Choquet integral
Gesture recognition
Human-robot interaction
Sensor fusion

Access to Document

10.20965/jaciii.2011.p0563

Cite this

Tang, Y., Vu, H. A., Le, P. Q., Masano, D., Thet, O. H., Fatichah, C., Liu, Z., Yamaguchi, M., Tangel, M. L., Dong, F., Yamazaki, Y., & Hirota, K. (2011). Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion. Journal of Advanced Computational Intelligence and Intelligent Informatics, 15(5), 563-572. https://doi.org/10.20965/jaciii.2011.p0563

@article{5edfe2c4f5ee4c1da070d67de11a3db9,

title = "Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion",

abstract = "A multimodal gesture recognition method for mascot robot system is proposed based on Choquet integral by fusing camera and 3D accelerometer data. The optimization of two fuzzy measures in the training phase for two recognition units, i.e., camera-based and accelerometer-based units, obtains enough recognition rate of 92.7% for 8 types of gestures by improving the recognition rate approximate 20% compared with that of each unit. The proposed method targets casual communication from humans to robots by integrating nonverbal gesture messages and verbal messages.",

keywords = "3D accelerometer, Choquet integral, Gesture recognition, Human-robot interaction, Sensor fusion",

author = "Yongkang Tang and Vu, {Hai An} and Le, {Phuc Q.} and Daisuke Masano and Thet, {Oo Han} and Chastine Fatichah and Zhentao Liu and Masashi Yamaguchi and Tangel, {Martin Leonard} and Fangyan Dong and Yoichi Yamazaki and Kaoru Hirota",

year = "2011",

month = jul,

doi = "10.20965/jaciii.2011.p0563",

language = "English",

volume = "15",

pages = "563--572",

journal = "Journal of Advanced Computational Intelligence and Intelligent Informatics",

issn = "1343-0130",

publisher = "Fuji Technology Press",

number = "5",

}

Tang, Y, Vu, HA, Le, PQ, Masano, D, Thet, OH, Fatichah, C, Liu, Z, Yamaguchi, M, Tangel, ML, Dong, F, Yamazaki, Y & Hirota, K 2011, 'Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion', Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 15, no. 5, pp. 563-572. https://doi.org/10.20965/jaciii.2011.p0563

TY - JOUR

T1 - Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion

AU - Tang, Yongkang

AU - Vu, Hai An

AU - Le, Phuc Q.

AU - Masano, Daisuke

AU - Thet, Oo Han

AU - Fatichah, Chastine

AU - Liu, Zhentao

AU - Yamaguchi, Masashi

AU - Tangel, Martin Leonard

AU - Dong, Fangyan

AU - Yamazaki, Yoichi

AU - Hirota, Kaoru

PY - 2011/7

Y1 - 2011/7

N2 - A multimodal gesture recognition method for mascot robot system is proposed based on Choquet integral by fusing camera and 3D accelerometer data. The optimization of two fuzzy measures in the training phase for two recognition units, i.e., camera-based and accelerometer-based units, obtains enough recognition rate of 92.7% for 8 types of gestures by improving the recognition rate approximate 20% compared with that of each unit. The proposed method targets casual communication from humans to robots by integrating nonverbal gesture messages and verbal messages.

AB - A multimodal gesture recognition method for mascot robot system is proposed based on Choquet integral by fusing camera and 3D accelerometer data. The optimization of two fuzzy measures in the training phase for two recognition units, i.e., camera-based and accelerometer-based units, obtains enough recognition rate of 92.7% for 8 types of gestures by improving the recognition rate approximate 20% compared with that of each unit. The proposed method targets casual communication from humans to robots by integrating nonverbal gesture messages and verbal messages.

KW - 3D accelerometer

KW - Choquet integral

KW - Gesture recognition

KW - Human-robot interaction

KW - Sensor fusion

UR - http://www.scopus.com/inward/record.url?scp=79960493103&partnerID=8YFLogxK

U2 - 10.20965/jaciii.2011.p0563

DO - 10.20965/jaciii.2011.p0563

M3 - Article

AN - SCOPUS:79960493103

SN - 1343-0130

VL - 15

SP - 563

EP - 572

JO - Journal of Advanced Computational Intelligence and Intelligent Informatics

JF - Journal of Advanced Computational Intelligence and Intelligent Informatics

IS - 5

ER -

Multimodal gesture recognition for mascot robot system based on Choquet integral using camera and 3D accelerometers fusion

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this