Study on user-generated 3D gestures for video conferencing system with see-through head mounted display

Guangchuan Li; Yue Liu; Yongtian Wang; Rempel David

doi:10.1007/978-981-13-1702-6_60

Study on user-generated 3D gestures for video conferencing system with see-through head mounted display

Guangchuan Li, Yue Liu^*, Yongtian Wang, Rempel David

^*Corresponding author for this work

School of Optics and Photonics

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

As video conferencing systems transition to using head-mounted displays (HMD), non-contacting (3D) hand gestures are likely to replace conventional input devices by providing more efficient interactions with less cost. This paper presents the design of an experimental video conferencing system with optical see-through HMD, Leap Motion hand tracker, and RGB cameras. Both the skeleton-based dynamic hand gesture recognition and ergonomic-based gesture lexicon design were studied. The proposed gesture recognition algorithm fused hand shape and hand direction feature and used Temporal Pyramid to obtain a high dimension feature and predicted the gesture classification through linear SVM machine learning. Subjects (N = 16) self-generated different hand gestures for 25 different tasks related to video conferencing and object manipulation and rated gestures on ease of making the gesture, match to the command, and arm fatigue. Based on these outcomes, a gesture lexicon is proposed for controlling a video conferencing system and for manipulating virtual objects.

Original language	English
Title of host publication	Image and Graphics Technologies and Applications - 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018, Revised Selected Papers
Editors	Yongtian Wang, Yuxin Peng, Zhiguo Jiang
Publisher	Springer Verlag
Pages	605-615
Number of pages	11
ISBN (Print)	9789811317019
DOIs	https://doi.org/10.1007/978-981-13-1702-6_60
Publication status	Published - 2018
Event	13th Conference on Image and Graphics Technologies and Applications, IGTA 2018 - Beijing, China Duration: 8 Apr 2018 → 10 Apr 2018

Publication series

Name	Communications in Computer and Information Science
Volume	875
ISSN (Print)	1865-0929

Conference

Conference	13th Conference on Image and Graphics Technologies and Applications, IGTA 2018
Country/Territory	China
City	Beijing
Period	8/04/18 → 10/04/18

Keywords

3D gestures
Gesture recognition
Optical see-through HMD
Video conferencing system

Access to Document

10.1007/978-981-13-1702-6_60

Cite this

Li, G., Liu, Y., Wang, Y., & David, R. (2018). Study on user-generated 3D gestures for video conferencing system with see-through head mounted display. In Y. Wang, Y. Peng, & Z. Jiang (Eds.), Image and Graphics Technologies and Applications - 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018, Revised Selected Papers (pp. 605-615). (Communications in Computer and Information Science; Vol. 875). Springer Verlag. https://doi.org/10.1007/978-981-13-1702-6_60

Li, Guangchuan ; Liu, Yue ; Wang, Yongtian et al. / Study on user-generated 3D gestures for video conferencing system with see-through head mounted display. Image and Graphics Technologies and Applications - 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018, Revised Selected Papers. editor / Yongtian Wang ; Yuxin Peng ; Zhiguo Jiang. Springer Verlag, 2018. pp. 605-615 (Communications in Computer and Information Science).

@inproceedings{b1a2a06cd9c54012879ff80d8691ffd5,

title = "Study on user-generated 3D gestures for video conferencing system with see-through head mounted display",

abstract = "As video conferencing systems transition to using head-mounted displays (HMD), non-contacting (3D) hand gestures are likely to replace conventional input devices by providing more efficient interactions with less cost. This paper presents the design of an experimental video conferencing system with optical see-through HMD, Leap Motion hand tracker, and RGB cameras. Both the skeleton-based dynamic hand gesture recognition and ergonomic-based gesture lexicon design were studied. The proposed gesture recognition algorithm fused hand shape and hand direction feature and used Temporal Pyramid to obtain a high dimension feature and predicted the gesture classification through linear SVM machine learning. Subjects (N = 16) self-generated different hand gestures for 25 different tasks related to video conferencing and object manipulation and rated gestures on ease of making the gesture, match to the command, and arm fatigue. Based on these outcomes, a gesture lexicon is proposed for controlling a video conferencing system and for manipulating virtual objects.",

keywords = "3D gestures, Gesture recognition, Optical see-through HMD, Video conferencing system",

author = "Guangchuan Li and Yue Liu and Yongtian Wang and Rempel David",

note = "Publisher Copyright: {\textcopyright} Springer Nature Singapore Pte Ltd., 2018.; 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018 ; Conference date: 08-04-2018 Through 10-04-2018",

year = "2018",

doi = "10.1007/978-981-13-1702-6_60",

language = "English",

isbn = "9789811317019",

series = "Communications in Computer and Information Science",

publisher = "Springer Verlag",

pages = "605--615",

editor = "Yongtian Wang and Yuxin Peng and Zhiguo Jiang",

booktitle = "Image and Graphics Technologies and Applications - 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018, Revised Selected Papers",

address = "Germany",

}

Li, G, Liu, Y , Wang, Y & David, R 2018, Study on user-generated 3D gestures for video conferencing system with see-through head mounted display. in Y Wang, Y Peng & Z Jiang (eds), Image and Graphics Technologies and Applications - 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018, Revised Selected Papers. Communications in Computer and Information Science, vol. 875, Springer Verlag, pp. 605-615, 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018, Beijing, China, 8/04/18. https://doi.org/10.1007/978-981-13-1702-6_60

Study on user-generated 3D gestures for video conferencing system with see-through head mounted display. / Li, Guangchuan; Liu, Yue ; Wang, Yongtian et al.
Image and Graphics Technologies and Applications - 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018, Revised Selected Papers. ed. / Yongtian Wang; Yuxin Peng; Zhiguo Jiang. Springer Verlag, 2018. p. 605-615 (Communications in Computer and Information Science; Vol. 875).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Study on user-generated 3D gestures for video conferencing system with see-through head mounted display

AU - Li, Guangchuan

AU - Liu, Yue

AU - Wang, Yongtian

AU - David, Rempel

N1 - Publisher Copyright: © Springer Nature Singapore Pte Ltd., 2018.

PY - 2018

Y1 - 2018

N2 - As video conferencing systems transition to using head-mounted displays (HMD), non-contacting (3D) hand gestures are likely to replace conventional input devices by providing more efficient interactions with less cost. This paper presents the design of an experimental video conferencing system with optical see-through HMD, Leap Motion hand tracker, and RGB cameras. Both the skeleton-based dynamic hand gesture recognition and ergonomic-based gesture lexicon design were studied. The proposed gesture recognition algorithm fused hand shape and hand direction feature and used Temporal Pyramid to obtain a high dimension feature and predicted the gesture classification through linear SVM machine learning. Subjects (N = 16) self-generated different hand gestures for 25 different tasks related to video conferencing and object manipulation and rated gestures on ease of making the gesture, match to the command, and arm fatigue. Based on these outcomes, a gesture lexicon is proposed for controlling a video conferencing system and for manipulating virtual objects.

AB - As video conferencing systems transition to using head-mounted displays (HMD), non-contacting (3D) hand gestures are likely to replace conventional input devices by providing more efficient interactions with less cost. This paper presents the design of an experimental video conferencing system with optical see-through HMD, Leap Motion hand tracker, and RGB cameras. Both the skeleton-based dynamic hand gesture recognition and ergonomic-based gesture lexicon design were studied. The proposed gesture recognition algorithm fused hand shape and hand direction feature and used Temporal Pyramid to obtain a high dimension feature and predicted the gesture classification through linear SVM machine learning. Subjects (N = 16) self-generated different hand gestures for 25 different tasks related to video conferencing and object manipulation and rated gestures on ease of making the gesture, match to the command, and arm fatigue. Based on these outcomes, a gesture lexicon is proposed for controlling a video conferencing system and for manipulating virtual objects.

KW - 3D gestures

KW - Gesture recognition

KW - Optical see-through HMD

KW - Video conferencing system

UR - http://www.scopus.com/inward/record.url?scp=85052225982&partnerID=8YFLogxK

U2 - 10.1007/978-981-13-1702-6_60

DO - 10.1007/978-981-13-1702-6_60

M3 - Conference contribution

AN - SCOPUS:85052225982

SN - 9789811317019

T3 - Communications in Computer and Information Science

SP - 605

EP - 615

BT - Image and Graphics Technologies and Applications - 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018, Revised Selected Papers

A2 - Wang, Yongtian

A2 - Peng, Yuxin

A2 - Jiang, Zhiguo

PB - Springer Verlag

T2 - 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018

Y2 - 8 April 2018 through 10 April 2018

ER -

Li G, Liu Y , Wang Y, David R. Study on user-generated 3D gestures for video conferencing system with see-through head mounted display. In Wang Y, Peng Y, Jiang Z, editors, Image and Graphics Technologies and Applications - 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018, Revised Selected Papers. Springer Verlag. 2018. p. 605-615. (Communications in Computer and Information Science). doi: 10.1007/978-981-13-1702-6_60

Study on user-generated 3D gestures for video conferencing system with see-through head mounted display

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this