跳到主要导航 跳到搜索 跳到主要内容

Tongue-Jaw Movement Recognition Through Acoustic Sensing on Smartphones

  • Yetong Cao
  • , Fan Li*
  • , Huijie Chen
  • , Xiaochen Liu
  • , Yu Wang
  • *此作品的通讯作者
  • Beijing Institute of Technology
  • Beijing University of Technology
  • Temple University

科研成果: 期刊稿件 › 文章 › 同行评审

摘要

Past tongue-jaw movement interaction systems typically require dedicated hardware and are uncomfortable to use, limiting their scalability and generalizability. This paper introduces CanalScan, the first system that recognizes tongue-jaw movements using commodity speakers and microphones mounted on ubiquitous off-the-shelf devices (e.g., smartphones). What inspires us is that tongue-jaw movements always cause ear canal deformations, and we find that for different tongue-jaw movements, dynamic features of ear canal deformations present unique patterns on acoustic reflections in the ear canal. Specifically, CanalScan first sends an acoustic signal to the ear canal, then parses the reflection signals for tongue-jaw movements recognition. To eliminate the impacts of body movements, we develop a body movement noise filtering method and a dynamic segmentation method to identify and separate the tongue-jaw movements-associated ear canal deformations from other types of body movements. We further propose a sensor position detection method and a data transformation mechanism to reduce the impacts of diversities in-ear canal shapes and relative positions between sensors and the ear canal. CanalScan explores twelve unique and consistent features and applies a random forest classifier to distinguish tongue-jaw movements. Extensive experiments with twenty participants validate the generalizability, effectiveness, robustness, and high accuracy of CanalScan.

源语言英语
页(从-至)879-894
页数16
期刊IEEE Transactions on Mobile Computing
卷23
期1
DOI
出版状态已出版 - 1 1月 2024

学术指纹

探究 'Tongue-Jaw Movement Recognition Through Acoustic Sensing on Smartphones' 的科研主题。它们共同构成独一无二的学术指纹。

引用此