Visual speech recognition using convolutional VEF snake and canonical correlations

Kun Lu*, Yuwei Wu, Yunde Jia

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Citations (Scopus)

Abstract

This paper presents a novel approach for automatic visual speech recognition using Convolutional VEF snake and canonical correlations. The utterance image sequences of isolated Chinese words are recorded with a head-mounted camera, and we use Convolutional VEF snake model to detect and track lip boundary rapidly and accurately. Geometric and motion features are both extracted from lip contour sequences and concatenated to form a joint feature descriptor. Canonical correlation is applied to measure the similarity of two utterance feature matrices and a linear discriminant function is introduced to make further improvement on the recognition accuracy. Experimental results demonstrate that our approach is promising and the joint feature descriptor is more robust than individual ones.

Original languageEnglish
Title of host publicationProceedings - 2010 IEEE Youth Conference on Information, Computing and Telecommunications, YC-ICT 2010
Pages154-157
Number of pages4
DOIs
Publication statusPublished - 2010
Event2010 IEEE Youth Conference on Information, Computing and Telecommunications, YC-ICT 2010 - Beijing, China
Duration: 28 Nov 201030 Nov 2010

Publication series

NameProceedings - 2010 IEEE Youth Conference on Information, Computing and Telecommunications, YC-ICT 2010

Conference

Conference2010 IEEE Youth Conference on Information, Computing and Telecommunications, YC-ICT 2010
Country/TerritoryChina
CityBeijing
Period28/11/1030/11/10

Keywords

  • Canonical correlation
  • Head-camera
  • Joint feature descriptor
  • Snake model
  • Visual speech recognition

Fingerprint

Dive into the research topics of 'Visual speech recognition using convolutional VEF snake and canonical correlations'. Together they form a unique fingerprint.

Cite this