An Automatic Lipreading System for Spoken Digits With Limited Training Data

Loading...
Thumbnail Image
File version
Author(s)
Wang, SL
Liew, AWC
Lau, WH
Leung, SH
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Keshab K Parhi
Date
2008
Size
215155 bytes
File type(s)
application/pdf
Location
License
Abstract

It is well known that visual cues of lip movement contain important speech relevant information. This paper presents an automatic lipreading system for small vocabulary speech recognition tasks. Using the lip segmentation and modeling techniques we developed earlier, we obtain a visual feature vector composed of outer and inner mouth features from the lip image sequence for recognition. A spline representation is employed to transform the discrete-time sampled features from the video frames into the continuous domain. The spline coefficients in the same word class are constrained to have similar expression and are estimated from the training data by the EM algorithm. For the multiple-speaker/speaker-independent recognition task, an adaptive multimodel approach is proposed to handle the variations caused by various talking styles. After building the appropriate word models from the spline coefficients, a maximum likelihood classification approach is taken for the recognition. Lip image sequences of English digits from 0 to 9 have been collected for the recognition test. Two widely used classification methods, HMM and RDA, have been adopted for comparison and the results demonstrate that the proposed algorithm deliver the best performance among these methods.

Journal Title
I E E E Transactions on Circuits and Systems for Video Technology
Conference Title
Book Title
Edition
Volume
18
Issue
12
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
© 2008 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Item Access Status
Note
Access the data
Related item(s)
Subject
Computer vision
Persistent link to this record
Citation
Collections