Abstract—In this paper, we propose a lip reading system for
patients who have inability to talk during mechanical ventilation.
Existing automatic lip reading system cannot be used for ICU
patient, because their endotracheal tube causes visual occlusion.
We are aiming to read patient's lip by using mouth model that is
attached endotracheal tube for this problem. In this paper, we
defined two models of mouth shape. One model represents
mouth of ordinary face with a set of keypoints extracted by
facial keypoints detector. The other model represents mouth of
ICU patient. In addition, we compared two models in terms of
sequential labeling of visemes and isolated word recognition.
Index Terms—Automatic lip reading, visual speech
recognition, hidden Markov model, viseme.
Yudai Nagano, Ryuhei Sakurai, Hirotake Yamazoe, and Joo-Ho Lee are
with Ritsumeikan University, Japan (e-mail: is0081hx@ed.ritsumei.ac.jp,
rsakurai@fc.ritsumei.ac.jp, yamazoe@fc.ritsumei.ac.jp).
Yu Kawazoe is with Tohoku University, Japan (e-mail:
ukz411@gmail.com).
Kyohei Miyamoto is with Wakayama Medical University, Japan (e-mail:
go.go.kyohei.miyamoto@gmail.com).
[PDF]
Cite: Yudai Nagano, Ryuhei Sakurai, Yu Kawazoe, Kyohei Miyamoto, Hirotake Yamazoe, and Joo-Ho Lee, "Automatic Lip Reading for Inability-to-Talk Patient during Mechanical Ventilation," International Journal of Knowledge Engineering vol. 2, no. 3, pp. 137-141, 2016.
