Lip Image Segmentation in Mobile Devices Based on Alternative Knowledge Distillation

Abstract

Lip image segmentation, as the first step in many lip-related tasks (e.g. automatic lipreading), is of vital significance for the subsequent procedures. Nowadays, with the increasing computational power of the mobile devices, mobile applications become more and more popular. In this paper, a new approach is proposed, which is able to segment the lip region in natural scenes and is of acceptable computational complexity to be implemented in mobile devices. Two networks including a complex teacher network and a compact student network with the same structure are employed. With the proposed remedy loss and the alternative knowledge distillation scheme, the student network can learn useful knowledge from the teacher network effectively and efficiently, and even rectify some of its segmentation errors. A dataset containing 49 people captured under natural scenes by various cellphone cameras is adopted for evaluation and the experiment results have demonstrated that the proposed student network even outperforms the teacher network with much less computational cost.