Using Viewer's Facial Expression and Heart Rate for Sports Video Highlights Detection
File version
Accepted Manuscript (AM)
Author(s)
Chakraborty, Prithwi Raj
Tjondronegoro, Dian
Zhang, Ligang
Chandran, Vinod
Griffith University Author(s)
Year published
2015
Metadata
Show full item recordAbstract
Viewer interest, evoked by video content, can potentially identify the highlights of the video. This paper explores the use of facial expressions (FE) and heart rate (HR) of viewers captured using camera and non-strapped sensor for identifying interesting video segments. The data from ten subjects with three videos showed that these signals are viewer dependent and not synchronized with the video contents. To address this issue, new algorithms are proposed to effectively combine FE and HR signals for identifying the time when viewer interest is potentially high. The results show that, compared with subjective annotation and ...
View more >Viewer interest, evoked by video content, can potentially identify the highlights of the video. This paper explores the use of facial expressions (FE) and heart rate (HR) of viewers captured using camera and non-strapped sensor for identifying interesting video segments. The data from ten subjects with three videos showed that these signals are viewer dependent and not synchronized with the video contents. To address this issue, new algorithms are proposed to effectively combine FE and HR signals for identifying the time when viewer interest is potentially high. The results show that, compared with subjective annotation and match report highlights, 'non-neutral' FE and 'relatively higher and faster' HR is able to capture 60%-80% of goal, foul, and shot-ongoal soccer video events. FE is found to be more indicative than HR of viewer interest, but the fusion of these two modalities outperforms each of them.
View less >
View more >Viewer interest, evoked by video content, can potentially identify the highlights of the video. This paper explores the use of facial expressions (FE) and heart rate (HR) of viewers captured using camera and non-strapped sensor for identifying interesting video segments. The data from ten subjects with three videos showed that these signals are viewer dependent and not synchronized with the video contents. To address this issue, new algorithms are proposed to effectively combine FE and HR signals for identifying the time when viewer interest is potentially high. The results show that, compared with subjective annotation and match report highlights, 'non-neutral' FE and 'relatively higher and faster' HR is able to capture 60%-80% of goal, foul, and shot-ongoal soccer video events. FE is found to be more indicative than HR of viewer interest, but the fusion of these two modalities outperforms each of them.
View less >
Conference Title
ICMR '15: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval
Copyright Statement
© ACM, 2015. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ICMR '15: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, ISBN: 9781450332743, https://doi.org/10.1145/2671188.2749361
Subject
Artificial Intelligence and Image Processing
Science & Technology
Computer Science, Artificial Intelligence
Computer Science, Information Systems
Imaging Science & Photographic Technology