Efficient-Frequency: A hybrid visual forensic framework for facial forgery detection
View/ Open
File version
Accepted Manuscript (AM)
Author(s)
Chau, Xuan Truong Du
Hoang, Duong Le
Thanh Trung, Huynh
Pham, Minh Tam
Nguyen, Quoc Viet Hung
Jo, Jun
Nguyen, Tam
Year published
2020
Metadata
Show full item recordAbstract
The recent years have witnessed the significant development of visual forgery techniques and their malicious applications such as spreading of fake news and rumours, defamation or blackmailing of politicians and celebrities, manipulation of election result in political warfare. The manipulated contents have reached to such sophisticated level that human cannot tell apart whether a given content is real or fake. To deal with this serious threat, a rich body of visual forensic techniques has been proposed for detecting forged video and images. However, existing techniques either rely solely on engineered features or require a ...
View more >The recent years have witnessed the significant development of visual forgery techniques and their malicious applications such as spreading of fake news and rumours, defamation or blackmailing of politicians and celebrities, manipulation of election result in political warfare. The manipulated contents have reached to such sophisticated level that human cannot tell apart whether a given content is real or fake. To deal with this serious threat, a rich body of visual forensic techniques has been proposed for detecting forged video and images. However, existing techniques either rely solely on engineered features or require a complex deep learning model to extract the underlying patterns. In this paper, we propose a novel end-to-end visual forensic framework that can incorporate different modalities to efficiently classify real and forged contents. The model employs both the original content and its frequency domain analysis to fully exploit the richness of the image latent patterns. They are forwarded into two separated EfficientNet, a light yet efficient neural network architecture specialized for image classification, for pattern extraction. Then, we design a late-fusion mechanism to fuse the learnt features in original and frequency domain based on the importance of the underlying information. Our experimental results show that our proposed technique outperforms other state-of-the-art forensic approaches in many datasets and being robust to various visual forgery techniques.
View less >
View more >The recent years have witnessed the significant development of visual forgery techniques and their malicious applications such as spreading of fake news and rumours, defamation or blackmailing of politicians and celebrities, manipulation of election result in political warfare. The manipulated contents have reached to such sophisticated level that human cannot tell apart whether a given content is real or fake. To deal with this serious threat, a rich body of visual forensic techniques has been proposed for detecting forged video and images. However, existing techniques either rely solely on engineered features or require a complex deep learning model to extract the underlying patterns. In this paper, we propose a novel end-to-end visual forensic framework that can incorporate different modalities to efficiently classify real and forged contents. The model employs both the original content and its frequency domain analysis to fully exploit the richness of the image latent patterns. They are forwarded into two separated EfficientNet, a light yet efficient neural network architecture specialized for image classification, for pattern extraction. Then, we design a late-fusion mechanism to fuse the learnt features in original and frequency domain based on the importance of the underlying information. Our experimental results show that our proposed technique outperforms other state-of-the-art forensic approaches in many datasets and being robust to various visual forgery techniques.
View less >
Conference Title
2020 IEEE Symposium Series on Computational Intelligence (SSCI)
Copyright Statement
© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Subject
Artificial intelligence