MDAM3: A Misinformation Detection and Analysis Framework for Multitype Multimodal Media

No Thumbnail Available
File version
Author(s)
Xu, Q
Du, H
Łukasik, S
Zhu, T
Wang, S
Yu, X
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
2025
Size
File type(s)
Location

Sydney, Australia

License
Abstract

Misinformation is a significant societal issue with potentially severe consequences. It appears in text, image, audio, and video modalities, encompassing various categories such as unimodal deception (fact-conflicting, AI-generated & offensive content) and cross-modal inconsistencies. However, current detection approaches often focus on text and image, overlooking the growing prevalence of misinformation in audio and video content. Moreover, these methods typically tend to address only one or two types of misinformation, failing to address all categories simultaneously. These detectors are also usually designed to make judgments without providing explanations, reducing transparency and limiting their broader applicability. To address these issues, we propose MDAM3, a Misinformation Detection and Analysis Framework for Multitype Multimodal Media. MDAM3 analyzes each input in internal detection and examines relationships across modalities to identify inconsistencies. It utilizes web resources and integrates Large Vision-Language Models (LVLMs) to deliver accurate detection results along with detailed analysis. To evaluate MDAM3, we curate MDAM3-DB, a specialized multitype multimodal misinformation dataset. A user study is conducted to explore MDAM3’s usability, interpretability, and effectiveness. We hope this research contributes to advancing misinformation detection methodologies and provides valuable insights for developing robust multimodal analysis tools.

Journal Title
Conference Title

WWW '25: Proceedings of the ACM on Web Conference 2025

Book Title
Edition
Volume
Issue
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
Item Access Status
Note
Access the data
Related item(s)
Subject
Persistent link to this record
Citation

Xu, Q; Du, H; Łukasik, S; Zhu, T; Wang, S; Yu, X, MDAM3: A Misinformation Detection and Analysis Framework for Multitype Multimodal Media, WWW '25: Proceedings of the ACM on Web Conference 2025, 2025, pp. 5285-5296