Graph Convolutional Incomplete Multi-modal Hashing
File version
Author(s)
Chen, Yinfan
Pan, Shirui
Liu, Weiwei
Zheng, Yuhui
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
Size
File type(s)
Location
Ottawa, Canada
License
Abstract
Multi-modal hashing (MMH) encodes multi-modal data into latent hash code, and has been widely applied for efficient large-scale multi-modal retrieval. In practice it is common that multi-modal data is often corrupted with missing modalities, e.g., social image often lacks its tags in image-text retrieval. Conventional MMHs can only learn on complete modalities, which however wastes a considerable amount of collected data. To fulfill this gap, this paper proposes Graph Convolutional Incomplete Multi-modal Hashing (GCIMH) to learn hash code on incomplete multi-modal data. GCIMH develops Graph Convolutional Autoencoder to reconstruct incomplete multi-modal data with effective exploit of its semantic structure. GCIMH further develops multi-modal and label networks to encode multiple modalities and label respectively. GCIMH can successfully transfer knowledge of autoencoder and label network to multi-modal hashing network using teacher-student learning framework. GCIMH can handle missing modalities in both offline training and online query stages. Extensive empirical studies on three benchmark datasets demonstrate the superiority of the proposed GCIMH over the state-of-the-arts on both complete and incomplete multi-modal retrieval.
Journal Title
Conference Title
MM '23: Proceedings of the 31st ACM International Conference on Multimedia
Book Title
Edition
Volume
Issue
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
Item Access Status
Note
Access the data
Related item(s)
Subject
Data management and data science
Database systems
Persistent link to this record
Citation
Shen, X; Chen, Y; Pan, S; Liu, W; Zheng, Y, Graph Convolutional Incomplete Multi-modal Hashing, MM '23: Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 7029-7037