CasFormer: Cascaded transformers for fusion-aware computational hyperspectral imaging
File version
Author(s)
Zhang, B
Hong, D
Zhou, J
Vivone, G
Li, S
Chanussot, J
Griffith University Author(s)
Primary Supervisor
Other Supervisors
Editor(s)
Date
Size
File type(s)
Location
License
Abstract
Computational hyperspectral imaging (CHI) is a cutting-edge technique, which plays a pivotal role in breaking through the quality bottleneck of hyperspectral images (HSI). Among the techniques employed in this domain, the coded aperture snapshot spectral imaging (CASSI) system holds widespread recognition. Nevertheless, the imaging capability of CASSI remains limited due to the hardware conditions and the fragility of outcomes associated with the ill-posed blind reconstruction process. To this end, we propose a novel cascaded transformer architecture, termed CasFormer, specifically crafted for fusion-aware CHI by means of a dual-imaging mechanism. CasFormer facilitates the effective enhancement of hyperspectral imaging quality by fusing RGB images, with a focus on spatial and spectral domains. As the name suggests, CasFormer is primarily composed of a series of cascade-attention blocks, enabling the fusion of high-spatial-resolution RGB images through spatial coherence alignment and the recovery of spectrally sequential information more compactly and accurately. Furthermore, CasFormer incorporates physical constraints through a decoupling-based loss function, ensuring spatial consistency and spectral fidelity in the fusion-aware CHI process. Extensive experiments conducted across multiple datasets demonstrate the superiority of CasFormer in achieving high-quality imaging results compared to SOTA CHI algorithms. Our code and benchmark datasets will be openly accessible at https://github.com/danfenghong/Information_Fusion_CasFormer.
Journal Title
Information Fusion
Conference Title
Book Title
Edition
Volume
108
Issue
Thesis Type
Degree Program
School
Publisher link
Patent number
Funder(s)
Grant identifier(s)
Rights Statement
Rights Statement
Item Access Status
Note
Access the data
Related item(s)
Subject
Artificial intelligence
Computer vision and multimedia computation
Data management and data science
Persistent link to this record
Citation
Li, C; Zhang, B; Hong, D; Zhou, J; Vivone, G; Li, S; Chanussot, J, CasFormer: Cascaded transformers for fusion-aware computational hyperspectral imaging, Information Fusion, 2024, 108, pp. 102408