Fusion of Hyperspectral and Multispectral Images Accounting for Localized Inter-image Changes

Abstract

The high spectral resolution of hyperspectral images (HSIs) generally comes at the expense of low spatial resolution, which hinders the application of HSIs. Fusing an HSI and a multispectral image (MSI) from different sensors to get an image with high spatial and spectral resolution is an economic and effective approach, but localized spatial and spectral changes between images acquired at different time instants can have negative impacts on the fusion results, which has rarely been considered in many fusion methods. In this paper, we propose a novel group sparsity constrained fusion method to fuse hyperspectral and multispectral images based on the matrix factorization. Specifically, we imposed ℓ2,1 norm on the residual term of the MSI to account for the localized inter-image changes occurring during the acquisition of the hyperspectral and multispectral images. Further, by exploiting the plug-and-play framework, we plugged a state-of-the-art denoiser, namely BM3D, as the prior of the subspace coefficients. We refer to the proposed fusion method as group sparsity constrained fusion method (GSFus). We performed fusion experiments on two kinds of datasets, i.e. with and without obvious localized changes between the HSIs and MSIs, and a full resolution data set. Extensive experiments in comparison with seven state-of-the-art fusion methods suggest that the proposed fusion method is more effective on fusing hyperspectral and multispectral images than the competitors.